Optical fiber scanning projection device

Abstract

The utility model discloses an optical fiber scanning projection device, which comprises a light source module and a light scanning module, and is characterized in that light emitted by the light source module is scanned and output by the light scanning module to serve as display image light; the light scanning module comprises an actuator, the emergent end of an optical fiber in the light source module is fixed on the actuator, the optical fiber exceeds the actuator and forms an optical fiber cantilever, and the optical fiber cantilever is driven by the actuator to scan in the space; the light source module at least comprises a group of light sources, and the light sources at least comprise R, G and B light-emitting units; the optical fiber scanning projection device further comprises an infrared sensor which is configured to receive infrared information reflected by the projection area of the optical fiber scanning projection device. According to the optical fiber scanning projection device provided by the embodiment of the invention, the infrared sensor and the infrared light-emitting unit are additionally arranged, so that objects in a projection area can be scanned and collected, and the functions of scanning detection and projection are realized at the same time.

Description

Technical Field

[0001] The present application relates to the field of projection imaging, and in particular to a processing method, system and storage medium based on three-dimensional point cloud of a fiber optic scanning projection device. Background Art

[0002] The imaging principle of fiber optic scanning imaging technology is to modulate the light corresponding to each pixel of the image to be displayed through a light source, and scan the light corresponding to each pixel on the surface of the projection carrier through an optical fiber. Considering that the surface of the projection carrier is an irregular surface (such as a curved surface), the final projection will be greatly distorted. Although the current HUD head-up display technology also involves the problem of projecting onto curved surfaces, it usually uses pre-adjustment of the image signal to fit the curved surface, rather than real-time feedback adjustment. It is limited to projection at specific positions on the car window and has low applicability. Utility Model Content

[0003] The object of the present invention is to provide a fiber optic scanning and projection device which has both scanning detection and projection functions, has high applicability, and can perform real-time feedback adjustment on image signals according to the surface of a projection carrier.

[0004] In order to achieve the above-mentioned purpose of the invention, the first aspect of the embodiment of the present invention provides a fiber optic scanning projection device, comprising: a light source module and a light scanning module, the light emitted by the light source module is scanned and output by the light scanning module as display image light; the light scanning module comprises an actuator, the optical fiber output end in the light source module is fixed on the actuator, the optical fiber extends beyond the actuator and forms a fiber optic cantilever, and the fiber optic cantilever is driven by the actuator to sweep in space; the light source module comprises at least one group of light sources, each group of light sources comprises at least three light emitting units of R, G, and B; the fiber optic scanning projection device also comprises an infrared sensor, which is configured to receive infrared information reflected by the projection area of ​​the fiber optic scanning projection device.

[0005] In some embodiments of the present application, each group of light sources of the light source module further includes an infrared light emitting unit for emitting infrared light, and the infrared light emitting unit is configured to output infrared light to a projection area of ​​the optical fiber scanning projection device.

[0006] In some embodiments of the present application, the infrared light emitting unit includes a first infrared light emitting unit and a second infrared light emitting unit, and the infrared light wavelengths generated by the first infrared light emitting unit and the second infrared light emitting unit are different.

[0007] In the embodiment of the present application, the infrared light emitting unit may include at least one infrared light source such as an infrared semiconductor laser, an infrared LED, etc.

[0008] In some embodiments of the present application, the infrared sensor is disposed on a peripheral side of the actuator.

[0009] In some embodiments of the present application, the optical scanning module includes a packaging shell, which packages the actuator and the imaging module, and the infrared sensor is located on the inner side or the outer side of the packaging shell.

[0010] An embodiment of the present application also provides a projection scanning device, including a projection device and an infrared sensor, wherein the infrared sensor is configured to receive infrared information reflected by a projection area of ​​the projection scanning device; the projection device includes any one or more of a cathode ray tube device, an LCD projection device, a digital light processing projection device, and a liquid crystal on silicon projection device.

[0011] Compared with the prior art, the present invention has the following beneficial effects:

[0012] In the solution provided in the embodiment of the present invention, while performing projection display, the infrared light-emitting unit and the infrared sensor can scan and collect the surface of objects in the projection area, which can effectively realize the integration and miniaturization of the projection display element and the scanning and collection element, and can construct a three-dimensional model of the surface of objects in the projection area. The processor of the fiber optic scanning projection device can provide real-time feedback and adjust the signal of the image to be projected according to the three-dimensional model, so as to improve the applicability of the fiber optic scanning projection device in the application scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor:

[0014] Figure 1A-1B Schematic diagram of the structure of the optical fiber scanning projection device in the embodiment of the present application;

[0015] Figure 2 A schematic diagram of the structure of a light source module provided in an embodiment of the present application;

[0016] Figure 3 A schematic diagram of the structure of the optical scanning module provided in an embodiment of the present application;

[0017] Figure 4 An exemplary flow chart of a three-dimensional point cloud-based projection method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0019] The fiber scanning projection device uses the actuator in the fiber scanner to drive the optical fiber to vibrate at high speed, and cooperates with the laser modulation algorithm to realize the display of image information. Figure 1A The optical fiber scanning projection device shown mainly includes: a processor 100 , a light source module 110 , an optical fiber scanner 120 , a transmission optical fiber 130 , a light source modulation circuit 140 , a scanning driving circuit 150 and a beam combining unit 160 .

[0020] The processor 100 may be a graphics processing unit (GPU), a central processing unit (CPU), or other chips or circuits with control functions and image processing functions, which are not specifically limited here.

[0021] When the system is working, the processor 100 can control the light source modulation circuit 140 to modulate the light source module 110 according to the image data to be displayed. In the embodiment of the present application, the laser modulation mode is internal modulation. The light source module 110 includes multiple monochromatic lasers, which emit light beams of different colors. Figure 1AAs can be seen in the figure, the light source module 110 can specifically use three-color lasers: red (Red, R), green (Green, G), and blue (Blue, B). The light beams emitted by each laser in the light source module 110 are combined into a beam of laser by the beam combining unit 160 and coupled into the transmission optical fiber 130. It should be noted that the number of three-color lasers in the light source module 110 is not limited to one group as shown in the figure, but can also be multiple groups to improve the brightness of the projected image. In addition, the three-color lasers here are visible light, which serves as the light source of the projected image. In some embodiments of the present specification, the light source module 110 may also include a light-emitting unit for invisible light, such as an infrared light-emitting unit (Infrared, I). The light beam emitted by the infrared light-emitting unit is also combined into a beam of laser by the beam combining unit 160 and coupled into the transmission optical fiber 130. Furthermore, since infrared light is invisible, it does not affect the quality of the final projected image. Accordingly, in the embodiment of the present application, an additional infrared light-emitting unit is provided. On the one hand, the infrared light emitted by the infrared light-emitting unit can be used for safety detection and image feedback correction. On the other hand, it can also be used as a scanning light source to scan the projection area and obtain relevant information of the projection area. In some embodiments, the number of infrared light-emitting units can be one or more. For example, the infrared light-emitting unit can include a first infrared light-emitting unit and a second infrared light-emitting unit, wherein the wavelengths of the infrared light emitted by the first infrared light-emitting unit and the second infrared light-emitting unit are different. For example, the first infrared light emitted by the first infrared light-emitting unit is used for safety detection and image feedback correction, and the second infrared light emitted by the second infrared light-emitting unit is used as a scanning light source to scan the projection area and obtain relevant information of the projection area. In some embodiments, the infrared light-emitting unit can include at least one infrared light source such as an infrared semiconductor laser and an infrared LED.

[0022] Furthermore, the fiber optic scanning projection device may include an infrared sensor 170, which is used to receive infrared information reflected from the projection area of ​​the fiber optic scanning projection device. The processor 100 may perform processing, analysis, etc. based on the infrared information. In some embodiments, the light source module 110 may not be provided with an infrared light-emitting unit, and the infrared sensor 170 may receive infrared information reflected from the object itself in the projection area. For example, when there is a living body in the projection area, the infrared sensor 170 may receive infrared information reflected by the user and send it to the processor 100. The processor 100 may identify the living body based on the infrared information, and control the working state of the fiber optic scanning projection device (for example, the opening and closing state) or the working state of the carrier (for example, the vehicle) where the fiber optic scanning projection device is located according to the recognition result. The vehicle in the embodiment of the present application is not limited to a car, but may also be other means of transportation, such as an electric bicycle, a motorcycle, a flying car, a scooter, etc.

[0023] It should be noted that the infrared light-emitting unit in the embodiment of the present application can also be replaced by other invisible light-emitting units, and accordingly, the infrared sensor can be replaced by a sensor corresponding to receiving other invisible light, which is not further limited here.

[0024] The processor 100 may also control the scanning driving circuit 150 to drive the optical fiber scanner 120 to scan, thereby scanning and outputting the light beam transmitted in the transmission optical fiber 130 .

[0025] The light beam scanned and output by the optical fiber scanner 120 acts on a certain pixel position on the surface of the medium, and forms a light spot at the pixel position, thereby realizing the scanning of the pixel position. Driven by the optical fiber scanner 120, the output end of the transmission optical fiber 130 sweeps along a certain scanning trajectory, so that the light beam moves to the corresponding pixel position for scanning. In the actual scanning process, the light beam output by the transmission optical fiber 130 will form a light spot with corresponding image information (such as color, grayscale or brightness) at each pixel position. In one frame of time, the light beam traverses each pixel position at a sufficiently high speed to complete the scanning of one frame of image. Due to the characteristic of "visual residual" when the human eye observes things, the human eye cannot perceive the movement of the light beam at each pixel position, but sees a complete frame of image.

[0026] Continue to refer Figure 1B , is the structure of the fiber scanner 120 (also known as an optical scanning module), which mainly includes: a piezoelectric actuator 121, a fiber cantilever 122, an imaging lens 123, a scanner package 124 and a fixing member 125. The piezoelectric actuator 121 is fixed in the scanner package 124 by the fixing member 125, and the transmission optical fiber 130 extends from the free end of the piezoelectric actuator 121 to form the fiber cantilever 122 (also known as a scanning optical fiber). When working, the piezoelectric actuator 121 vibrates along the Y-axis direction and the X-axis direction under the drive of the scanning drive signal. Driven by the piezoelectric actuator 121, the free end of the fiber cantilever 122 sweeps along a preset trajectory and emits a light beam, and the emitted light beam can be scanned on the surface of the medium through the imaging lens 123. Among them, the Y-axis direction intersects with the X-axis direction. Obviously, the Y-axis direction and the X-axis direction can be perpendicular. It should be noted that the transmission optical fiber 130 is not limited to the above-mentioned optical fiber. For example, in other alternative embodiments, the optical fiber may include a first optical fiber and a second optical fiber. The light beam emitted by the three-color laser (and the infrared light-emitting unit) of the laser module is coupled to the first optical fiber. The light output end of the first optical fiber is connected to the light input end of the second optical fiber. The second optical fiber extends along the length direction of the piezoelectric actuator 121 of the optical scanning module, and the second optical fiber extends at the free end of the piezoelectric actuator to form a fiber cantilever 122.

[0027] Please refer to Figure 2, is a schematic diagram of the structure of the coupling of the optical fiber and the light source module provided in the embodiment of the present application, the light source module includes at least three light-emitting units of R, G, and B, the wavelength beam combining device 230 is arranged on the outgoing light path of the light source 220, so that the light beams emitted by different lasers (R, G, and B three-color lasers) are combined into an image beam; the infrared beam combining device 250 is located behind the outgoing beam of the wavelength beam combining device 230, the infrared light-emitting unit 240 is located on the side of the infrared beam combining device 250, the outgoing beam of the infrared light-emitting unit 240 is perpendicular to the outgoing beam of the wavelength beam combining device 230, the outgoing beam of the infrared light-emitting unit 240 and the image beam emitted by the wavelength beam combining device 230 are combined into one beam by the infrared beam combining device 250, and then coupled into the first optical fiber 270 through the coupling lens 260. Optionally, the light source module can also be provided with two or more lasers of the same color, so as to increase the energy density of the light source. For example, if six lasers are arranged in the light source assembly, the number of lasers of the colors R, G, or B is two.

[0028] In the embodiment of the present application, the wavelength beam combining device 230 and the infrared beam combining device 250 are both wavelength division thin film filters, and the wavelength beam combining device 230 and the infrared beam combining device 250 correspond to 4 wavelength cut-off and pass, thereby combining the light beam and the infrared light emitted by the light source 220. In the embodiment of the present application, the filter can be fixed to the bottom of the first housing 210 with a thermal curing adhesive or a UV adhesive.

[0029] In the embodiment of the present application, the light source is fixed to the side wall of the first housing 210 by solder sintering, so that there is no gap between the light source and the first housing 210. In the embodiment of the present application, the laser can use a TO tube, which is also called a coaxial packaged laser diode. When the laser is fixed to the first housing 30, it is sintered with solder, and there is no gap between the laser and the first housing 210, and the heat dissipation is very good. The laser is sintered in the first housing 30, and the laser and the first housing 210 are not adjusted. The device is stable and the optical coupling efficiency is not affected by laser welding.

[0030] like Figure 3As shown, the optical fiber scanning projection device includes a second housing 310, one end of the second housing 310 is a light output end, an imaging lens 320 is provided at the light output end of the second housing 310, and an optical fiber scanner 330 is provided inside the second housing 310 for outputting image light to the outside through the imaging lens 320. The light beams output by each laser in the light source module are coupled to the first optical fiber 270 through the coupling lens 260, and the light output end of the first optical fiber 270 is connected to the light input end of the second optical fiber 340 of the optical fiber scanner 330. The optical fiber scanner 330 includes an actuator 331 and a second optical fiber 340, the light output end of the second optical fiber 340 is fixedly arranged at the free end of the actuator 331 in a cantilever support manner, the actuator 331 is fixedly installed in the second housing 310, and the free end of the actuator performs two-dimensional scanning vibration under the drive of the driving signal. Optionally, the actuator 331 is a piezoelectric ceramic actuator. When driven by a driving signal, the free end of the piezoelectric ceramic actuator performs high-frequency vibration along a first direction to achieve line scanning, and the free end of the piezoelectric ceramic actuator performs low-frequency vibration along a second direction to achieve frame scanning.

[0031] In some embodiments, the infrared sensor 170 of the optical fiber scanning projection device can be arranged on the peripheral side of the second shell 310, for example, the infrared sensor 170 can be located outside the peripheral side of the second shell 310. In other embodiments, the actuator 331 can include a shell, and along the length direction of the optical fiber, the outer side of the shell and the inner side of the second shell 310 have a gap, and the infrared sensor 170 can be located in the gap, or arranged on the inner side of the second shell or the outer side of the actuator shell. In other embodiments, the infrared sensor 170 can also be independently arranged relative to the second shell 310. When the optical fiber scanning projection device is installed on a corresponding carrier, the infrared sensor 170 can be arranged in the carrier. Here, the installation position of the infrared sensor 170 is not limited, and the position that can receive the infrared information reflected from the projection area can be sufficient.

[0032] The fiber optic scanning projection device provided in the embodiment of the present specification can obtain an imaging image of the projection area through the infrared sensor while realizing projection by additionally setting up an infrared light-emitting unit and an infrared sensor, and transmit the imaging image information to the processor. The processor corrects the imaging image based on the imaging image information fed back by the infrared sensor.

[0033] The fiber optic scanning projection device provided in the embodiments of this specification can also be used in other scenarios. The following is an explanation using liveness detection as a specific example. The fiber optic scanning projection device can be used to detect whether there is a living body in the projection area, and control the working state of the fiber optic scanning projection device or the working state of the carrier where the scanning projection device is located according to the liveness detection result. For example, when the liveness detection result shows that there is a living body in the projection area, the fiber optic scanning projection device is controlled to be in a closed state or other structures such as apertures that can be opened and closed are controlled to block the imaging lens to prevent the laser from causing damage to the eyes of the living body in the projection area.

[0034] The fiber optic scanning projection device can also be used to identify user actions and control the working state of the carrier where the fiber optic scanning projection device is located. As a specific example, the fiber optic scanning projection device is used to control the opening and closing state of the trunk of a vehicle. The infrared light-emitting unit of the fiber optic scanning projection device can scan the user's body movements in the projection area, and the infrared sensor receives infrared information reflected by the user's body from the projection area. For example, when the user's limbs (hands, legs, arms) pass near the trunk, the processor of the fiber optic scanning projection device can judge the user's limb movements based on the infrared information and control the trunk to open or close. In addition, the fiber optic scanning projection device can also be set near the door or sunroof to control the opening and closing of the door or sunroof.

[0035] The fiber optic scanning projection device provided in the embodiments of this specification can also scan objects in the external environment and establish corresponding three-dimensional models. For example, when the fiber optic scanning projection device is used in industrial production, three-dimensional modeling of molds and parts is performed to determine product quality. For another example, the fiber optic scanning projection device is used to measure user sizes to achieve the production of customized clothing.

[0036] This application also provides a projection method based on three-dimensional point cloud, for specific reference Figure 4 As shown in the flowchart, the method flow 400 includes the following steps:

[0037] Step 410: Acquire point cloud data of the projection carrier surface corresponding to the optical fiber scanning projection device, and pre-process the point cloud data.

[0038] In some embodiments, the projection carrier surface may be a plane, a curved surface, or an irregular surface. The embodiments of this specification mainly use curved surfaces as examples for explanation. For example, when the optical fiber scanning projection device is applied to a vehicle, the optical fiber projection scanning device projects an image on the vehicle window, roof, or seat surface of the vehicle, or the ground near the vehicle. These projection carrier surfaces are usually irregular surfaces.

[0039] The point cloud data may include three-dimensional coordinate information, color information, and laser reflection intensity information of the projection carrier surface. The point cloud data may be acquired by the infrared light emitting unit and infrared sensor of the optical fiber projection scanning device. In some embodiments, the preprocessing of the point cloud data may include any one or more of filtering processing, key point processing, feature description processing, classification and segmentation processing, and registration processing. The preprocessing of the point cloud data may be processed by a processor of the optical fiber scanning projection device.

[0040] Step 420: construct a surface model of the projection carrier surface based on the preprocessed point cloud data.

[0041] The surface model is a mathematical model that reflects the uneven surface of the projection carrier. In some embodiments, the point cloud data can be surface fitted or meshed using a principal component analysis (PCA) algorithm, a radial basis function (RBF), a B-spline surface algorithm, etc. to obtain a surface model.

[0042] Step 430: adjust the image information to be projected based on the curved surface model, and update the image information to be projected.

[0043] In some embodiments, adjusting the image information to be projected based on the surface model includes: constructing a mapping function or transformation matrix corresponding to the surface model and the image information to be projected, and spatially transforming the image information to be projected according to the above mapping function or transformation matrix, thereby updating the image information to be projected, so that the image emitted by the fiber optic scanning projection device matches the irregular projection carrier surface.

[0044] Step 440: projecting the image onto the surface of the carrier to be projected based on the updated image information to be projected.

[0045] This step is performed by a fiber optic projection scanning device.

[0046] Considering that when the fiber optic projection scanning device projects an image onto the surface of the carrier to be projected, the image may be distorted or at an inappropriate angle, which affects the image quality, the image needs to be corrected. In some embodiments, the projection method based on the three-dimensional point cloud may also include: collecting the imaging image projected on the surface of the projection carrier, obtaining imaging image information based on the imaging image, and continuing to update the image information to be projected according to the imaging image information to correct the imaging image. Furthermore, the collection of point cloud data on the surface of the projection carrier can be performed by the first infrared light-emitting unit, and the collection of the imaging image projected on the surface of the projection carrier can be performed by the second infrared light-emitting unit, wherein the wavelengths of the infrared light emitted by the first infrared light-emitting unit and the second infrared light-emitting unit may be different.

[0047] The embodiment of the present specification also provides a projection system based on three-dimensional point cloud, including: at least one processor; at least one memory for storing at least one program; when the at least one program is executed by the at least one processor, the at least one processor implements the above-mentioned projection method based on three-dimensional point cloud.

[0048] The embodiments of the present specification also provide a storage medium, in which a program executable by a processor is stored. When the program executable by the processor is executed by the processor, it is used to implement the above-mentioned projection method based on three-dimensional point cloud.

[0049] The embodiments of the present specification also provide a projection scanning device, including a projection device and an infrared sensor, wherein the infrared sensor is configured to receive infrared information reflected by the projection area of ​​the projection scanning device. In some embodiments, the projection device may include any one or more of a cathode ray tube device, an LCD projection device, a digital light processing projection device, and a liquid crystal on silicon projection device. Furthermore, the projection device also includes an infrared light emitting unit for emitting infrared light, and the infrared light emitting unit is configured to output infrared light to the projection area of ​​the projection scanning device.

[0050] All features disclosed in this specification, or steps in all methods or processes disclosed, except mutually exclusive features and / or steps, can be combined in any manner.

[0051] Any feature disclosed in this specification (including any additional claims, abstract and drawings), unless otherwise stated, may be replaced by other equivalent or alternative features with similar purposes. That is, unless otherwise stated, each feature is only an example of a series of equivalent or similar features.

[0052] The present application is not limited to the aforementioned specific implementations, but extends to any new features or any new combination disclosed in this specification, as well as any new method or process steps or any new combination disclosed.

Claims

1. A fiber scanning projection device, characterized in that: The optical scanning module comprises a light source module and a light scanning module. The light emitted by the light source module is scanned and output by the light scanning module to serve as display image light. The light scanning module comprises an actuator. The optical fiber output end in the light source module is fixed on the actuator. The optical fiber exceeds the actuator and forms an optical fiber cantilever. The optical fiber cantilever is driven by the actuator to sweep in space. The light source module includes at least one group of light sources, and each group of light sources includes at least three types of light-emitting units: R, G, and B; The optical fiber scanning projection device further includes an infrared sensor configured to receive infrared information reflected by a projection area of ​​the optical fiber scanning projection device.

2. The optical fiber scanning projection device according to claim 1, characterized in that: Each group of light sources of the light source module further includes an infrared light emitting unit for emitting infrared light, and the infrared light emitting unit is configured to output infrared light to a projection area of ​​the optical fiber scanning projection device.

3. The optical fiber scanning projection device according to claim 2, characterized in that: The infrared light emitting unit includes a first infrared light emitting unit and a second infrared light emitting unit, and the infrared light wavelengths generated by the first infrared light emitting unit and the second infrared light emitting unit are different.

4. The optical fiber scanning projection device according to claim 1, characterized in that: The infrared sensor is disposed on a peripheral side of the actuator.

5. The optical fiber scanning projection device according to claim 1, characterized in that: The optical scanning module comprises a packaging shell, which packages the actuator and the imaging module, and the infrared sensor is located on the inner side or the outer side of the packaging shell.

Images