A structured light module with a microcontroller

Abstract

A structured light module with a microcontroller includes: an infrared fill light for projecting infrared floodlight; a laser for projecting multiple discrete infrared beams with a pattern; an infrared sensor for receiving laser light reflected from the object to be photographed and generating raw data; a microcontroller for receiving a strobe signal from the infrared sensor and, based on the strobe signal, alternately turning on the infrared fill light and the laser, while maintaining complete synchronization between the projection time and the exposure time of the infrared sensor; and a system-on-a-chip for sequentially receiving the raw data from the infrared sensor and sequentially transmitting the image data according to the received order; the image data is the raw data or obtained by processing the raw data. This invention significantly reduces the problems caused by data transmission issues between the main system and the infrared fill light and laser, resulting in more stable and reliable image acquisition.

Description

Technical Field

[0001] This invention relates to structured light projection, and more specifically, to a structured light module with a microcontroller. Background Technology

[0002] Structured light projection technology is one of the most widely used technologies in 3D cameras. Due to its high precision and increasing technological maturity, it is being applied to more and more scenarios, and products are gradually becoming more integrated. Currently, various integrated structured light projectors have emerged, which can connect to on-chip systems via USB, enabling the system to control the structured light projector.

[0003] In a structured light projector USB camera solution, the infrared illuminator and laser light need to be turned on alternately to acquire infrared and depth images. Furthermore, to improve image quality and maximize the signal-to-noise ratio, the operating time of the infrared illuminator and laser light must be perfectly synchronized with the exposure time of the infrared sensor. Therefore, the motion control of the infrared illuminator, laser light, and infrared sensor becomes a critical issue.

[0004] In existing technologies, structured light projectors, infrared illuminator controllers, and infrared cameras are typically connected to a system-on-a-chip (SoC). By receiving signals from the SoC, they alternately acquire infrared and depth images. To ensure a high signal-to-noise ratio (SNR), the structured light projector or infrared illuminator needs to be turned on synchronously with the camera exposure to achieve optimal image quality. Otherwise, if the structured light projector or infrared illuminator turns on later than the infrared camera, the infrared camera will lack an active light signal during exposure, capturing only environmental noise and severely impacting image quality. SoCs, such as single-core Arm SoCs, generate numerous interrupts due to ISP and USB transmission. The priority and uncertainty of these interrupts mean that timely responses to light-on and light-off interrupts cannot be guaranteed. This leads to asynchronous infrared sensor exposure and light-on timing, resulting in SNR loss. In extreme cases, frame skipping may even occur (severe timing discrepancies between the laser and infrared lights, causing the laser to turn on during the infrared image acquisition period, resulting in data anomalies). Summary of the Invention

[0005] To address this, the present invention incorporates a microcontroller, eliminating the need for the on-chip system to control the switching between the infrared fill light and the laser light. This significantly reduces the problems caused by data transmission issues between the on-chip system and the infrared fill light and laser light during switching, resulting in more stable and reliable image acquisition.

[0006] This invention provides a structured light module with a microcontroller, characterized in that it comprises:

[0007] Infrared fill light, used to project infrared floodlight;

[0008] Laser lights are used to project multiple discrete infrared beams with patterns.

[0009] An infrared sensor is used to receive the laser light reflected from the object to be photographed and generate raw data;

[0010] A microcontroller is used to receive the Strobe signal from the infrared sensor and, based on the Strobe signal, alternately turn on the infrared fill light and the laser light, while keeping the projection time completely synchronized with the exposure time of the infrared sensor.

[0011] The system-on-a-chip is used to sequentially receive raw data from the infrared sensor; and sequentially transmit the image data in the order of receipt; the image data is the raw data or obtained by processing the raw data.

[0012] Optionally, the structured light module with a microcontroller is characterized in that, when the microcontroller responds to the Strobe signal, it includes the following steps:

[0013] S301: In response to the Strobe signal, determine whether the currently turned-on light is an infrared fill light or a laser light; if it is the infrared fill light, proceed to step S302; if it is the laser light, proceed to step S304.

[0014] S302: When the infrared fill light illumination reaches the first preset duration, turn off the infrared fill light and stop the infrared sensor exposure to obtain the first image; after waiting for the first time interval, execute step S303;

[0015] S303: Turn on the laser lamp and simultaneously start the infrared sensor to begin exposure; proceed to step S304;

[0016] S304: When the laser light illumination reaches the second preset duration, turn off the laser light and stop the infrared sensor exposure to obtain the second image; after waiting for the second time interval, execute step S305;

[0017] S305: Turn on the infrared fill light and simultaneously start the infrared sensor to begin exposure; execute step S302.

[0018] Optionally, the structured light module with a microcontroller is characterized in that, when the microcontroller responds to the Strobe signal, it includes the following steps:

[0019] S301: In response to the Strobe signal, determine whether the currently turned-on light is an infrared fill light or a laser light; if it is the infrared fill light, proceed to step S302; if it is the laser light, proceed to step S307.

[0020] S302: When the infrared fill light illumination reaches the first preset duration, turn off the infrared fill light and stop the infrared sensor exposure to obtain the first image; after waiting for the first time interval, execute step S303;

[0021] S303: Turn on the laser lamp and simultaneously start the infrared sensor to begin exposure; proceed to step S304;

[0022] S304: When the laser light illumination reaches the second preset duration, turn off the laser light and stop the infrared sensor exposure to obtain the second image; after waiting for the second time interval, execute step S306;

[0023] S306: Turn on the laser lamp and simultaneously start the exposure of the infrared sensor; proceed to step S307;

[0024] S307: When the laser light illumination reaches the second preset duration, turn off the laser light and stop the infrared sensor exposure to obtain the third image; after waiting for the second time interval, execute step S305;

[0025] S305: Turn on the infrared fill light and simultaneously start the infrared sensor to begin exposure; execute step S302.

[0026] Optionally, the structured light module with a microcontroller is characterized in that the system-on-a-chip transmits the first image, the second image, and the third image received adjacently as a combination.

[0027] Optionally, the structured light module with a microcontroller is characterized in that, when the microcontroller responds to the Strobe signal, it includes the following steps:

[0028] S401: Respond to the Strobe signal and determine whether only the infrared fill light is currently on or both the infrared fill light and the laser light are on. If only the infrared fill light is on, proceed to step S402. If both the infrared fill light and the laser light are on, proceed to step S404.

[0029] S402: When the infrared fill light illumination reaches the first preset duration, turn off the infrared fill light and stop the infrared sensor exposure to obtain the first image; after waiting for the first time interval, execute step S403;

[0030] S403: Turn on the laser lamp and the infrared fill light, and simultaneously start the exposure of the infrared sensor; proceed to step S404;

[0031] S404: When the laser light and the infrared fill light have been irradiated for a second preset time, turn off the laser light and the infrared fill light, and stop the infrared sensor exposure to obtain the sixth image; after waiting for the second time interval, execute step S405;

[0032] S405: Turn on the infrared fill light and simultaneously start the infrared sensor to begin exposure; execute step S402.

[0033] Optionally, the structured light module with a microcontroller is characterized in that, when the microcontroller responds to the Strobe signal, it includes the following steps:

[0034] S401: In response to the Strobe signal, determine whether only the infrared fill light is currently turned on or both the infrared fill light and the laser light are turned on; if only the infrared fill light is turned on, proceed to step S402; if both the infrared fill light and the laser light are turned on, proceed to step S407.

[0035] S402: When the infrared fill light illumination reaches the first preset duration, turn off the infrared fill light and stop the infrared sensor exposure to obtain the first image; after waiting for the first time interval, execute step S403;

[0036] S403: Turn on the laser lamp and the infrared fill light, and simultaneously start the exposure of the infrared sensor; proceed to step S404;

[0037] S404: When the laser light and the infrared fill light have been irradiated for a second preset time, turn off the laser light and the infrared fill light, and stop the infrared sensor exposure to obtain the sixth image; after waiting for the second time interval, execute step S406;

[0038] S406: Turn on the laser light and the infrared fill light, and simultaneously start the exposure of the infrared sensor; proceed to step S407;

[0039] S407: When the laser light and the infrared fill light have been irradiated for a second preset time, turn off the laser light and the infrared fill light, and stop the infrared sensor exposure to obtain the seventh image; after waiting for the second time interval, execute step S405;

[0040] S405: Turn on the infrared fill light and simultaneously start the infrared sensor to begin exposure; execute step S402.

[0041] Optionally, the structured light module with a microcontroller is characterized in that the system-on-a-chip subtracts the first image from the sixth image to obtain the image data.

[0042] Optionally, the structured light module with a microcontroller is characterized in that the system-on-a-chip subtracts the first image from the sixth image to obtain first image data; the system-on-a-chip subtracts the first image from the seventh image to obtain second image data; the first image data and the second image data together constitute the image data.

[0043] Optionally, the structured light module with a microcontroller is characterized in that, when the on-chip system transmits the image data sequentially according to the received order, it includes the following steps:

[0044] Step S501: Send the image data sequentially and receive the return result; if sending fails, proceed to step S502;

[0045] Step S502: Mark the image data that failed to be sent as the first data to be sent, and send it again;

[0046] Step S503: Stop sending the image data received after the first data to be sent, store it sequentially according to the receiving time, and mark it as the second data to be sent;

[0047] Step S504: When the successful transmission signal of the first data to be transmitted is received, the second data to be transmitted is transmitted sequentially, and the successfully transmitted second data to be transmitted is deleted from the storage;

[0048] Step S505: If the number of times the first data to be sent fails to be sent reaches a first preset value or the capacity of the second data to be sent reaches a second preset value, then stop projecting structured light.

[0049] Optionally, the structured light module with a microcontroller is characterized in that the image data sent by the system-on-chip is a fourth image obtained by processing the first image and the second image, or a fourth image and a fifth image obtained by processing the first image, the second image and the third image.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] This invention uses a microcontroller to control the switching of structured light sources. It is highly adaptable to solutions with interfaces such as USB, making the switching of structured light sources independent of the on-chip system. While maintaining the direct control of the structured light source by the on-chip system, it increases independence and allows the collected data to be transmitted to the on-chip system in a timely manner. This greatly enhances stability while ensuring efficiency.

[0052] In this invention, the microcontroller is placed inside the structured light module, minimizing the distance between the microcontroller and the infrared fill light, laser light, and infrared sensor. This reduces control latency to the greatest extent, enabling precise control of light source switching, improving control accuracy, and resulting in higher quality image data, thus making the final depth map more reliable.

[0053] This invention controls the switching of structured light sources using a microcontroller, reducing the data transmission volume between the structured light module and the system-on-a-chip (SoC), resulting in smoother and more stable communication between the two systems. This invention facilitates the mass production and application of structured light sources, enabling the SoC to control multiple structured light sources simultaneously with high stability.

[0054] Meanwhile, the presence of the microcontroller allows the structured light module to connect to a wider variety of systems through an interface, improving the system's versatility and adaptability, and making it more conducive to the promotion and application of the structured light module. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort. Other features, objects, and advantages of the present invention will become more apparent by reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0056] Figure 1 This is a structural block diagram of a structured light module with a microcontroller according to an embodiment of the present invention;

[0057] Figure 2 This is a flowchart illustrating the steps of a microcontroller alternately turning on an infrared fill light and a laser light in an embodiment of the present invention;

[0058] Figure 3 This is a flowchart illustrating the steps of another microcontroller alternately turning on the infrared fill light and the laser light in an embodiment of the present invention;

[0059] Figure 4 This is a flowchart illustrating the steps of a microcontroller alternately turning on an infrared fill light and a laser light in another embodiment of the present invention;

[0060] Figure 5 This is a flowchart illustrating the steps of a microcontroller alternately turning on an infrared fill light and a laser light in an embodiment of the present invention;

[0061] Figure 6 This is a flowchart illustrating the steps of sequentially sending image data in an embodiment of the present invention. Detailed Implementation

[0062] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0063] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0064] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0065] The present invention provides a structured light module with a microcontroller, which aims to solve the problems existing in the prior art.

[0066] The technical solutions of the present invention and how they solve the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0067] Figure 1 This is a structural block diagram of a structured light module with a microcontroller according to an embodiment of the present invention. Figure 1 As shown, an embodiment of the present invention provides a structured light module with a microcontroller, including an infrared fill light 110, a laser light 120, an infrared sensor 130, and a microcontroller 140.

[0068] Specifically, the infrared fill light 110 is used to project infrared floodlight. The infrared fill light 110 uses LED light-emitting diodes, offering high cost-effectiveness and providing good illumination within 150 meters, meeting the needs of structured light illumination. The infrared fill light can receive signals from the on-chip system to start operating.

[0069] Laser lamp 120 is used to project multiple discrete infrared beams with patterns. The laser lamp 120 projects a predetermined coded pattern, which is then received by an infrared sensor 130 from the reflection of the object being photographed, obtaining raw data from which a point cloud can be reconstructed. Laser lamp 120 can receive signals from the on-chip system to begin operation.

[0070] Infrared sensor 130 is used to receive laser light reflected from the object to be photographed, generating raw data. Infrared sensor 130 is also used to receive light reflected from the object by infrared fill light 110 and laser light 120, and then expose the object to obtain raw data. Infrared sensor 130 can receive signals from the on-chip system to start operation.

[0071] Microcontroller 140 receives the strobe signal from the infrared sensor and, based on the strobe signal, alternately turns on the infrared fill light and the laser light, maintaining complete synchronization between the projection time and the exposure time of the infrared sensor. Microcontroller 140 is connected to the infrared fill light 110, the laser light 120, and the infrared sensor 130. Microcontroller 140 controls the operation and shutdown of the infrared fill light 110 and the laser light 120, and, when either is operating, controls the infrared sensor 130 to synchronously expose, receiving signals to generate the original image.

[0072] The system-on-chip 150 is configured to receive raw data sequentially from the infrared sensor and transmit the image data sequentially in the order of receipt; the image data is the raw data or obtained by processing the raw data.

[0073] In this embodiment, the infrared fill light 110, laser 120, and infrared sensor 130 receive the instruction to project structured light from the system-on-a-chip (SoC) and begin projecting structured light. The SoC is the control system for the structured light source. The SoC's instructions are directly sent to the fill light, laser, and infrared sensor, which then operate according to the signals. This step eliminates the need for a microcontroller. Compared to schemes where the microcontroller issues initial instructions, this embodiment improves signal transmission efficiency without altering the original system's control path, offering better compatibility.

[0074] Unlike existing technologies where infrared sensors send strobe signals to the on-chip system, in this embodiment, the infrared sensor sends the strobe signal to the microcontroller. The microcontroller, located within the structured light module, can transmit signals quickly and stably. Upon first receiving the strobe signal from the infrared sensor, the microcontroller responds, prepares to receive signals, and determines whether an infrared fill light or a laser light is currently activated, thus facilitating subsequent control.

[0075] The microcontroller controls the on / off state of the infrared fill light and laser light, simultaneously initiating exposure of the infrared sensor when the laser light or infrared fill light is turned on, and simultaneously stopping exposure of the infrared sensor when the laser light or infrared fill light is turned off, ensuring that the desired image can be obtained. The microcontroller controls the infrared fill light and laser light at a preset frequency, such as 24 frames / second or 30 frames / second. In some embodiments, the microcontroller receives the frequency value set by the on-chip system and uses it as the control frequency for the infrared fill light and laser light.

[0076] For example, the laser light is turned on at 0ms and the infrared sensor is exposed simultaneously, and the laser light is turned off at 20ms and the infrared sensor is turned off simultaneously; the infrared fill light is turned on at 40ms and the infrared sensor is exposed simultaneously, and the laser light is turned off at 60ms and the infrared sensor is turned off simultaneously.

[0077] The raw data output by the external sensor is discontinuous, therefore the raw data received by the microcontroller is also discontinuous. The raw data is directly generated by the infrared sensor, and after 3D reconstruction, a depth map can be generated for display on a monitor and perceived by the human eye. The raw data received by the on-chip system needs to be recorded in sequence for convenient subsequent operations. In some embodiments, after receiving the raw data sequentially from the infrared sensor, the on-chip system adds the current time to each image, with the minimum unit being milliseconds. In some embodiments, after receiving the raw data sequentially from the infrared sensor, the on-chip system adds a sequence number to each image.

[0078] The image data is the original data or obtained by processing the original data. When the on-chip system processes the original data, it can process a single piece of original data or perform related processing on two or more images. When the on-chip system sends image data, the sending order is the same as the receiving order; that is, the image data corresponding to the original data received first is sent earlier. In some embodiments, the on-chip system sends the image data one by one in the order of receipt. In some embodiments, the on-chip system sends the image data in combination in the order of receipt. The image data is a fourth image obtained by processing the first image and the second image, or a fourth and fifth image obtained by processing the first image, the second image, and the third image.

[0079] The infrared fill light, laser, and infrared sensor receive a stop signal from the on-chip system and cease structured light projection and exposure, ending the structured light operation. The on-chip system completes image data transmission and also stops operating after detecting that the fill light, laser, and infrared sensor have stopped. In some embodiments, the on-chip system receives the stop signal, and then the microcontroller controls the infrared fill light, laser, and infrared sensor to stop operating.

[0080] This embodiment incorporates a microcontroller within the structured light module to control the operation and shutdown of the infrared fill light and laser light. This eliminates the need for the on-chip system to directly control the structured light module; only start and stop signals are required. This increases the stability of the structured light module while reducing data transmission with the on-chip system and its load. This facilitates large-area deployment of the structured light module and expands its application areas.

[0081] Figure 2 This is a flowchart illustrating the steps of a microcontroller alternately turning on an infrared fill light and a laser light in an embodiment of the present invention. Figure 2 As shown in the flowchart, an embodiment of the present invention provides a step-by-step process for a microcontroller to alternately turn on an infrared fill light and a laser light, including the following steps:

[0082] S301: In response to the Strobe signal, determine whether the currently turned-on light is an infrared fill light or a laser light; if it is the infrared fill light, proceed to step S302; if it is the laser light, proceed to step S304.

[0083] In this step, in response to the Strobe signal, it is necessary to determine whether the currently operating light is an infrared fill light or a laser light. In some embodiments, a fixed operating mode is used, where the first light to operate is a fixed type of light, i.e., a laser light or an infrared fill light. In this case, the determination stage can be skipped, and the corresponding subsequent steps can be executed directly.

[0084] S302: When the infrared fill light reaches the first preset duration, turn off the infrared fill light and stop the infrared sensor exposure to obtain the first image.

[0085] In this step, the first preset duration is the illumination duration of the infrared supplementary light, which is also the duration for the infrared sensor to collect the floodlight. Multiple first images can be obtained through multiple acquisitions. After each acquisition, there is a fixed first time interval to distinguish the data. After waiting for the first time interval, step S303 is executed.

[0086] S303: Turn on the laser lamp and simultaneously start the exposure of the infrared sensor.

[0087] In this step, the laser light is controlled to start illuminating, and raw data during laser illumination is collected. Execute step S304.

[0088] S304: When the laser light illumination reaches the second preset duration, the laser light is turned off and the infrared sensor exposure is stopped to obtain the second image.

[0089] In this step, the second preset duration is the illumination duration of the laser lamp, which is also the acquisition duration of the structured light by the infrared sensor. Multiple acquisitions can yield multiple second images. After each acquisition, there is a fixed second time interval to distinguish the data. The second image has the same size as the first image. The second preset duration can be equal to or unequal to the first preset duration. When the first and second preset durations are equal, the first and second images are acquired within the same exposure time, resulting in better comparability. When the first and second preset durations are unequal, this can be combined with adjusting the intensity of the light source to ensure the comparability of the final images, but this can shorten the exposure time and increase the illumination frequency of the structured light. After waiting for the second time interval, step S305 is executed.

[0090] S305: Turn on the infrared fill light and simultaneously start the exposure of the infrared sensor.

[0091] In this step, the infrared fill light is controlled to start illuminating, and raw data from the laser light illumination is collected. Execute step S302.

[0092] In this embodiment, the illumination of the infrared fill light and the laser light is controlled at intervals, and the exposure of the infrared sensor is synchronized. The raw data of the infrared image and the speckle image are collected alternately to ensure that the exposure and illumination are synchronized. The patterns of the laser light and the infrared fill light are collected alternately. By adjusting the values ​​of the first preset duration and the second preset duration, more image acquisition options are available. It is possible to adjust the acquisition frequency, reduce the requirements of the laser light and the infrared fill light, make it suitable for more scenarios, and reduce costs.

[0093] Figure 3 This is a flowchart illustrating the steps of alternately turning on the infrared fill light and the laser light using a microcontroller in another embodiment of the present invention. Figure 3 As shown in the flowchart, another microcontroller according to an embodiment of the present invention alternately turns on the infrared fill light and the laser light, including the following steps:

[0094] S301: In response to the Strobe signal, determine whether the currently turned-on light is an infrared fill light or a laser light; if it is the infrared fill light, proceed to step S302; if it is the laser light, proceed to step S307.

[0095] In this step, in response to the Strobe signal, it is necessary to determine whether the currently operating light is an infrared fill light or a laser light. In some embodiments, a fixed operating mode is used, where the first light to operate is a fixed type of light, i.e., a laser light or an infrared fill light. In this case, the determination stage can be skipped, and the corresponding subsequent steps can be executed directly.

[0096] S302: When the infrared fill light reaches the first preset duration, turn off the infrared fill light and stop the infrared sensor exposure to obtain the first image.

[0097] In this step, the first preset duration is the illumination duration of the infrared supplementary light, which is also the duration for the infrared sensor to collect the floodlight. Multiple first images can be obtained through multiple acquisitions. After each acquisition, there is a fixed first time interval to distinguish the data. After waiting for the first time interval, step S303 is executed.

[0098] S303: Turn on the laser lamp and simultaneously start the exposure of the infrared sensor.

[0099] In this step, the laser light is controlled to start illuminating, and raw data during laser illumination is collected. Execute step S304.

[0100] S304: When the laser light illumination reaches the second preset duration, the laser light is turned off and the infrared sensor exposure is stopped to obtain the second image;

[0101] In this step, the second preset duration is the illumination duration of the laser lamp, which is also the acquisition duration of the structured light by the infrared sensor. Multiple acquisitions can yield multiple second images. After each acquisition, there is a fixed second time interval to distinguish the data. The second image has the same size as the first image. The second preset duration can be equal to or unequal to the first preset duration. When the first and second preset durations are equal, the first and second images are acquired within the same exposure time, resulting in better comparability. When the first and second preset durations are unequal, this can be combined with adjusting the intensity of the light source to ensure the comparability of the final images, but this can shorten the exposure time and increase the illumination frequency of the structured light. After waiting for the second time interval, step S306 is executed.

[0102] S306: Turn on the laser lamp and simultaneously start the exposure of the infrared sensor.

[0103] In this step, the laser lamp is turned on again, and the infrared sensor is synchronized to begin exposure in order to acquire speckle images again. Proceed to step S307.

[0104] S307: When the laser light illumination reaches the second preset duration, the laser light is turned off and the infrared sensor exposure is stopped to obtain a third image.

[0105] In this step, the third image has the same size as the second and first images. This step involves repeatedly acquiring speckle images. In this embodiment, the third and second images share a first image, which increases the frequency of data acquisition. For example, if the first image a1, second image b1, third image c1, first image a2, second image b2, and third image c3 are acquired sequentially, then the third image c1 and second image b2 share the first image a2. In subsequent image processing, they are all compared with a2, thereby increasing the acquisition frequency of speckle light. After waiting for the second time interval, step S305 is executed.

[0106] S305: Turn on the infrared fill light and simultaneously start the exposure of the infrared sensor.

[0107] In this embodiment, the system-on-a-chip (SoC) transmits the first image, the second image, and the third image received adjacently as a combination. Specifically, the SoC transmits the first image and the second image received adjacently as a first combination, and transmits the first image and the third image received adjacently as a second combination. The SoC transmits the combinations in the order of the second or third image within each combination. Step S302 is then executed.

[0108] In some embodiments, adjacent first and second images are processed to obtain a fourth image, and adjacent first and third images are processed to obtain a fifth image. The fourth and fifth images are then transmitted, with the fourth image in the same order as the second image, and the fifth image in the same order as the third image. Processing the original data through the on-chip system reduces the amount of image data transmitted, decreasing the data transmission volume and the load on the on-chip system. This improves transmission stability and the stability of the on-chip system when large-scale structural light source deployments are implemented.

[0109] This embodiment increases the proportion of laser projection, thereby increasing the structured light acquisition frequency and resulting in more effective image data with the same number of projections. Furthermore, the first, second, and third images can be combined to achieve efficient reuse of image data and can be sent in combination, making backend processing more convenient.

[0110] Figure 4 This is a flowchart illustrating the steps of a microcontroller alternately turning on an infrared fill light and a laser light in another embodiment of the present invention. Figure 4 As shown in the flowchart of another embodiment of the present invention, a microcontroller alternately turns on an infrared fill light and a laser light, including the following steps:

[0111] S401: Responds to the Strobe signal to determine whether only the infrared fill light is currently on or both the infrared fill light and the laser light are on.

[0112] In this step, the infrared fill light illuminates the object to be photographed and the background. When both the laser light and the infrared fill light work together, an image of the object to be photographed and the background with light spots can be obtained. If only the infrared fill light is used, proceed to step S402; if both the infrared fill light and the laser light are used, proceed to step S404.

[0113] S402: When the infrared fill light reaches the first preset duration, turn off the infrared fill light and stop the infrared sensor exposure to obtain the first image.

[0114] In this step, the first image is the background image obtained when illuminated by an infrared fill light. After waiting for the first time interval, step S403 is executed.

[0115] S403: Turn on the laser light and the infrared fill light, and simultaneously start the exposure of the infrared sensor.

[0116] In this step, the laser light and the infrared fill light work simultaneously to make the light intensity stronger. Proceed to step S404.

[0117] S404: When the laser light and the infrared fill light have been irradiated for a second preset time, the laser light and the infrared fill light are turned off, and the infrared sensor exposure is stopped to obtain the sixth image.

[0118] In this step, the light intensity in the sixth image includes both the light intensity of the laser and the light intensity of the infrared sensor. After waiting for the second time interval, step S405 is executed.

[0119] S405: Turn on the infrared fill light and simultaneously start the exposure of the infrared sensor.

[0120] In this step, the infrared fill light and infrared sensor are turned on to collect background information. Execute step S402.

[0121] In this embodiment, the on-chip system subtracts the first image from the sixth image to obtain the image data.

[0122] This embodiment collects background information separately, collects target information and background information together, and then removes background information by image subtraction. This can filter out background interference and has good adaptability to scenes with large ambient light interference. It improves the effective detection distance of the structured light module, making it adaptable to more scenarios and facilitating the promotion and application of structured light modules.

[0123] Figure 5 This is a flowchart illustrating the steps of a microcontroller alternately turning on an infrared fill light and a laser light in an embodiment of the present invention; as shown below. Figure 5 As shown in the flowchart of another embodiment of the present invention, a microcontroller alternately turns on an infrared fill light and a laser light, including the following steps:

[0124] S401: Responds to the Strobe signal to determine whether only the infrared fill light is currently on or both the infrared fill light and the laser light are on.

[0125] In this step, the current laser irradiation status is determined. If the structured light module is fixed, this step can be skipped, and the corresponding steps can be executed directly. If only an infrared fill light is used, step S402 is executed; if both an infrared fill light and a laser light are used, step S407 is executed.

[0126] S402: When the infrared fill light reaches the first preset duration, turn off the infrared fill light and stop the infrared sensor exposure to obtain the first image.

[0127] In this step, the first image is an image illuminated by an infrared lamp. After waiting for the first time interval, step S403 is executed;

[0128] S403: Turn on the laser light and the infrared fill light, and simultaneously start the exposure of the infrared sensor.

[0129] In this step, the laser light and infrared supplementary light work together to provide stronger illumination. Execute step S404;

[0130] S404: When the laser light and the infrared fill light have been irradiated for a second preset time, the laser light and the infrared fill light are turned off, and the infrared sensor exposure is stopped to obtain the sixth image.

[0131] In this step, the light intensity in the sixth image includes both the light intensity of the laser and the light intensity of the infrared sensor. After waiting for the second time interval, proceed to step S406;

[0132] S406: Turn on the laser light and the infrared fill light, and synchronize the exposure of the infrared sensor.

[0133] In this step, turn on the laser light and infrared fill light again. Execute step S407;

[0134] S407: When the laser light and the infrared fill light have been irradiated for a second preset time, the laser light and the infrared fill light are turned off, and the infrared sensor exposure is stopped to obtain the seventh image.

[0135] In this step, the light intensity in the seventh image includes both the light intensity of the laser lamp and the light intensity of the infrared sensor. The seventh image and the sixth image have the same exposure time, image size, etc. After waiting for the second time interval, step S405 is executed;

[0136] S405: Turn on the infrared fill light and simultaneously start the exposure of the infrared sensor.

[0137] In this step, the infrared fill light and infrared sensor are turned on again to acquire floodlight images. Execute step S402.

[0138] The on-chip system subtracts the first image from the sixth image to obtain first image data; the on-chip system subtracts the first image from the seventh image to obtain second image data; the first image data and the second image data together constitute the image data.

[0139] This embodiment collects background information separately, collects target information and background information together, and then removes the background information by image subtraction. Adjacent images share the same first image. While filtering out background interference, it also increases the acquisition frequency. It has good adaptability to scenes with large ambient light interference, improves the effective detection distance of the structured light module, and makes it adaptable to more scenarios. This is conducive to the promotion and application of structured light modules and the acquisition of more data.

[0140] Figure 6 This is a flowchart illustrating the steps of sequentially sending image data in an embodiment of the present invention. Figure 4 As shown in the flowchart, an embodiment of the present invention provides a method for sequentially sending image data, which includes the following steps:

[0141] Step S501: Send the image data sequentially and receive the return result; if the sending fails, proceed to step S502.

[0142] In this step, considering the potential image data transmission failures during USB transfer or other adapter connections, appropriate processing is required to improve the stability and reliability of data transmission. When an image data transmission failure is detected, step S502 is executed.

[0143] Step S502: Mark the image data that failed to be sent as the first data to be sent, and send it again.

[0144] In this step, the first data to be sent is always given priority in the transmission order. The first data to be sent is the earliest received among the unsent image data. By prioritizing its transmission, the on-chip system can receive the image data in sequence, and the transmission of data by the on-chip system can be kept consistent.

[0145] If the first data to be sent is a single image, then continue sending the first data to be sent.

[0146] If the first data to be sent consists of at least two images, then the first data to be sent is sent sequentially according to the order in which they are received.

[0147] Step S503: Stop sending the image data received after the first data to be sent, store it sequentially according to the receiving time, and mark it as the second data to be sent.

[0148] In this step, since only the first data to be sent is transmitted, the raw data received after the first data to be sent needs to be stored and identified as the second data to be sent. The second data to be sent needs to be identified during storage; the specific identification method is described in the aforementioned embodiment and will not be repeated here. The second data to be sent is stored in the on-chip system.

[0149] Step S504: When the successful transmission signal of the first data to be transmitted is received, the second data to be transmitted is transmitted sequentially, and the successfully transmitted second data to be transmitted is deleted from the storage.

[0150] In this step, the first data transmission was successful, indicating data transmission recovery. The transmission of the second data to be transmitted then begins sequentially. During this step, newly received raw data still needs to be stored. Since the raw data is received intermittently, while image data can be transmitted continuously, the amount of information stored on the on-chip system gradually decreases. Successfully transmitted second data to be transmitted needs to be deleted from storage to free up storage space.

[0151] Step S505: If the number of times the first data to be sent fails to be sent reaches a first preset value or the capacity of the second data to be sent reaches a second preset value, then stop projecting structured light.

[0152] In this step, a mechanism is set to terminate structured light projection in case of connection failure. If the number of failed transmissions of the first set of data to be sent reaches a first preset value, the connection is considered to have been interrupted for an extended period and cannot be repaired in the short term, thus stopping structured light projection. This approach is suitable for cases with large storage capacity. If the number of failed transmissions of the second set of data to be sent reaches a second preset value, it is considered that there is no remaining storage space to store new raw data, thus stopping structured light projection. The first preset value is highly time-dependent; generally, a duration exceeding one second is considered an interruption. The second preset value is highly time-dependent and can be set to the available storage capacity or slightly lower.

[0153] The various embodiments described in this specification are presented in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0154] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A structured light module with a microcontroller, characterized in that, include: Infrared fill light, used to project infrared floodlight; Laser lights are used to project multiple discrete infrared beams with patterns. An infrared sensor is used to receive the laser light reflected from the object being photographed and generate raw data; A microcontroller is used to receive the Strobe signal from the infrared sensor and, based on the Strobe signal, alternately turn on the infrared fill light and the laser light, while keeping the projection time completely synchronized with the exposure time of the infrared sensor. The system-on-a-chip is used to sequentially receive raw data from the infrared sensor and sequentially transmit image data in the order of receipt; the image data is the raw data or obtained by processing the raw data.

2. A structured light module with a microcontroller according to claim 1, characterized in that, When the microcontroller responds to the Strobe signal, it includes the following steps: S301: In response to the Strobe signal, determine whether the currently turned-on light is an infrared fill light or a laser light; if it is the infrared fill light, proceed to step S302; if it is the laser light, proceed to step S304. S302: When the infrared fill light illumination reaches the first preset duration, turn off the infrared fill light and stop the infrared sensor exposure to obtain the first image; after waiting for the first time interval, execute step S303; S303: Turn on the laser lamp and simultaneously start the infrared sensor to begin exposure; proceed to step S304; S304: When the laser light illumination reaches the second preset duration, turn off the laser light and stop the infrared sensor exposure to obtain the second image; after waiting for the second time interval, execute step S305; S305: Turn on the infrared fill light and simultaneously start the infrared sensor to begin exposure; execute step S302.

3. A structured light module with a microcontroller according to claim 1, characterized in that, When the microcontroller responds to the Strobe signal, it includes the following steps: S301: In response to the Strobe signal, determine whether the currently turned-on light is an infrared fill light or a laser light; if it is the infrared fill light, proceed to step S302; if it is the laser light, proceed to step S307. S302: When the infrared fill light illumination reaches the first preset duration, turn off the infrared fill light and stop the infrared sensor exposure to obtain the first image; after waiting for the first time interval, execute step S303; S303: Turn on the laser lamp and simultaneously start the infrared sensor to begin exposure; proceed to step S304; S304: When the laser light illumination reaches the second preset duration, turn off the laser light and stop the infrared sensor exposure to obtain the second image; after waiting for the second time interval, execute step S306; S306: Turn on the laser lamp and simultaneously start the exposure of the infrared sensor; proceed to step S307; S307: When the laser light illumination reaches the second preset duration, turn off the laser light and stop the infrared sensor exposure to obtain the third image; after waiting for the second time interval, execute step S305; S305: Turn on the infrared fill light and simultaneously start the infrared sensor to begin exposure; execute step S302.

4. A structured light module with a microcontroller according to claim 3, characterized in that, The on-chip system transmits the first image, the second image, and the third image received adjacently as a combination.

5. A structured light module with a microcontroller according to claim 1, characterized in that, When the microcontroller responds to the Strobe signal, it includes the following steps: S401: Respond to the Strobe signal and determine whether only the infrared fill light is currently on or both the infrared fill light and the laser light are on. If only the infrared fill light is on, proceed to step S402. If both the infrared fill light and the laser light are on, proceed to step S404. S402: When the infrared fill light illumination reaches the first preset duration, turn off the infrared fill light and stop the infrared sensor exposure to obtain the first image; after waiting for the first time interval, execute step S403; S403: Turn on the laser light and the infrared fill light, and simultaneously start the exposure of the infrared sensor; Execute step S404; S404: When the laser light and the infrared fill light have been irradiated for a second preset time, turn off the laser light and the infrared fill light, and stop the infrared sensor exposure to obtain the sixth image; after waiting for the second time interval, execute step S405; S405: Turn on the infrared fill light and simultaneously start the infrared sensor to begin exposure; execute step S402.

6. A structured light module with a microcontroller according to claim 1, characterized in that, When the microcontroller responds to the Strobe signal, it includes the following steps: S401: In response to the Strobe signal, determine whether only the infrared fill light is currently turned on or both the infrared fill light and the laser light are turned on; if only the infrared fill light is turned on, proceed to step S402; if both the infrared fill light and the laser light are turned on, proceed to step S407. S402: When the infrared fill light illumination reaches the first preset duration, turn off the infrared fill light and stop the infrared sensor exposure to obtain the first image; after waiting for the first time interval, execute step S403; S403: Turn on the laser light and the infrared fill light, and simultaneously start the exposure of the infrared sensor; Execute step S404; S404: When the laser light and the infrared fill light have been irradiated for a second preset time, turn off the laser light and the infrared fill light, and stop the infrared sensor exposure to obtain the sixth image; after waiting for the second time interval, execute step S406; S406: Turn on the laser light and the infrared fill light, and simultaneously start the exposure of the infrared sensor; proceed to step S407; S407: When the laser light and the infrared fill light have been irradiated for a second preset time, turn off the laser light and the infrared fill light, and stop the infrared sensor exposure to obtain the seventh image; after waiting for the second time interval, execute step S405; S405: Turn on the infrared fill light and simultaneously start the infrared sensor to begin exposure; execute step S402.

7. A structured light module with a microcontroller according to claim 5, characterized in that, The on-chip system subtracts the first image from the sixth image to obtain the image data.

8. A structured light module with a microcontroller according to claim 6, characterized in that, The on-chip system subtracts the first image from the sixth image to obtain first image data; the on-chip system subtracts the first image from the seventh image to obtain second image data; the first image data and the second image data together constitute the image data.

9. A structured light module with a microcontroller according to claim 1, characterized in that, When the on-chip system transmits the image data sequentially according to the received order, it includes the following steps: Step S501: Send the image data sequentially and receive the return result; if sending fails, proceed to step S502; Step S502: Mark the image data that failed to be sent as the first data to be sent, and send it again; Step S503: Stop sending the image data received after the first data to be sent, store it sequentially according to the receiving time, and mark it as the second data to be sent; Step S504: When the successful transmission signal of the first data to be transmitted is received, the second data to be transmitted is transmitted sequentially, and the successfully transmitted second data to be transmitted is deleted from the storage; Step S505: If the number of times the first data to be sent fails to be sent reaches a first preset value or the capacity of the second data to be sent reaches a second preset value, then stop projecting structured light.

10. A structured light module with a microcontroller according to claim 2, characterized in that, The image data sent by the system-on-chip is a fourth image obtained by processing the first image and the second image.

11. A structured light module with a microcontroller according to claim 3, characterized in that, The image data sent by the system-on-chip is a fourth and a fifth image obtained by processing the first image, the second image, and the third image.

Images