Imaging method and device of structured light camera, equipment and medium

Abstract

The invention discloses an imaging method and device of a structured light camera, equipment and a medium, and belongs to the technical field of optical imaging. The method comprises the following steps: generating a speckle image and a black image, and generating an alternate projection sequence according to the speckle image and the black image; under the condition that the imaging trigger signal is recognized, performing image projection according to an alternate projection sequence, and synchronously controlling the structured light camera to collect at least two continuous frames of target scene images according to the alternate projection sequence; identifying a frame type corresponding to each target scene image; and matching each target scene image according to the frame type corresponding to each target scene image to obtain a differential image pair, and performing differential processing on the differential image pair to obtain a target output image of the structured light camera. According to the technical scheme, on the basis of effectively inhibiting ambient light interference, high-precision synchronization and high-stability operation of projection and acquisition in the imaging process of the structured light camera is realized, and the hardware cost and the system complexity are reduced.

Description

Technical Field

[0001] This application belongs to the field of optical imaging technology, specifically relating to an imaging method, apparatus, device, and medium for a structured light camera. Background Technology

[0002] The working principle of a structured light camera is as follows: it projects known coded patterns such as speckle and stripes onto the scene being measured, uses an image sensor to acquire the pattern modulated by the object's surface, and finally reconstructs the scene's three-dimensional information based on the principle of triangulation. In this process, the structured light projector needs to have high-speed, accurate, and programmable coded pattern generation capabilities. Projection solutions based on Digital Light Processing (DLP) chips have become the mainstream choice in the industry because they meet these core requirements.

[0003] To achieve functions such as ambient light suppression and texture enhancement, existing technologies typically use projection switches to control the projection and shutdown of coded patterns. However, this control method has significant drawbacks in practical deployments, posing serious challenges to the stable operation of the system: First, the pattern switching of DLP chips has a fixed hardware timing sequence. If the camera exposure window cannot be precisely aligned with this timing sequence, it is easy to acquire a mixed image with half a frame containing a pattern and the other half without a pattern, directly leading to the failure of subsequent decoding and matching processes. Second, the switching command can easily introduce uncertain delays or phase drifts in the transmission and control links, destroying the timing determinism between projection and acquisition, further reducing system reliability. Third, the transient switching operation of the light source will cause inter-frame brightness fluctuations. To ensure image quality, additional current stabilization and temperature control modules are required, which not only increases the overall complexity of the system but also significantly increases hardware costs.

[0004] Therefore, how to make the synchronous control of projection and acquisition more stable and simple is a problem that urgently needs to be solved by people in this field. Summary of the Invention

[0005] This application provides an imaging method, apparatus, device, and medium for a structured light camera, aiming to achieve high-precision synchronization and high-stability operation of projection and acquisition during the imaging process of the structured light camera while effectively suppressing ambient light interference, thereby reducing hardware costs and system complexity.

[0006] In a first aspect, embodiments of this application provide an imaging method using a structured light camera, the method comprising: A speckle image and a black image are generated, and an alternating projection sequence is generated based on the speckle image and the black image; wherein the alternating projection sequence includes the speckle image and the black image arranged alternately; Upon detection of an imaging trigger signal, image projection is performed according to the alternating projection sequence, and the structured light camera is synchronously controlled to acquire at least two consecutive frames of target scene images according to the alternating projection sequence. Identify the frame type corresponding to each target scene image; wherein, the frame type includes speckle frames and black frames; Match each target scene image according to the frame type corresponding to each target scene image to obtain a difference image pair, and perform difference processing on the difference image pair to obtain the target output image of the structured light camera.

[0007] Optionally, the step of synchronously controlling the structured light camera to acquire at least two consecutive frames of target scene images according to the alternating projection sequence includes: A frame synchronization signal is generated based on the alternating projection sequence; The frame synchronization signal triggers the image acquisition device in the structured light camera to perform synchronous exposure acquisition, thereby obtaining at least two consecutive frames of target scene images; wherein, the exposure duration of the image acquisition device does not exceed the single-frame stable projection duration of the alternating projection sequence.

[0008] Optionally, identifying the frame type corresponding to each target scene image includes: Differential statistical calculations are performed on the initial two target scene images in each target scene image to obtain differential statistical feature values, and the frame type corresponding to the initial two target scene images is determined based on the differential statistical feature values; The frame type corresponding to the other target scene images is determined based on the timestamps of the other target scene images in each target scene image and the timestamps of the initial two target scene images.

[0009] Optionally, determining the frame type corresponding to the initial two target scene images based on the differential statistical feature values ​​includes: When the differential statistical feature value is greater than zero, the frame type corresponding to the previous initial target scene image is determined to be a speckle frame, and the frame type corresponding to the next initial target scene image is determined to be a black frame. When the differential statistical feature value is less than zero, the frame type corresponding to the previous initial target scene image is determined to be a black frame, and the frame type corresponding to the next initial target scene image is determined to be a speckle frame.

[0010] Optionally, generating the speckle image includes: Generate an initial binary mask image; The initial binary mask image is corrected to obtain a speckle image.

[0011] Optionally, the step of correcting the initial binary mask image to obtain a speckle image includes: Calculate the proportion of binary pixels in the initial binary mask image within a preset sliding window; The pixel colors of the initial binary mask image within a preset sliding window are flipped based on the binary pixel ratio; The preset sliding window is moved according to a preset step size and the above steps are repeated until the preset sliding window has traversed the initial binary mask image.

[0012] Optionally, the step of flipping the pixel colors of the initial binary mask image within a preset sliding window based on the binary pixel ratio includes: When the ratio of binary pixels exceeds a preset ratio threshold, the connected pixel region within the preset sliding window is identified; The pixel colors of the connected pixel regions are flipped in descending order of area until the ratio of the binary pixels reaches a preset ratio threshold.

[0013] Secondly, embodiments of this application provide an imaging apparatus for a structured light camera, the apparatus comprising: A sequence generation module is used to generate speckle images and black images, and to generate an alternating projection sequence based on the speckle images and black images; wherein the alternating projection sequence includes the speckle images and black images arranged alternately; The image acquisition module is used to project images according to the alternating projection sequence when an imaging trigger signal is detected, and to synchronously control the structured light camera to acquire at least two consecutive frames of target scene images according to the alternating projection sequence. A type recognition module is used to identify the frame type corresponding to each target scene image; wherein, the frame type includes speckle frames and black frames; The image output module is used to match each target scene image according to the frame type corresponding to each target scene image to obtain a difference image pair, and to perform difference processing on the difference image pair to obtain the target output image of the structured light camera.

[0014] Optionally, the image acquisition module is specifically used for: A frame synchronization signal is generated based on the alternating projection sequence; The frame synchronization signal triggers the image acquisition device in the structured light camera to perform synchronous exposure acquisition, thereby obtaining at least two consecutive frames of target scene images; wherein, the exposure duration of the image acquisition device does not exceed the single-frame stable projection duration of the alternating projection sequence.

[0015] Optionally, the type recognition module is specifically used for: Differential statistical calculations are performed on the initial two target scene images in each target scene image to obtain differential statistical feature values, and the frame type corresponding to the initial two target scene images is determined based on the differential statistical feature values; The frame type corresponding to the other target scene images is determined based on the timestamps of the other target scene images in each target scene image and the timestamps of the initial two target scene images.

[0016] Optionally, the type recognition module is specifically used for: When the differential statistical feature value is greater than zero, the frame type corresponding to the previous initial target scene image is determined to be a speckle frame, and the frame type corresponding to the next initial target scene image is determined to be a black frame. When the differential statistical feature value is less than zero, the frame type corresponding to the previous initial target scene image is determined to be a black frame, and the frame type corresponding to the next initial target scene image is determined to be a speckle frame.

[0017] Optionally, the sequence generation module is specifically used for: Generate an initial binary mask image; The initial binary mask image is corrected to obtain a speckle image.

[0018] Optionally, the sequence generation module is specifically used for: Calculate the proportion of binary pixels in the initial binary mask image within a preset sliding window; The pixel colors of the initial binary mask image within a preset sliding window are flipped based on the binary pixel ratio; The preset sliding window is moved according to a preset step size and the above steps are repeated until the preset sliding window has traversed the initial binary mask image.

[0019] Optionally, the sequence generation module is specifically used for: When the ratio of binary pixels exceeds a preset ratio threshold, the connected pixel region within the preset sliding window is identified; The pixel colors of the connected pixel regions are flipped in descending order of area until the ratio of the binary pixels reaches a preset ratio threshold.

[0020] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the method described in the first aspect.

[0021] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the method described in the first aspect.

[0022] In this embodiment, a speckle image and a black image are generated, and an alternating projection sequence is generated based on the speckle image and the black image. The alternating projection sequence includes alternating arrangements of the speckle image and the black image. Upon detection of an imaging trigger signal, image projection is performed according to the alternating projection sequence, and the structured light camera is synchronously controlled to acquire at least two consecutive frames of target scene images according to the alternating projection sequence. The frame type corresponding to each target scene image is identified; the frame type includes speckle frames and black frames. Each target scene image is matched according to its corresponding frame type to obtain a difference image pair, and the difference image pair is subjected to differential processing to obtain the target output image of the structured light camera. The above-described structured light camera imaging method, while effectively suppressing ambient light interference, achieves high-precision synchronization and high-stability operation of projection and acquisition during the structured light camera imaging process, reducing hardware costs and system complexity. Attached Figure Description

[0023] Figure 1 This is a schematic flowchart of the imaging method of the structured light camera provided in Embodiment 1 of this application; Figure 2 This is a schematic flowchart of the imaging method of the structured light camera provided in Embodiment 2 of this application; Figure 3 This is a schematic flowchart of the imaging method of the structured light camera provided in Embodiment 3 of this application; Figure 4 This is a schematic diagram of the imaging device of the structured light camera provided in Embodiment 4 of this application; Figure 5 This is a schematic diagram of the structure of the electronic device provided in Embodiment 5 of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, specific embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. It should also be noted that, for ease of description, only the parts relevant to this application are shown in the drawings, not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0025] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0026] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0027] The imaging method, apparatus, device, and medium of the structured light camera provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0028] First, this application is applicable to scenarios requiring low-noise target scene depth information output by a structured light camera, such as high-precision 3D reconstruction, real-time ranging of dynamic targets, industrial defect detection, and human-computer interaction gesture recognition. The structured light camera can be an active imaging device integrating a structured light projection module and an image acquisition module. It projects a pre-defined structured light pattern onto the target scene, acquires the pattern image modulated by the target scene surface, and calculates the 3D spatial information of the target scene using triangulation principles or phase analysis algorithms.

[0029] Example 1 Figure 1 This is a schematic flowchart of the imaging method of the structured light camera provided in Embodiment 1 of this application. Figure 1 As shown, the specific steps include the following: S101, generate a speckle image and a black image, and generate an alternating projection sequence based on the speckle image and the black image; wherein, the alternating projection sequence includes the speckle image and the black image arranged alternately; A speckle image can be a structured pattern composed of randomly distributed bright and dark speckles. One way to generate a speckle image is to randomly generate a binary mask image of a preset size. The binary mask image can be a grayscale image containing only two pixel values ​​(e.g., 0 and 255), where pixel areas with a value of 255 are bright speckles, and pixel areas with a value of 0 are dark speckles.

[0030] A black image can be an image where all pixels have a value of zero (i.e., the color is black). One way to generate a black image is to directly generate a matrix of all zero pixels.

[0031] An alternating projection sequence can be an image sequence arranged in an alternating cyclic pattern of "speckle image - black image - speckle image - black image...". To generate the alternating projection sequence based on the speckle and black images, an image queue can be set up, and the speckle and black images can be arranged cyclically in the order of "speckle image - black image - speckle image - black image" to obtain the alternating projection sequence.

[0032] S102, upon detecting an imaging trigger signal, image projection is performed according to the alternating projection sequence, and the structured light camera is synchronously controlled to acquire at least two consecutive frames of target scene images according to the alternating projection sequence. The imaging trigger signal can be an externally input level trigger signal (such as a high-level pulse generated when the user presses the start imaging button), a timed trigger signal (such as a periodic trigger command generated according to a preset frame rate), an event trigger signal (such as a sensing signal that detects the target scene entering the camera's field of view), or an internal automatic trigger signal of the camera (such as a start signal after completing an initialization).

[0033] Image projection following an alternating projection sequence can be achieved using DLP technology. DLP technology is a digital imaging technique based on a micromirror array, with digital micromirror devices as its core component.

[0034] A continuous frame can be an image frame that is continuously acquired in the time dimension with a fixed frame interval and no dropped frames; the target scene image can be the raw imaging data containing speckle patterns or pure black patterns obtained by the image acquisition device of the structured light camera after exposing the target scene with images projected in an alternating sequence.

[0035] Based on the method of synchronously controlling the structured light camera to acquire at least two consecutive frames of target scene images according to the alternating projection sequence, the frame switching signal of the alternating projection sequence can be used as the acquisition trigger source. When the digital micromirror device switches to a certain frame image and stabilizes, the image acquisition device is immediately triggered to perform exposure acquisition.

[0036] Optionally, the step of synchronously controlling the structured light camera to acquire at least two consecutive frames of target scene images according to the alternating projection sequence includes: A frame synchronization signal is generated based on the alternating projection sequence; The frame synchronization signal triggers the image acquisition device in the structured light camera to perform synchronous exposure acquisition, thereby obtaining at least two consecutive frames of target scene images; wherein, the exposure duration of the image acquisition device does not exceed the single-frame stable projection duration of the alternating projection sequence.

[0037] The frame synchronization signal can be an electrical signal (e.g., a TTL level signal or an LVDS differential signal) that is strictly aligned with the frame switching timing of the alternating projection sequence. The signal is a periodic pulse, with each pulse corresponding to the stable start time of a frame's projection. The rising or falling edge of the pulse serves as the acquisition trigger reference, and it has a fixed pulse width and frame period. Based on the method of generating the frame synchronization signal according to the alternating projection sequence, the duration of a single frame of the alternating projection sequence can be read to generate a pulse signal synchronized with the frame switching action as the frame synchronization signal.

[0038] The image acquisition device can be a CMOS image sensor or a CCD image sensor, integrating an optical lens, signal readout circuit and analog-to-digital conversion module, supporting externally triggered exposure and continuous frame acquisition, and the frame rate can be matched with the frame frequency of the alternating projection sequence; the exposure time of the image acquisition device can be the effective time window from when the photosensitive element in the image acquisition device opens to receive light, until the shutter closes to stop receiving light, which is essentially the duration of time for the photosensitive element to accumulate photogenerated charge.

[0039] By triggering the image acquisition device in the structured light camera to perform synchronous exposure acquisition through a frame synchronization signal, at least two consecutive frames of target scene images can be obtained. This can be achieved by having the image acquisition device immediately start the exposure process after recognizing the frame synchronization signal, controlling the exposure duration through the internal register of the image acquisition device, and automatically completing image storage after the exposure is completed, waiting for the next frame synchronization signal to be triggered.

[0040] The stable projection duration of a single frame in an alternating projection sequence can be the effective duration from when a single frame image (speckle image or black and white image) is switched from the micromirror array to its position and stabilized until the next frame switching command is issued. Its value is equal to the total duration of a single frame in the alternating projection sequence minus the frame switching transition duration.

[0041] The exposure time of the image acquisition device does not exceed the stable projection time of a single frame in the alternating projection sequence, which ensures that the exposure process is completely in the stable projection stage of the micromirror array, avoids image blurring or spot shift caused by the transition state during micromirror switching, and prevents light crosstalk between adjacent frames.

[0042] The advantages of this scheme are twofold: firstly, it achieves precise timing alignment between projection and acquisition through frame synchronization signals, ensuring that each frame of the target scene image can completely record the effective information of the corresponding projection pattern, thus improving the inter-frame consistency of the image; secondly, by limiting the exposure time to no more than the stable projection time of a single frame, it avoids imaging interference and inter-frame crosstalk caused by micromirror switching from a timing perspective.

[0043] S103, identify the frame type corresponding to each target scene image; wherein, the frame type includes speckle frames and black frames; Frame type can be an identifier used to characterize the category of the projected image corresponding to the target scene image, including two core types: speckle frames and black frames. Frame types include speckle frames and black frames. It can be understood that speckle frames correspond to speckle images, and black frames correspond to black images.

[0044] One way to identify the frame type of each target scene image is to read the working status signal of the digital micromirror device. If the digital micromirror device is currently outputting a speckle image driving signal, then the frame type of the corresponding target scene image is a speckle frame; if it is outputting a black image driving signal, then the frame type of the corresponding target scene image is a black frame.

[0045] S104, Match each target scene image according to the frame type corresponding to each target scene image to obtain a difference image pair, and perform difference processing on the difference image pair to obtain the target output image of the structured light camera.

[0046] A differential image pair can be an image pair consisting of a speckle frame and a black frame acquired within the same acquisition time period. The acquisition field of view, exposure parameters, and environmental conditions of the two images are completely identical, with only the projected image differing. Differential image pairs are obtained by matching each target scene image according to the frame type corresponding to each target scene image. One method is to pair adjacent speckle frames and black frames one by one according to the acquisition order.

[0047] The target output image can be a clean speckle image obtained by differential processing the difference image pair, eliminating ambient light background and camera dark noise. This image retains only the reflection information of the target scene surface to the speckle structured light. The method for obtaining the target output image of the structured light camera by differential processing the difference image pair can include direct subtraction of the difference images or normalized differential processing (which can include global normalized differential processing or normalized differential processing for black frames). No specific limitations are imposed on the differential processing method here.

[0048] The formula for direct subtraction is as follows: Where I_diff is the target output image, I_P is the target scene image of the corresponding speckle frame in the difference image pair, I_Black is the target scene image of the corresponding black frame in the difference image pair, and clip() is the clipping (limiting) function.

[0049] The formula for calculating the global normalized difference is as follows: Where I_diff is the target output image, I_P is the target scene image of the corresponding speckle frame in the difference image pair, I_Black is the target scene image of the corresponding black frame in the difference image pair, s is global exposure / gain compensation, and ε is the error regularization term.

[0050] The formula for calculating the normalized difference for black frames is as follows: Where I_diff is the target output image, I_P is the target scene image of the corresponding speckle frame in the difference image pair, I_Black is the target scene image of the corresponding black frame in the difference image pair, s is global exposure / gain compensation, and ε is the error regularization term.

[0051] Specifically, global exposure / gain compensation can be a light intensity signal level calibration coefficient set for the difference in global exposure parameters between speckle frames and black frames; the error regularization term can be a very small positive number introduced to avoid the denominator approaching 0.

[0052] In this embodiment, a speckle image and a black image are generated, and an alternating projection sequence is generated based on the speckle image and the black image. The alternating projection sequence includes alternating arrangements of the speckle image and the black image. Upon detection of an imaging trigger signal, image projection is performed according to the alternating projection sequence, and the structured light camera is synchronously controlled to acquire at least two consecutive frames of target scene images according to the alternating projection sequence. The frame type corresponding to each target scene image is identified; the frame type includes speckle frames and black frames. Each target scene image is matched according to its corresponding frame type to obtain a difference image pair, and the difference image pair is subjected to differential processing to obtain the target output image of the structured light camera. The above-described structured light camera imaging method, while effectively suppressing ambient light interference, achieves high-precision synchronization and high-stability operation of projection and acquisition during the structured light camera imaging process, reducing hardware costs and system complexity.

[0053] Example 2 Figure 2 This is a schematic flowchart of the imaging method of the structured light camera provided in Embodiment 2 of this application. Figure 2 As shown, the specific steps include the following: S201, Generate a speckle image and a black image, and generate an alternating projection sequence based on the speckle image and the black image; wherein, the alternating projection sequence includes the speckle image and the black image arranged alternately; S202, upon detecting an imaging trigger signal, image projection is performed according to the alternating projection sequence, and the structured light camera is synchronously controlled to acquire at least two consecutive frames of target scene images according to the alternating projection sequence; S203, perform differential statistical calculations on the initial two target scene images in each target scene image to obtain differential statistical feature values, and determine the frame type corresponding to the initial two target scene images based on the differential statistical feature values; wherein, the frame type includes speckle frames and black frames; The initial two target scene images can be the first two consecutive target scene images acquired by the structured light camera (i.e., the earliest two target scene images in the acquisition sequence). They are adjacent in time and correspond to the first two frames of the alternating projection sequence. The differential statistical eigenvalues ​​can be quantitative indicators used to characterize the gray-level difference distribution between the two images.

[0054] To obtain the difference statistical feature value, the method of performing difference statistical calculation on the initial two target scene images in each target scene image can be to count the number of bright spots (the number of pixels with a pixel value of 255) in the initial two target scene images in each target scene image, and then calculate the difference between the number of bright spots in the initial two target scene images to obtain the difference statistical feature value.

[0055] The method of determining the frame type corresponding to the two initial target scene images based on the differential statistical feature value can be as follows: if the differential statistical feature value is the difference in the number of bright spots and it is greater than zero, the frame type corresponding to the first initial target scene image is determined to be a speckle frame and the frame type corresponding to the second initial target scene image is determined to be a black frame; if the differential statistical feature value is the difference in the number of bright spots and it is less than zero, the frame type corresponding to the first initial target scene image is determined to be a black frame and the frame type corresponding to the second initial target scene image is determined to be a speckle frame.

[0056] Optionally, the differential statistical feature values ​​determine the frame type corresponding to the initial two target scene images, including: When the differential statistical feature value is greater than zero, the frame type corresponding to the previous initial target scene image is determined to be a speckle frame, and the frame type corresponding to the next initial target scene image is determined to be a black frame. When the differential statistical feature value is less than zero, the frame type corresponding to the previous initial target scene image is determined to be a black frame, and the frame type corresponding to the next initial target scene image is determined to be a speckle frame.

[0057] When the differential statistical eigenvalue is greater than zero, that is, the pixel brightness of the previous initial target scene image is higher than that of the next initial target scene image (the brightness of the speckle frame is higher than that of the black frame), the frame type corresponding to the previous initial target scene image is determined to be a speckle frame, and the frame type corresponding to the next initial target scene image is determined to be a black frame.

[0058] When the differential statistical eigenvalue is less than zero, that is, the pixel brightness of the previous initial target scene image is lower than that of the next initial target scene image (the brightness of the corresponding black frame is lower than that of the speckle frame), the frame type corresponding to the previous initial target scene image is determined to be a black frame, and the frame type corresponding to the next initial target scene image is determined to be a speckle frame.

[0059] The advantage of this scheme is that it directly correlates the brightness difference between speckle frames and black frames by the positive and negative values ​​of the differential statistical feature values, without the need for additional threshold settings or complex calculations, thus achieving a fast and accurate determination of the initial two frame types.

[0060] S204, determine the frame type corresponding to the other target scene images based on the timestamps of the other target scene images in each target scene image and the timestamps of the initial two target scene images; A timestamp can be a time identifier that is synchronously recorded by the image acquisition device when acquiring each frame of the target scene image, and is used to characterize the order in which the images are acquired.

[0061] The method for determining the frame type of other target scene images based on the timestamps of other target scene images in each target scene image and the timestamps of the initial two target scene images can be as follows: For each target scene image, calculate the duration difference between the timestamp of the other target scene image and the timestamp of the initial first target scene image, divide the duration difference by the duration of a single frame, round the result of the division, and divide by 2 to obtain the final calculation result. If the final calculation result is an integer, it means that the frame type of the other target scene image is the same as the frame type of the initial first target scene image. If the final calculation result is not an integer, it means that the frame type of the other target scene image is the same as the frame type of the initial second target scene image.

[0062] Accordingly, based on the timestamps of other target scene images in each target scene image and the timestamps of the initial two target scene images, the formula for determining the frame type corresponding to other target scene images is as follows: Where, round() means rounding, t is the timestamp of other target scene images, t_ref is the timestamp of the initial first target scene image, and T is the duration of a single frame; if k / 2 is an integer, it means that the frame type corresponding to the other target scene image is the same as the frame type of the initial first target scene image, and if k / 2 is not an integer, it means that the frame type corresponding to the other target scene image is the same as the frame type of the initial second target scene image.

[0063] S205, Match each target scene image according to the frame type corresponding to each target scene image to obtain a difference image pair, and perform difference processing on the difference image pair to obtain the target output image of the structured light camera.

[0064] The advantage of this scheme is that the frame type baseline can be determined by performing differential statistics on the initial two frames. The frame type of other images can then be directly derived from the timestamps. This eliminates the need to repeatedly perform feature calculations on each frame, significantly reducing the computational load for frame type identification and improving the real-time performance of the imaging process.

[0065] Example 3 Figure 3 This is a schematic flowchart of the imaging method of the structured light camera provided in Embodiment 3 of this application. Figure 3 As shown, the specific steps include the following: S301, Generate the initial binary mask image; The initial binary mask image can be a randomly generated binary mask image of a preset size (N×N).

[0066] One way to generate the initial binary mask image is to create a blank image region of a preset size (N×N), divide the blank image region into grids to obtain block units, and use a random number generator to randomly assign pixel values ​​of 0 or 255 to the pixels in each block unit to obtain the initial binary mask image.

[0067] S302, The initial binary mask image is corrected to obtain a speckle image; To obtain a speckle image by correcting the initial binary mask image, the initial binary mask image can be divided into multiple non-overlapping sub-regions. For each sub-region, the proportion of bright spots to dark spots is counted. If the proportion is not within a preset range, the pixel values ​​within the sub-region are randomly flipped at a preset ratio until the proportion is within the preset range, thus obtaining the speckle image.

[0068] Optionally, the step of correcting the initial binary mask image to obtain a speckle image includes: Calculate the proportion of binary pixels in the initial binary mask image within a preset sliding window; The pixel colors of the initial binary mask image within a preset sliding window are flipped based on the binary pixel ratio; The preset sliding window is moved according to a preset step size and the above steps are repeated until the preset sliding window has traversed the initial binary mask image.

[0069] The preset sliding window can be a rectangular area with a preset size (M×M, M≤N); the binary pixel ratio can be the percentage of bright spots (pixels with a pixel value of 255) or dark spots (pixels with a pixel value of 0) within the preset sliding window out of the total number of pixels in the preset sliding window. The binary pixel ratio of the initial binary mask image within the preset sliding window can be calculated by traversing all pixels covered by the preset sliding window, counting the number of pixels with a pixel value of 255 (or counting the number of pixels with a pixel value of 0), and dividing this number by the total number of pixels in the preset sliding window to obtain the binary pixel ratio.

[0070] The method of flipping the pixel color of the initial binary mask image within a preset sliding window based on the binary pixel ratio can be achieved by randomly dividing the pixel region in the preset sliding window into multiple non-overlapping sub-regions when the binary pixel ratio exceeds a preset ratio threshold, and randomly flipping the pixel value of each sub-region by a preset ratio.

[0071] Optionally, the step of flipping the pixel colors of the initial binary mask image within a preset sliding window based on the binary pixel ratio includes: When the ratio of binary pixels exceeds a preset ratio threshold, the connected pixel region within the preset sliding window is identified; The pixel colors of the connected pixel regions are flipped in descending order of area until the ratio of the binary pixels reaches a preset ratio threshold.

[0072] The preset ratio threshold can be an upper limit of the binary pixel ratio set based on the uniformity requirements of the speckle image, such as 60%.

[0073] A connected pixel region can be a set of pixels with the same pixel value and adjacent positions within a preset sliding window. Specifically, if the binary pixel ratio is the proportion of bright spots, then the connected pixel region is a connected pixel region where all pixels have a value of 255; if the binary pixel ratio is the proportion of dark spots, then the connected pixel region is a connected pixel region where all pixels have a value of 0.

[0074] One way to identify connected pixel regions within a preset sliding window is to traverse all pixels within the preset sliding window and, for each unmarked target pixel, search using a 4-neighbor or 8-neighbor method to mark all adjacent pixels with the same value as the same connected pixel region.

[0075] The pixel colors of connected pixel regions are flipped in descending order of their area until the binary pixel ratio reaches a preset threshold. This can be achieved by sorting all identified bright spot connected pixel regions (or dark spot connected pixel regions) by area from largest to smallest, generating a queue of bright spot connected pixel regions (or dark spot connected pixel regions). Bright spot connected pixel regions (or dark spot connected pixel regions) are selected sequentially from the head of the queue, and all pixel values ​​within the region are flipped (updated from 255 to 0 or from 0 to 255). After each pair of bright spot connected pixel regions (or dark spot connected pixel regions) is flipped, the binary pixel ratio is recalculated. If the binary pixel ratio reaches the preset threshold, the flipping process is terminated. If the binary pixel ratio still exceeds the preset threshold, the next bright spot connected pixel region (or dark spot connected pixel region) is flipped.

[0076] The advantage of this scheme is that it prioritizes flipping large areas of connected pixels, which can quickly adjust the proportion of binary pixels to the threshold range, improving the correction efficiency. At the same time, it avoids over-flipping of small-area speckle patterns, ensuring the randomness of the speckle pattern and the integrity of texture features. The resulting speckle image conforms to the Bernoulli distribution and has better matching accuracy and anti-interference ability.

[0077] The preset step size can be the distance that the preset sliding window moves each time it moves across the initial binary mask image. The preset sliding window completes its traversal of the initial binary mask image if, during one complete traversal, the proportion of binary pixels within the preset sliding window does not exceed a preset proportion threshold; otherwise, the traversal needs to be repeated.

[0078] The speckle image obtained after correction conforms to the Bernoulli distribution.

[0079] The advantage of this scheme is that it ensures the texture coverage of the target surface after projection by using a uniform ratio of light and dark, suppresses the aggregation of the same color and avoids large areas of low recognition, and ultimately improves the feature matching accuracy of speckle images.

[0080] S303, Generate a black image, and generate an alternating projection sequence based on the speckle image and the black image; wherein, the alternating projection sequence includes alternating arrangements of the speckle image and the black image; S304, upon detecting an imaging trigger signal, image projection is performed according to the alternating projection sequence, and the structured light camera is synchronously controlled to acquire at least two consecutive frames of target scene images according to the alternating projection sequence; S305, Identify the frame type corresponding to each target scene image; wherein, the frame type includes speckle frames and black frames; S306, Match each target scene image according to the frame type corresponding to each target scene image to obtain a difference image pair, and perform difference processing on the difference image pair to obtain the target output image of the structured light camera.

[0081] The advantage of this scheme is that by correcting the initial binary mask image, the distortion and brightness unevenness caused by the optical system and projection device are eliminated, thus improving the projection accuracy of the speckle image. The subsequent combination of alternating projection and differential processing further ensures the quality of the target output image, ultimately providing more accurate speckle feature information for applications such as 3D reconstruction.

[0082] Example 4 Figure 4 This is a schematic diagram of the imaging device of the structured light camera provided in Embodiment 4 of this application. Figure 4 As shown, the device includes: The sequence generation module 410 is used to generate a speckle image and a black image, and to generate an alternating projection sequence based on the speckle image and the black image; wherein, the alternating projection sequence includes the speckle image and the black image arranged alternately; The image acquisition module 420 is used to project an image according to the alternating projection sequence when an imaging trigger signal is detected, and to synchronously control the structured light camera to acquire at least two consecutive frames of target scene images according to the alternating projection sequence. The type recognition module 430 is used to identify the frame type corresponding to each target scene image; wherein, the frame type includes speckle frames and black frames; The image output module 440 is used to match each target scene image according to the frame type corresponding to each target scene image to obtain a difference image pair, and to perform difference processing on the difference image pair to obtain the target output image of the structured light camera.

[0083] Optionally, the image acquisition module 420 is specifically used for: A frame synchronization signal is generated based on the alternating projection sequence; The frame synchronization signal triggers the image acquisition device in the structured light camera to perform synchronous exposure acquisition, thereby obtaining at least two consecutive frames of target scene images; wherein, the exposure duration of the image acquisition device does not exceed the single-frame stable projection duration of the alternating projection sequence.

[0084] Optionally, the type recognition module 430 is specifically used for: Differential statistical calculations are performed on the initial two target scene images in each target scene image to obtain differential statistical feature values, and the frame type corresponding to the initial two target scene images is determined based on the differential statistical feature values; The frame type corresponding to the other target scene images is determined based on the timestamps of the other target scene images in each target scene image and the timestamps of the initial two target scene images.

[0085] Optionally, the type recognition module 430 is specifically used for: When the differential statistical feature value is greater than zero, the frame type corresponding to the previous initial target scene image is determined to be a speckle frame, and the frame type corresponding to the next initial target scene image is determined to be a black frame. When the differential statistical feature value is less than zero, the frame type corresponding to the previous initial target scene image is determined to be a black frame, and the frame type corresponding to the next initial target scene image is determined to be a speckle frame.

[0086] Optionally, the sequence generation module 410 is specifically used for: Generate an initial binary mask image; The initial binary mask image is corrected to obtain a speckle image.

[0087] Optionally, the sequence generation module is specifically used for: Calculate the proportion of binary pixels in the initial binary mask image within a preset sliding window; The pixel colors of the initial binary mask image within a preset sliding window are flipped based on the binary pixel ratio; The preset sliding window is moved according to a preset step size and the above steps are repeated until the preset sliding window has traversed the initial binary mask image.

[0088] Optionally, the sequence generation module 410 is specifically used for: When the ratio of binary pixels exceeds a preset ratio threshold, the connected pixel region within the preset sliding window is identified; The pixel colors of the connected pixel regions are flipped in descending order of area until the ratio of the binary pixels reaches a preset ratio threshold.

[0089] In this embodiment, a sequence generation module is used to generate speckle images and black images, and to generate an alternating projection sequence based on the speckle images and black images; wherein the alternating projection sequence includes alternating arrangements of the speckle images and the black images; an image acquisition module is used to project images according to the alternating projection sequence when an imaging trigger signal is detected, and to synchronously control the structured light camera to acquire at least two consecutive frames of target scene images according to the alternating projection sequence; a type identification module is used to identify the frame type corresponding to each target scene image; wherein the frame type includes speckle frames and black frames; an image output module is used to match each target scene image according to the frame type corresponding to each target scene image to obtain differential image pairs, and to perform differential processing on the differential image pairs to obtain the target output image of the structured light camera. The above-described structured light camera imaging device, while effectively suppressing ambient light interference, achieves high-precision synchronization and high-stability operation of projection and acquisition during the structured light camera imaging process, reducing hardware costs and system complexity.

[0090] The imaging device of the structured light camera in this application embodiment can be a device, or it can be a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not specifically limit the scope.

[0091] The imaging device of the structured light camera in this embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this embodiment does not specifically limit the specific operating system used.

[0092] The imaging device of the structured light camera provided in this application embodiment can realize the various processes implemented in the above embodiments. To avoid repetition, it will not be described again here.

[0093] Example 5 like Figure 5 As shown, this application embodiment also provides an electronic device 500, including a processor 501, a memory 502, and a program or instructions stored in the memory 502 and executable on the processor 501. When the program or instructions are executed by the processor 501, they implement the various processes of the above-described structured light camera imaging method embodiment and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0094] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0095] Example 6 This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described imaging method embodiments of the structured light camera and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0096] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0097] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0098] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0099] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0100] The above description is merely a preferred embodiment and the technical principles employed in this application. This application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions that can be made by those skilled in the art will not depart from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of this application, the scope of which is determined by the scope of the claims.

Claims

1. An imaging method for a structured light camera, characterized in that, The method includes: A speckle image and a black image are generated, and an alternating projection sequence is generated based on the speckle image and the black image; wherein the alternating projection sequence includes the speckle image and the black image arranged alternately; Upon detection of an imaging trigger signal, image projection is performed according to the alternating projection sequence, and the structured light camera is synchronously controlled to acquire at least two consecutive frames of target scene images according to the alternating projection sequence. Identify the frame type corresponding to each target scene image; wherein, the frame type includes speckle frames and black frames; Match each target scene image according to the frame type corresponding to each target scene image to obtain a difference image pair, and perform difference processing on the difference image pair to obtain the target output image of the structured light camera.

2. The imaging method of the structured light camera according to claim 1, characterized in that, The step of synchronously controlling the structured light camera to acquire at least two consecutive frames of target scene images according to the alternating projection sequence includes: A frame synchronization signal is generated based on the alternating projection sequence; The frame synchronization signal triggers the image acquisition device in the structured light camera to perform synchronous exposure acquisition, thereby obtaining at least two consecutive frames of target scene images; wherein, the exposure duration of the image acquisition device does not exceed the single-frame stable projection duration of the alternating projection sequence.

3. The imaging method of the structured light camera according to claim 1, characterized in that, The identification of the frame type corresponding to each target scene image includes: Differential statistical calculations are performed on the initial two target scene images in each target scene image to obtain differential statistical feature values, and the frame type corresponding to the initial two target scene images is determined based on the differential statistical feature values; The frame type corresponding to the other target scene images is determined based on the timestamps of the other target scene images in each target scene image and the timestamps of the initial two target scene images.

4. The imaging method of the structured light camera according to claim 3, characterized in that, Determining the frame type corresponding to the initial two target scene images based on the differential statistical feature values ​​includes: When the differential statistical feature value is greater than zero, the frame type corresponding to the previous initial target scene image is determined to be a speckle frame, and the frame type corresponding to the next initial target scene image is determined to be a black frame. When the differential statistical feature value is less than zero, the frame type corresponding to the previous initial target scene image is determined to be a black frame, and the frame type corresponding to the next initial target scene image is determined to be a speckle frame.

5. The imaging method of the structured light camera according to claim 1, characterized in that, The generation of the speckle image includes: Generate an initial binary mask image; The initial binary mask image is corrected to obtain a speckle image.

6. The imaging method of the structured light camera according to claim 5, characterized in that, The step of correcting the initial binary mask image to obtain a speckle image includes: Calculate the proportion of binary pixels in the initial binary mask image within a preset sliding window; The pixel colors of the initial binary mask image within a preset sliding window are flipped based on the binary pixel ratio; The preset sliding window is moved according to a preset step size and the above steps are repeated until the preset sliding window has traversed the initial binary mask image.

7. The imaging method of the structured light camera according to claim 6, characterized in that, The step of flipping the pixel colors of the initial binary mask image within a preset sliding window based on the binary pixel ratio includes: When the ratio of binary pixels exceeds a preset ratio threshold, the connected pixel region within the preset sliding window is identified; The pixel colors of the connected pixel regions are flipped in descending order of area until the ratio of the binary pixels reaches a preset ratio threshold.

8. An imaging device for a structured light camera, characterized in that, The device includes: A sequence generation module is used to generate speckle images and black images, and to generate an alternating projection sequence based on the speckle images and black images; wherein the alternating projection sequence includes the speckle images and black images arranged alternately; The image acquisition module is used to project images according to the alternating projection sequence when an imaging trigger signal is detected, and to synchronously control the structured light camera to acquire at least two consecutive frames of target scene images according to the alternating projection sequence. A type recognition module is used to identify the frame type corresponding to each target scene image; wherein, the frame type includes speckle frames and black frames; The image output module is used to match each target scene image according to the frame type corresponding to each target scene image to obtain a difference image pair, and to perform difference processing on the difference image pair to obtain the target output image of the structured light camera.

9. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the imaging method of the structured light camera as described in any one of claims 1-7.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the imaging method of the structured light camera as described in any one of claims 1-7.

Images