Structured light camera parameter correction method, device, storage medium and program product

Abstract

Embodiments of the present application provide a structured light camera parameter correction method, device, storage medium and program product. The method corrects the light source output intensity of the structured light camera based on the collected image when the brightness difference degree of the image collected by the structured light camera is greater than or equal to a brightness difference degree threshold, to compensate for the brightness of the underexposed area and / or suppress the brightness of the overexposed area; until the brightness difference degree of the image collected by the structured light camera is less than the brightness difference degree threshold, the corrected light source output intensity is determined as the final light source output intensity, which can supplement the effective signal intensity of the edge and both sides of the field of view, suppress the effective signal intensity of the center area of the field of view, make the brightness values in the full field of view range consistent, solve the problem of uneven brightness distribution of the field of view, and improve the imaging integrity while meeting the timeliness requirements of industrial real-time imaging.

Description

Technical Field

[0001] This application relates to the field of structured light three-dimensional imaging technology, and in particular to a structured light camera parameter correction method, device, storage medium and program product. Background Technology

[0002] Structured light imaging systems project specific patterns using point light sources, combine them with reflected images acquired by binocular cameras, and reconstruct 3D point clouds using triangulation principles. Although this system has advantages such as simple structure and controllable cost, it generally suffers from uneven brightness distribution in practical applications, characterized by "bright center and dark periphery." This results in the central area of ​​the field of view being prone to overexposure, while the edges and sides are prone to underexposure.

[0003] The aforementioned uneven brightness issue directly leads to serious potential problems in engineering applications: In scenarios involving highly reflective materials, the image brightness on both sides of the field of view is significantly lower than that in the center due to the above reasons. If overexposure is prevented in the center of the field of view, signal loss is likely to occur due to underexposure on both sides. Conversely, if underexposure is prevented on both sides of the camera's field of view, overexposure is likely to occur in the center. Summary of the Invention

[0004] This application provides a method, device, storage medium, and program product for correcting the parameters of a structured light camera, in order to solve the problem of uneven brightness distribution in the field of view in the prior art.

[0005] In a first aspect, embodiments of this application provide a method for correcting parameters of a structured light camera, including:

[0006] Acquire the initial image captured by the structured light camera;

[0007] If the brightness difference of the initial image is greater than or equal to the brightness difference threshold, the light source output intensity of the structured light camera is corrected based on the initial image to compensate for the brightness of underexposed areas and / or suppress the brightness of overexposed areas.

[0008] Acquire real-time images from the structured light camera after the correction;

[0009] If the brightness difference of the real-time image is greater than or equal to the brightness difference threshold, the light source output intensity of the structured light camera is corrected based on the real-time image.

[0010] The corrected light source output intensity is determined as the final light source output intensity until the brightness difference of the real-time image acquired by the structured light camera is less than the brightness difference threshold.

[0011] In one possible implementation, the light source output intensity of the structured light camera is corrected based on the image acquired by the structured light camera, including:

[0012] Based on the brightness value and average brightness of each pixel in the acquired image, determine the brightness compensation value of each pixel;

[0013] The light source output intensity of the structured light camera is adjusted according to the brightness compensation value of each pixel in order to perform brightness correction on the field of view of the structured light camera.

[0014] The images acquired by the structured light camera include the initial image and the real-time image.

[0015] In one possible implementation, after adjusting the light source output intensity of the structured light camera according to the brightness compensation value of each pixel, the method further includes:

[0016] The output intensity of the light source after adjustment by the structured light camera is smoothed.

[0017] In one possible implementation, determining the brightness compensation value for each pixel based on the brightness value and average brightness value of each pixel in the acquired image includes:

[0018] The brightness values ​​of each pixel in the acquired image are extracted to form a brightness sequence;

[0019] The average brightness of the acquired image is determined based on the brightness sequence.

[0020] In one possible implementation, determining the average brightness of the acquired image based on the brightness sequence includes:

[0021] The brightness sequence is filtered to reduce noise or outliers in the brightness sequence;

[0022] The average brightness of the acquired image is determined based on the filtered brightness sequence.

[0023] In one possible implementation, after acquiring the real-time image captured by the structured light camera after correction, and before determining that the brightness difference of the real-time image is greater than or equal to a brightness difference threshold, the method further includes:

[0024] Determine the horizontal brightness difference of the entire field of view of the real-time image, and use the horizontal brightness difference of the entire field of view of the real-time image as the brightness difference of the real-time image.

[0025] Determine whether the brightness difference of the real-time image is greater than or equal to the brightness difference threshold.

[0026] In one possible implementation, the structured light camera includes a projection unit and a photosensitive unit, and acquiring the initial image captured by the structured light camera includes:

[0027] The imaging parameters of the structured light camera are set according to the preset imaging parameters;

[0028] The projection unit is controlled to project a pattern onto a standard grayscale plate placed at a standard working distance, and the photosensitive unit is controlled to collect the projected pattern to obtain the initial image captured by the structured light camera.

[0029] In one possible implementation, after acquiring the real-time image captured by the structured light camera after correction, the method further includes:

[0030] Generate the lateral brightness distribution curve of the field of view of the real-time image;

[0031] And / or,

[0032] A comparison diagram of the lateral brightness distribution curves of the images acquired before and after the correction is generated.

[0033] Secondly, embodiments of this application provide a structured light camera parameter correction device, including: a memory and a processor;

[0034] The memory stores computer-executed instructions;

[0035] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0036] Thirdly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0037] Fourthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0038] The structured light camera parameter correction method, device, storage medium, and program product provided in this application embodiment are as follows: When the brightness difference of the initial image acquired by the structured light camera is greater than or equal to a brightness difference threshold, the method corrects the light source output intensity of the structured light camera based on the initial image to compensate for the brightness of underexposed areas and / or suppress the brightness of overexposed areas; it acquires a real-time image acquired by the structured light camera after correction; if the brightness difference of the real-time image acquired after correction is still greater than or equal to the brightness difference threshold, the method corrects the light source output intensity of the structured light camera based on the real-time image; this process continues until the brightness difference of the real-time image acquired by the structured light camera is less than the brightness difference threshold, and the corrected light source output intensity is determined as the final light source output intensity. The solution in this embodiment can dynamically correct the output intensity of the light source of the structured light camera based on the brightness difference of the real-time image acquired by the structured light camera (reflecting the uniformity of the field of view brightness). This allows for targeted supplementation of the effective signal intensity in the edge and lateral regions of the field of view (i.e., underexposed regions) and suppression of the effective signal intensity in the central region of the field of view (i.e., overexposed regions). This makes the brightness value in the entire field of view more consistent, solving the problem of uneven brightness distribution in the field of view. It improves the integrity of the image while meeting the timeliness requirements of industrial real-time imaging. Attached Figure Description

[0039] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0040] Figure 1 This is a schematic diagram illustrating an application scenario of the structured light camera parameter correction method provided in an embodiment of this application;

[0041] Figure 2 A schematic flowchart of a structured light camera parameter correction method provided for an exemplary embodiment of this application;

[0042] Figure 3 This is an example diagram comparing the lateral brightness distribution curves of the images acquired before and after correction in this embodiment.

[0043] Figure 4 A schematic diagram of the overall flow of a structured light camera parameter correction method provided for an exemplary embodiment of this application;

[0044] Figure 5 An example image of a point cloud of a reflective object captured in a single exposure without correction, provided for an embodiment of this application;

[0045] Figure 6 An example image of a point cloud of a reflective object under a single exposure, acquired after correction, provided for an embodiment of this application;

[0046] Figure 7 A schematic diagram of the structure for the parameter correction of the structured light camera provided in this application.

[0047] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0048] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0049] First, let me explain the terms used in this application:

[0050] Structured light camera: A device used to measure the three-dimensional features of an object, also called a structured light 3D camera. It includes an optical engine and a camera that cooperate with each other and are fixed in position. The optical engine projects a grating stripe pattern onto the surface of the object being measured, and then the camera takes pictures of the pattern on the surface of the object. Based on pre-encoded rules, the captured pattern data is decoded and processed to obtain a high-precision three-dimensional (3D) point cloud of the object being measured.

[0051] Digital galvanometer (DMD): A digital micromirror device is a component that makes up an optical engine lens. The optical engine lens contains a large number of digital galvanometers, which are usually rectangular. Each digital galvanometer can correspond to one or more pixels. The mirror surface of the digital galvanometer is connected to the optical engine through a hinge axis and can rotate around the hinge axis. By rotating the mirror surface of the digital galvanometer around the hinge axis, the mirror angle is changed to achieve rotation relative to the optical engine itself. By changing the mirror angle of the digital galvanometer, the shape of the pattern projected by the optical engine is changed.

[0052] Optical axis: A theoretical axis of symmetry in an optical system. For a single lens or lens group, it is a straight line passing through the center of curvature of all the lens surfaces. In a camera system, the optical axis typically passes through the center of the lens, is perpendicular to the image sensor plane, and ultimately lies in the central region of the sensor.

[0053] On-axis rays: These are rays whose direction of propagation is parallel to or coincides with the optical axis. In camera systems, these rays originate from objects at the center of the field of view and are incident almost perpendicularly on the front of the lens and the sensor surface.

[0054] Off-axis rays: These are rays that form a certain angle with the optical axis. In camera systems, these rays originate from objects at the edge of the field of view and are incident at an oblique angle.

[0055] Structured light imaging systems project specific patterns using point light sources, combine them with reflected images acquired by binocular cameras, and reconstruct 3D point clouds using triangulation principles. Although this system has advantages such as simple structure and controllable cost, it generally suffers from uneven brightness distribution in practical applications, characterized by "bright center and dark periphery." This results in the central area of ​​the field of view being prone to overexposure, while the edges and sides are prone to underexposure.

[0056] The core factors contributing to this phenomenon include the following three points:

[0057] 1. Inherent attenuation of the optical system: According to the fourth cosine law of the camera lens, the transmission efficiency of on-axis rays (center of the field of view) is significantly higher than that of off-axis rays (both sides of the field of view), resulting in severe natural gradient attenuation of light energy as it propagates to the edge of the field of view.

[0058] 2. Projection characteristics of structured light point source: The core energy of structured light point source is concentrated at the center of the optical axis. As the angle of deviation from the center increases, the light intensity gradually decreases, which directly leads to insufficient light illumination energy on both sides of the field of view.

[0059] 3. Influence of light reflection direction: The signal energy reflected back to the camera from the central area of ​​the field of view is concentrated, while the light energy from the edges and sides is mostly reflected outside the camera's light-sensing range, resulting in weak reflected signals and obvious differences in brightness.

[0060] The aforementioned uneven brightness issue directly leads to serious potential problems in engineering applications: In scenarios involving highly reflective materials, the image brightness on both sides of the field of view is significantly lower than that in the center due to the above reasons. If overexposure is prevented in the center of the field of view, signal loss is likely to occur due to underexposure on both sides. Conversely, if underexposure is prevented on both sides of the camera's field of view, overexposure is likely to occur in the center.

[0061] The existing solution is to use multiple exposures to superimpose and compensate for the signal loss in the areas to obtain a complete image. However, this method requires multiple exposures, which increases the overall scanning time exponentially and cannot meet the timeliness requirements of industrial real-time imaging.

[0062] To address the aforementioned problem of uneven brightness distribution in the field of view, this application provides a method for correcting parameters of a structured light camera. For an initial image acquired by the structured light camera, if the brightness difference of the initial image is greater than or equal to a brightness difference threshold, the light source output intensity of the structured light camera is corrected based on the initial image to compensate for the brightness of underexposed areas and / or suppress the brightness of overexposed areas. A real-time image acquired by the structured light camera after correction is then obtained. If the brightness difference of the real-time image acquired after correction is still greater than or equal to the brightness difference threshold, the light source output intensity of the structured light camera is corrected based on the real-time image. This process is repeated until the brightness difference of the real-time image acquired by the structured light camera is less than the brightness difference threshold, at which point the corrected light source output intensity is determined as the final light source output intensity. The solution in this embodiment can dynamically correct the output intensity of the light source of the structured light camera based on the brightness difference of the real-time image acquired by the structured light camera (reflecting the uniformity of the field of view brightness). This allows for targeted supplementation of the effective signal intensity in the edge and lateral regions of the field of view (i.e., underexposed regions) and suppression of the effective signal intensity in the central region of the field of view (i.e., overexposed regions). This makes the brightness value in the entire field of view more consistent, solving the problem of uneven brightness distribution in the field of view. It improves the integrity of the image while meeting the timeliness requirements of industrial real-time imaging.

[0063] The solution in this embodiment can be applied to parameter correction of oscillating line scan structured light cameras and area structured light cameras.

[0064] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0065] Figure 1 This is a schematic diagram illustrating an application scenario of the structured light camera parameter correction method provided in an embodiment of this application. For example... Figure 1 As shown, in this application scenario, the grayscale board is placed at the standard working distance of the structured light camera, and the ambient light intensity is set to the level of a typical industrial scene, such as 300-1500 lx (lux), to simulate a typical industrial environment. The grayscale board can be a standard grayscale board commonly used for camera calibration, such as an 18% standard grayscale board. The standard working distance of the structured light camera is within its operating range and can be set according to the specific application scenario.

[0066] Figure 2 This is a schematic flowchart illustrating a structured light camera parameter correction method provided as an exemplary embodiment of this application. Figure 2 As shown, the method specifically includes the following steps:

[0067] S201. Acquire the initial image captured by the structured light camera.

[0068] The structured light camera can be a laser line-scanning structured light camera or a surface structured light camera. Based on Figure 1 The scene shown uses a structured light camera to capture images of a grayscale plate. The image captured using the structured light camera before parameter calibration is called the initial image.

[0069] Before acquiring the initial image using a structured light camera, adjust the initial imaging parameters of the structured light camera (such as adjusting the exposure time of the camera's photosensitive unit) to ensure that the initial image acquired by the structured light camera based on the initial imaging parameters is not overexposed. The initial imaging parameters can be set according to the actual application scenario, as long as the initial image is not overexposed; no specific limitations are imposed here.

[0070] For example, the imaging parameters of the structured light camera are set according to the preset imaging parameters; the projection unit is controlled to project a pattern onto a standard grayscale plate placed at a standard working distance, and the photosensitive unit of the structured light camera is controlled to collect the projected pattern to obtain the initial image collected by the structured light camera.

[0071] Taking a structured light camera as an example, the initial imaging parameters of the structured light camera are set according to preset imaging parameters. Based on the initial imaging parameters, the structured light camera is controlled to project and acquire a fully bright image onto a grayscale plate. The acquired fully bright image is used as the initial image.

[0072] Taking a oscillating line-scan structured light camera as an example, the initial imaging parameters of the oscillating line-scan structured light camera are set according to preset imaging parameters. Based on the initial imaging parameters, the oscillating line-scan structured light camera is controlled to project and acquire laser line images at different positions in the field of view. The brightness of the center of the laser line in the laser line image is extracted, and a fully bright image is generated by stitching together the brightness of the center of the laser line at different positions in the field of view. This fully bright image is used as the initial image.

[0073] S202. If the brightness difference of the initial image is greater than or equal to the brightness difference threshold, the light source output intensity of the structured light camera is corrected based on the initial image to compensate for the brightness of underexposed areas and / or suppress the brightness of overexposed areas.

[0074] In this embodiment, the convergence condition for image brightness uniformity is set as the brightness difference of the image being less than a preset brightness difference threshold. The brightness difference threshold can be configured and adjusted according to actual application requirements; for example, it can be set to 30%, but this is not specifically limited here.

[0075] Optionally, the brightness difference of the image can be the horizontal brightness difference across the entire field of view, which can be expressed as D = ((MAX - MIN) / MAX) × 100%. Here, D represents the brightness difference of the image, MAX represents the maximum horizontal brightness value across the entire field of view, and MIN represents the minimum horizontal brightness value across the entire field of view. Taking a brightness difference threshold of 30% as an example, the convergence condition for image brightness uniformity is: ((MAX - MIN) / MAX) × 100% < 30%.

[0076] For example, the horizontal brightness difference of the entire field of view of the initial image is determined, and the horizontal brightness difference of the entire field of view of the initial image is used as the brightness difference of the initial image; it is determined whether the brightness difference of the initial image is greater than or equal to the brightness difference threshold, that is, whether the initial image satisfies the image brightness uniformity convergence condition.

[0077] Optionally, the brightness difference of the image can be the brightness difference across the entire field of view of the image, which can be expressed as D = ((MAX'-MIN') / MAX')×100%. Where MAX' represents the maximum brightness value within the entire field of view of the image, and MIN' represents the minimum brightness value within the entire field of view of the image.

[0078] For example, the brightness difference across the entire field of view of the initial image is determined, and this brightness difference is used as the brightness difference of the initial image. It is then determined whether the brightness difference of the initial image is greater than or equal to a brightness difference threshold, i.e., whether the initial image satisfies the image brightness uniformity convergence condition.

[0079] If the brightness difference of the initial image is greater than or equal to the brightness difference threshold, it indicates that the brightness difference within the field of view of the initial image is large, and there is a problem of uneven brightness in the field of view. Therefore, the output intensity of the light source of the structured light camera needs to be corrected.

[0080] In this embodiment, when correcting the light source output intensity of the structured light camera, the initial image is used as the reference image for correction. The light source output intensity of the structured light camera is corrected based on the initial image to compensate for the brightness of underexposed areas in the initial image and / or suppress the brightness of overexposed areas in the initial image, thereby compensating for the brightness of underexposed areas in the field of view and suppressing the brightness of overexposed areas in the field of view, making the brightness of the field of view more uniform.

[0081] For example, if there are underexposed areas in the initial image, the light source output intensity of the structured light camera can be corrected based on the initial image to compensate for the brightness of the underexposed areas in the initial image, thereby compensating for the brightness of the underexposed areas in the field of view and making the brightness of the field of view more uniform.

[0082] For example, if there are overexposed areas in the initial image, the light source output intensity of the structured light camera can be corrected based on the initial image to suppress the brightness of the overexposed areas in the initial image, thereby suppressing the brightness of the overexposed areas in the field of view and making the brightness of the field of view more uniform.

[0083] For example, when both underexposed and overexposed areas exist in the initial image, the light source output intensity of the structured light camera can be corrected based on the initial image. This can compensate for the brightness of the underexposed areas in the initial image and suppress the brightness of the overexposed areas in the initial image, thereby compensating for the brightness of the underexposed areas in the field of view and suppressing the brightness of the overexposed areas in the field of view, making the brightness of the field of view more uniform.

[0084] If the brightness difference of the initial image is less than the brightness difference threshold, it indicates that the brightness difference within the field of view of the initial image is small, there is no problem of uneven brightness in the field of view, and there is no need to correct the output intensity of the light source of the structured light camera.

[0085] S203. Acquire real-time images captured by the structured light camera after calibration.

[0086] After correcting the output intensity of the light source of the structured light camera, the projection unit of the structured light camera is controlled to project a structured light pattern based on the corrected output intensity of the light source, and the photosensitive unit of the structured light camera is controlled to acquire the projected pattern to obtain a real-time image.

[0087] Furthermore, the brightness difference of the real-time image is determined, and it is judged whether the brightness difference of the real-time image is greater than or equal to the brightness difference threshold, that is, whether the real-time image meets the convergence condition of image brightness uniformity.

[0088] The method for determining the brightness difference of the real-time image is the same as the principle for determining the brightness difference of the initial image in step S202 above, and will not be repeated here.

[0089] S204. When the brightness difference of the real-time image is greater than or equal to the brightness difference threshold, the output intensity of the light source of the structured light camera is corrected based on the real-time image.

[0090] If the brightness difference in the real-time image is greater than or equal to the brightness difference threshold, it indicates that the brightness difference within the field of view of the currently acquired real-time image is significant, and the problem of uneven brightness in the field of view still exists. Based on the real-time image, the output intensity of the structured light camera's light source can be corrected again. Using the currently acquired real-time image as a reference image for correction, the output intensity of the structured light camera's light source is corrected based on the currently acquired real-time image to compensate for the brightness of underexposed areas and suppress the brightness of overexposed areas, thereby compensating for the brightness of underexposed areas and suppressing the brightness of overexposed areas within the field of view, resulting in a more uniform brightness within the field of view.

[0091] S205. Until the brightness difference of the real-time image acquired by the structured light camera is less than the brightness difference threshold, the corrected light source output intensity is determined as the final light source output intensity.

[0092] By iteratively executing steps S203-S204, the output intensity of the structured light camera's light source is corrected based on the real-time acquired images to compensate for the brightness of underexposed areas within the field of view and suppress the brightness of overexposed areas, resulting in a more uniform brightness within the field of view. The process continues until the brightness difference of the acquired real images is less than the brightness difference threshold, at which point the acquired real images satisfy the image brightness uniformity convergence condition, indicating uniform brightness within the field of view. The corrected light source output intensity is then determined as the final light source output intensity.

[0093] The method in this application, when the brightness difference of the image acquired by the structured light camera is greater than or equal to a brightness difference threshold, corrects the light source output intensity of the structured light camera based on the currently acquired image to compensate for the brightness of underexposed areas and / or suppress the brightness of overexposed areas, making the brightness values ​​across the entire field of view more uniform. After one or more iterations of the aforementioned correction of the light source output intensity of the structured light camera, until the brightness difference of the real-time image acquired by the structured light camera is less than the brightness difference threshold, that is, when the acquired real-time image meets the image brightness uniformity convergence condition, the brightness values ​​across the entire field of view tend to be uniform, solving the problem of uneven brightness distribution in the field of view. This improves imaging integrity while meeting the timeliness requirements of industrial real-time imaging.

[0094] In one optional embodiment, the output intensity of the light source of the structured light camera is corrected based on the image acquired by the structured light camera. This can be achieved in the following way:

[0095] Based on the brightness value and average brightness of each pixel in the acquired image, a brightness compensation value for each pixel is determined. Then, based on the brightness compensation values ​​of each pixel, the output intensity of the light source from the structured light camera is adjusted to perform brightness correction on the field of view of the structured light camera. The images acquired by the structured light camera include initial images and real-time images.

[0096] For example, determining the brightness compensation value for each pixel based on its brightness value and average brightness value in the acquired image can be achieved as follows: For any pixel in the acquired image, the difference between the average brightness value and the pixel's brightness value is used as the pixel's brightness compensation value. This process can be represented as: .in, This represents the brightness value of the i-th pixel in the image. This represents the average brightness of the image. This represents the brightness compensation value of the i-th pixel in the image.

[0097] If the pixel's brightness compensation value A value > 0 indicates that the brightness at the pixel's location (usually at the image edge) is too low, requiring an increase in the corresponding light source output intensity (e.g., light source power); if the pixel's brightness compensation value is... A value less than 0 indicates that the brightness of the pixel's location (usually in the center of the image) is too high, and the corresponding light source output intensity needs to be reduced.

[0098] For example, the average brightness of the acquired image can be determined as follows:

[0099] The brightness values ​​of each pixel in the acquired image are extracted to form a brightness sequence; based on the brightness sequence, the average brightness of the acquired image is determined. The brightness sequence can be a two-dimensional sequence / matrix composed of the brightness values ​​of each pixel in the image arranged in order of their position within the image.

[0100] Optionally, before determining the average brightness of the acquired image based on the brightness sequence, the brightness sequence can be filtered to reduce noise or outliers (such as pure black dots with zero brightness caused by bad pixels in the photosensitive unit), thereby ensuring the accuracy of the subsequent calculation of the average brightness value. Further, the average brightness of the acquired image is determined based on the filtered brightness sequence.

[0101] For example, the brightness sequence can be filtered using methods such as median filtering, mean filtering, or Gaussian filtering, or other algorithms with similar capabilities. No specific limitations are made here.

[0102] For example, for a structured light camera, the average of all brightness values ​​in the brightness sequence can be determined as the average brightness of the image.

[0103] For example, for a oscillating line scan structured light camera, since the brightness values ​​in the same column of the brightness sequence of the acquired image are consistent, the average brightness value of any horizontal sequence in the brightness sequence can be determined as the average brightness value of the image. Here, any horizontal sequence in the brightness sequence, i.e., a row in the brightness sequence, contains the brightness values ​​of the pixels in that horizontal row of the image.

[0104] Optionally, for a oscillating line-scan structured light camera, the brightness values ​​of a horizontal pixel sequence in the acquired image can be extracted to form a horizontal brightness sequence (one-dimensional sequence); the mean of the horizontal brightness sequence is then determined as the average brightness value of the acquired image. This process can be expressed as: .in, This represents the brightness value of the j-th pixel in the horizontal pixel sequence, where m represents the number of pixels in the horizontal pixel sequence. This represents the average brightness of the image.

[0105] In this embodiment, the output intensity of the light source of the structured light camera is adjusted according to the brightness compensation value of each pixel. Specifically, this can be achieved as follows: for each light source in the structured light camera, the output intensity (i.e., output power) of the light source is adjusted according to the compensation value of the pixel to which the light source is projected.

[0106] For example, for any light source in a structured light camera, if the light source is projected onto a pixel, the compensation value of the pixel onto which the light source is projected is mapped into the light source output intensity increment according to a pre-designed mapping rule from the compensation value to the light source output intensity increment; and the output intensity of the light source is adjusted according to the light source output intensity increment.

[0107] For any light source in a structured light camera, if the light source projects onto multiple pixels, the average of the compensation values ​​of the multiple pixels onto which the light source projects is mapped into the light source output intensity increment according to a pre-designed mapping rule from compensation values ​​to light source output intensity increments; the output intensity of the light source is then adjusted according to the light source output intensity increment. The pre-designed mapping rule from compensation values ​​to light source output intensity increments can be designed and adjusted according to actual application requirements, and is not specifically limited here.

[0108] If the compensation value (or the average of the compensation values) of the pixel projected by the light source is greater than 0, the incremental increase of the light source output intensity is mapped to be greater than 0, so as to provide supplementary lighting to the local area within the projection range of the light source.

[0109] If the compensation value (or the average of the compensation values) of the pixel projected by the light source is less than 0, the incremental increase of the light source output intensity obtained by mapping is less than 0, so as to reduce the light in the local area within the projection range of the light source.

[0110] Optionally, a pre-designed mapping rule from compensation value to light source output intensity increment can be integrated into the hardware control module. The hardware control module maps the compensation value of the pixel projected by the light source to the light source output intensity increment according to the pre-designed mapping rule, thereby improving the efficiency of the aforementioned mapping process and thus improving the efficiency of structured light camera parameter correction.

[0111] Optionally, a machine learning model (called a projection brightness compensation model) for projection brightness compensation can be pre-established and trained. This model maps the compensation value of the pixel projected by the light source to the light source's output intensity increment, thereby accurately determining the light source output intensity increment of the structured light camera and making the total gray value of the light reflection signal in the entire field of view more consistent.

[0112] In one optional embodiment, after adjusting the light source output intensity of the structured light camera according to the brightness compensation value of each pixel, the adjusted light source output intensity of the structured light camera can be smoothed to ensure that the brightness transitions naturally within the field of view and avoid local brightness jumps or abrupt changes.

[0113] Optionally, a sliding window-based filtering method can be used to smooth the adjusted light source output intensity of the structured light camera. This sliding window-based filtering method can be a median filtering method, a mean filtering method, a Gaussian filtering method, etc., and is not specifically limited here.

[0114] For example, the adjusted light source output intensities of the structured light camera are arranged into a two-dimensional sequence / matrix, referred to as the light source output intensity sequence. The light source output intensity sequence is scanned using a preset window (such as a 3×3 window), and the light source output intensities falling within the window are filtered (such as median filtering, mean filtering, or Gaussian filtering).

[0115] In an optional embodiment, for a swing line scan structured light camera, after acquiring the real-time image captured by the structured light camera after calibration, a lateral brightness distribution curve of the field of view of the real-time image can be generated to intuitively display the lateral brightness within the field of view of the real-time image, making it easier for users to observe whether the lateral brightness within the field of view of the real-time image is uniform.

[0116] Optionally, for a oscillating line scan structured light camera, after acquiring the real-time image captured by the structured light camera after correction, a comparison chart of the lateral brightness distribution curves of the images before and after correction can be generated. For example, the lateral brightness distribution curves of the images before and after correction can be displayed in the same coordinate system to facilitate comparison of the uniformity of lateral brightness within the field of view before and after correction.

[0117] For example, Figure 3 FIG. is an example diagram of a comparison graph of the horizontal luminance distribution curves of the images acquired before and after calibration provided for this embodiment. As Figure 3 shown, Image 1 (original image) refers to the image acquired by the structured light camera before calibration, and Image 2 (calibrated image) refers to the image acquired by the structured light camera after calibration. As Figure 3 shown, after calibration, Image 2 suppresses overexposure in the central region of the field of view relative to the original Image 1, significantly improves the luminance in the edge region, makes the luminance distribution within the field of view tend to be stable, and the luminance within the field of view is more uniform.

[0118] For a surface structured light camera, acquire a real-time image acquired by the surface structured light camera after calibration, and generate a horizontal luminance distribution curve and / or a vertical luminance distribution curve of the real-time image to visually display the horizontal luminance and / or vertical luminance within the field of view of the real-time image, facilitating the user to observe whether the luminance in the horizontal and / or vertical directions within the field of view of the real-time image is uniform.

[0119] In addition, the horizontal luminance distribution curve of the image acquired before calibration and the horizontal luminance distribution curve of the image acquired after calibration can be displayed in the same coordinate system to facilitate the user to compare the uniformity of the horizontal luminance within the field of view before and after calibration. The vertical luminance distribution curve of the image acquired before calibration and the vertical luminance distribution curve of the image acquired after calibration are displayed in the same coordinate system to facilitate the user to compare the uniformity of the vertical luminance within the field of view before and after calibration.

[0120] Figure 4 FIG. is a schematic diagram of the overall process of the structured light camera parameter calibration method provided for an exemplary embodiment of the present application. On the basis of the foregoing embodiment, this embodiment details the overall process of the structured light camera parameter calibration. As Figure 4 shown, the overall process of the structured light camera parameter calibration is as follows:

[0121] Initialization and deployment: Place a gray scale plate (such as an 18% gray scale plate) at the standard working distance of the structured light camera, turn on the ambient light source in the scene, and set the convergence condition for the image luminance uniformity. For example, the convergence condition for the image luminance uniformity is set as: the horizontal luminance difference degree D in the full field of view < m%, where m% is the luminance difference degree threshold.

[0122] Acquisition parameter setting: Adjust the imaging parameters of the structured light camera (such as camera exposure and gain) so that the structured light camera acquires a fully bright image without overexposure.

[0123] Data preprocessing: Perform filtering processing (such as median filtering) on the acquired image to reduce abnormal points in the image.

[0124] Determination of image brightness difference: Determine the brightness values and the average brightness of each pixel in the currently acquired image, and determine the brightness difference of the currently acquired image, such as D = ((MAX - MIN) / MAX) × 100%.

[0125] Based on the brightness difference D of the currently acquired image, determine whether the currently acquired image meets the image brightness uniformity convergence condition, that is, determine whether D < m% holds.

[0126] If the currently acquired image meets the image brightness uniformity convergence condition, that is, D < m% holds, it means that the brightness of the currently acquired image is uniform, and determine the light source output intensity of the structured light camera as the final light source output intensity. The structured light camera performs subsequent shooting tasks based on the final light source output intensity.

[0127] If the currently acquired image does not meet the image brightness uniformity convergence condition, that is, D < m% does not hold, it means that the brightness of the currently acquired image is not uniform. Determine the brightness compensation value for the full frame based on the brightness values and the average brightness of each pixel in the currently acquired image, obtain the brightness compensation value for each pixel in the image, and adjust the light source output intensity of the structured light camera according to the brightness compensation value of each pixel to perform brightness correction on the field of view area of the structured light camera.

[0128] Furthermore, perform smoothing processing on the corrected light source output intensity to ensure natural transition of brightness within the field of view and avoid local brightness steps or mutations.

[0129] The structured light camera projects a pattern based on the corrected light source output intensity and acquires a real-time image by projecting the pattern.

[0130] Furthermore, for the currently acquired real-time image, perform the aforementioned data preprocessing, determination of image brightness difference, determination of whether the currently acquired real-time image meets the image brightness uniformity convergence condition, and subsequent correction processing; until the acquired image meets the image brightness uniformity convergence condition, determine the corrected light source output intensity this time as the final light source output intensity. The structured light camera performs subsequent shooting tasks based on the final light source output intensity, such as acquiring the point cloud data of the target object.

[0131] Effect verification: When using the structured light camera to acquire the point cloud of a reflective object, for each projected pattern, perform correction according to the correction method of this embodiment and then project it, and acquire the point cloud. Statistically analyze the integrity of the point clouds acquired before and after correction. Experiments prove that under normal indoor light and oblique light conditions, the problem of point cloud missing of the workpiece has been significantly improved, and the integrity of the point cloud has been greatly improved.

[0132] For example, Figure 5The image shows the point clouds of reflective objects placed on the left, center, and right sides of the field of view in a single exposure without correction using the correction method of this embodiment, respectively corresponding to... Figure 5 Images on the left, middle, and right sides. (Example) Figure 5 As shown in the left image, the left side of a reflective object placed on the left side of the field of view is insufficiently bright, resulting in severe deficiencies in the left-side point cloud. Figure 5 As shown in the image on the right, the right side of a reflective object placed on the right side of the field of view is not bright enough, resulting in severe loss of point cloud on the right side.

[0133] For example, Figure 6 The image shows the point clouds of reflective objects placed on the left, center, and right sides of the field of view under a single exposure after correction using the correction method of this embodiment, respectively corresponding to... Figure 6 Images on the left, middle, and right sides. (Example) Figure 6 As shown in the left-hand image, reflective objects placed on the left side of the field of view exhibit more uniform brightness, and the integrity of the point cloud is significantly improved. Figure 6 As shown in the image on the right, the brightness of reflective objects placed on the right side of the field of view is more uniform, and the integrity of the point cloud is significantly improved.

[0134] The solution in this application embodiment achieves dynamic correction of the field-of-view brightness uniformity of the structured light imaging system, addressing the imaging characteristics of structured light point sources. It aims to solve the problems of point cloud loss and measurement errors caused by factors such as optical system attenuation and uneven projection energy distribution. This solution is suitable for applications requiring high imaging integrity and measurement accuracy, such as industrial inspection and 3D reconstruction. The method provided in this application embodiment has the following beneficial effects:

[0135] 1. Strong dynamic adaptability: It can adapt to various complex industrial lighting environments such as normal indoor light and oblique light in real time, and also takes into account the influence of ambient light, which can significantly expand the application boundaries of the solution and has strong effectiveness and practicality.

[0136] 2. Significant Correction Effect: It can accurately compensate for areas that traditional solutions struggle to cover, including edges and corners. Tests show that the median integrity of point cloud data acquired by the structured light camera after correction based on this embodiment reaches >90%, meeting the requirements for high-precision industrial-grade measurement.

[0137] 3. High imaging efficiency: Unlike traditional methods that require multiple exposures to compensate for signal loss areas, this method accurately determines the brightness compensation value of each pixel based on real-time acquired images, precisely corrects the light source output intensity of the structured light camera, and directly achieves brightness consistency in a single imaging, greatly shortening the scanning cycle and meeting the timeliness requirements of industrial real-time imaging.

[0138] 4. High engineering compatibility: The core logic of the method provided in this application embodiment has low complexity and can be directly integrated into the image processing module of the structured camera without significantly modifying the hardware architecture of the existing structured light system. It has low development cost and low implementation difficulty.

[0139] Figure 7 This is a schematic diagram illustrating the structure for parameter correction of the structured light camera provided in this application. Figure 7 As shown, the structured light camera parameter correction device 50 provided in this embodiment includes at least one processor 501 and a memory 502. Optionally, the device 50 further includes a communication component 503. The processor 501, memory 502, and communication component 503 are connected via a bus 504.

[0140] In a specific implementation, at least one processor 501 executes computer execution instructions stored in memory 502, causing at least one processor 501 to perform the above-described method.

[0141] The specific implementation process of processor 501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0142] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0143] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0144] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0145] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0146] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0147] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0148] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0149] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0150] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0151] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0152] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0153] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0154] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for correcting parameters of a structured light camera, characterized in that, include: Acquire the initial image captured by the structured light camera; If the brightness difference of the initial image is greater than or equal to the brightness difference threshold, the light source output intensity of the structured light camera is corrected based on the initial image to compensate for the brightness of underexposed areas and / or suppress the brightness of overexposed areas. Acquire real-time images from the structured light camera after the correction; If the brightness difference of the real-time image is greater than or equal to the brightness difference threshold, the light source output intensity of the structured light camera is corrected based on the real-time image. The corrected light source output intensity is determined as the final light source output intensity until the brightness difference of the real-time image acquired by the structured light camera is less than the brightness difference threshold.

2. The method according to claim 1, characterized in that, Based on the images acquired by the structured light camera, the output intensity of the light source of the structured light camera is corrected, including: Based on the brightness value and average brightness of each pixel in the acquired image, determine the brightness compensation value of each pixel; The light source output intensity of the structured light camera is adjusted according to the brightness compensation value of each pixel in order to perform brightness correction on the field of view of the structured light camera. The images acquired by the structured light camera include the initial image and the real-time image.

3. The method according to claim 2, characterized in that, After adjusting the light source output intensity of the structured light camera according to the brightness compensation value of each pixel, the method further includes: The output intensity of the light source after adjustment by the structured light camera is smoothed.

4. The method according to claim 2, characterized in that, The method further includes: The brightness values ​​of each pixel in the acquired image are extracted to form a brightness sequence; The average brightness of the acquired image is determined based on the brightness sequence.

5. The method according to claim 4, characterized in that, Determining the average brightness of the acquired image based on the brightness sequence includes: The brightness sequence is filtered to reduce noise or outliers in the brightness sequence; The average brightness of the acquired image is determined based on the filtered brightness sequence.

6. The method according to claim 1, characterized in that, After acquiring the real-time image captured by the structured light camera after correction, and before determining that the brightness difference of the real-time image is greater than or equal to a brightness difference threshold, the method further includes: Determine the horizontal brightness difference of the entire field of view of the real-time image, and use the horizontal brightness difference of the entire field of view of the real-time image as the brightness difference of the real-time image. Determine whether the brightness difference of the real-time image is greater than or equal to the brightness difference threshold.

7. The method according to claim 1, characterized in that, The structured light camera includes a projection unit and a photosensitive unit. Acquiring the initial image captured by the structured light camera includes: The imaging parameters of the structured light camera are set according to the preset imaging parameters; The projection unit is controlled to project a pattern onto a standard grayscale plate placed at a standard working distance, and the photosensitive unit is controlled to collect the projected pattern to obtain the initial image captured by the structured light camera.

8. The method according to any one of claims 1-7, characterized in that, After acquiring the real-time image captured by the structured light camera after correction, the process further includes: Generate the lateral brightness distribution curve of the field of view of the real-time image; And / or, A comparison diagram of the lateral brightness distribution curves of the images acquired before and after the correction is generated.

9. A structured light camera parameter correction device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-8.

11. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-8.

Images