An image processing method, apparatus, electronic device, and storage medium

Abstract

This invention discloses an image processing method, apparatus, electronic device, and storage medium. The method includes: responding to a light projection device projecting a first projection pattern and a second projection pattern; acquiring a first imaging image corresponding to the first projection pattern and a second imaging image corresponding to the second projection pattern using an image acquisition device; calculating the difference between the grayscale information of the first imaging image and the grayscale information of the second imaging image pixel by pixel to obtain a first image; performing an inverse perspective transformation on the first image based on a homography matrix to obtain a second image; determining the average grayscale information corresponding to the second image based on the grayscale information of the second image; determining grayscale compensation coefficients corresponding to each pixel position in the second image based on the average grayscale information and the grayscale information of the second image; and determining light intensity attenuation correction information based on the grayscale compensation coefficients corresponding to each pixel position in the second image, thereby achieving pixel-level precise determination of the grayscale compensation coefficients.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more particularly to an image processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] Photopolymerization printing equipment based on digital light projection uses a digital projector to project ultraviolet light into a resin tank for photopolymerization. Due to factors such as the characteristics of the projection lens, the light source, and the inherent fourth-power cosine law of the optical system, the light spot projected by digital projection equipment usually exhibits uneven light intensity distribution, with a bright center and dark edges.

[0003] To address these issues, the American National Standards Institute (ANSI) uniformity correction method, built into digital projection equipment, is typically employed. The ANSI uniformity correction method reduces the light intensity in the central region by using pre-set light intensity attenuation correction information within the 3D printing equipment, thus aligning the light intensity in the central region with that of the edge regions. However, because the light intensity attenuation correction information cannot dynamically adjust to changes in the digital projection equipment's light source attenuation, lens changes, and printing environment, the ANSI uniformity correction method suffers from poor adaptive correction capabilities. This leads to a continuous decline in light intensity correction accuracy and poor overall correction precision over long-term use. Summary of the Invention

[0004] This invention provides an image processing method, apparatus, electronic device, and storage medium to accurately determine the light intensity attenuation correction information corresponding to an optical projection device.

[0005] According to one aspect of the present invention, an image processing method is provided, comprising: In response to the projection of a first projection pattern and a second projection pattern by the optical projection device, a first imaging image corresponding to the first projection pattern and a second imaging image corresponding to the second projection pattern are acquired by the image acquisition device. The grayscale information of the first projection pattern and the grayscale information of the second projection pattern are different. The image content of the first projection pattern and the image content of the second projection pattern are the same. Both the first imaging image and the second imaging image are located in the camera image coordinate system. The difference between the grayscale information of the first imaging image and the grayscale information of the second imaging image is calculated pixel by pixel to obtain the first image. The grayscale information of each pixel position in the first image represents the deviation between the grayscale information of the corresponding pixel position in the first imaging image and the grayscale information of the corresponding pixel position in the second imaging image. The first image is subjected to inverse perspective transformation based on the homography matrix to obtain the second image. The second image is located in the coordinate system of the optical projection device. The homography matrix is ​​pre-calibrated and is used to characterize the transformation relationship between the coordinate system of the optical projection device and the camera image coordinate system. The average grayscale information of the second image is determined based on the grayscale information of the second image. The gray level compensation coefficients corresponding to each pixel position in the second image are determined based on the average gray level information and the gray level information of the second image. The light intensity attenuation correction information for the light projection device is determined based on the grayscale compensation coefficients corresponding to each pixel position in the second image.

[0006] According to another aspect of the present invention, an image processing apparatus is provided, comprising: An imaging image acquisition module is used to respond to the projection of a first projection pattern and a second projection pattern by a light projection device, and to acquire a first imaging image corresponding to the first projection pattern and a second imaging image corresponding to the second projection pattern based on the image acquisition device. The grayscale information of the first projection pattern and the grayscale information of the second projection pattern are different, the image content of the first projection pattern and the image content of the second projection pattern are the same, and both the first imaging image and the second imaging image are located in the camera image coordinate system. The first image determination module is used to calculate the difference between the grayscale information of the first imaging image and the grayscale information of the second imaging image pixel by pixel to obtain the first image. The grayscale information of each pixel position in the first image represents the deviation between the grayscale information of the corresponding pixel position in the first imaging image and the grayscale information of the corresponding pixel position in the second imaging image. The second image determination module is used to perform inverse perspective transformation on the first image based on the homography matrix to obtain the second image. The second image is located in the coordinate system of the optical projection device. The homography matrix is ​​pre-calibrated and is used to characterize the transformation relationship between the coordinate system of the optical projection device and the camera image coordinate system. The average grayscale information determination module is used to determine the average grayscale information corresponding to the second image based on the grayscale information of the second image; The grayscale compensation coefficient determination module is used to determine the grayscale compensation coefficient corresponding to each pixel position in the second image based on the average grayscale information and the grayscale information of the second image. The light intensity attenuation correction information determination module is used to determine the light intensity attenuation correction information corresponding to the light projection device based on the grayscale compensation coefficients corresponding to each pixel position in the second image.

[0007] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and A memory that is communicatively connected to at least one processor; wherein, The memory stores a computer program that can be executed by at least one processor, such that the at least one processor can perform the image processing method of any embodiment of the present invention.

[0008] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions for causing a processor to execute and implement the image processing method of any embodiment of the present invention.

[0009] According to another aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements an image processing method as described in any embodiment of the present invention.

[0010] The technical solution of this invention, in response to a light projection device projecting a first projection pattern and a second projection pattern, acquires a first imaging image corresponding to the first projection pattern and a second imaging image corresponding to the second projection pattern using an image acquisition device. The grayscale information of the first projection pattern and the second projection pattern are different, but the image content of the first and second projection patterns is the same. Both the first and second imaging images are located in the camera image coordinate system, providing comprehensive data support for subsequent analysis and processing. The difference between the grayscale information of the first and second imaging images is calculated pixel-by-pixel to obtain the first image. The grayscale information of each pixel in the first image represents the deviation between the grayscale information of the corresponding pixel in the first imaging image and the grayscale information of the corresponding pixel in the second imaging image, achieving accurate calculation of the deviation of grayscale information at different pixel positions, providing accurate data support for subsequent analysis and processing. Based on homography moments... The first image is subjected to inverse perspective transformation to obtain the second image, which is located in the coordinate system of the optical projection device. The homography matrix is ​​pre-calibrated and used to characterize the transformation relationship between the coordinate system of the optical projection device and the camera image coordinate system, realizing accurate transformation from the camera image coordinate system to the optical projection device coordinate system. Based on the grayscale information of the second image, the average grayscale information corresponding to the second image is determined, providing a global reference for subsequent analysis and processing. Based on the average grayscale information and the grayscale information of the second image, the grayscale compensation coefficients corresponding to each pixel position in the second image are determined, which can accurately quantify the light intensity deviation of each pixel position in the optical projection coordinate system and realize pixel-level light intensity uniformity correction. Based on the grayscale compensation coefficients corresponding to each pixel position in the second image, the light intensity attenuation correction information corresponding to the optical projection device is determined, which solves the problem of low accuracy in the existing light intensity uniformity correction methods, realizes light intensity correction at different pixel positions, and can improve the accuracy of light intensity correction.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a schematic diagram of the structure of an image acquisition device in a photopolymerization printing scenario provided by an embodiment of the present invention; Figure 2 This is a flowchart of an image processing method provided in an embodiment of the present invention; Figure 3 This is a flowchart of another image processing method provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an image processing device provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0014] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0015] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0016] To implement the image processing method provided in any embodiment of the present invention, the present invention also provides an image acquisition device for a photopolymerization printing scenario. This image acquisition device includes a photopolymerization printing device and an image acquisition device. The photopolymerization printing device includes a light projection device and a printing stage. A cosine correction device can be deployed on the surface of the printing stage, which can break down and scatter the directional light projected by the light projection device, converting it into a Lambertian light source. Optionally, the surface of the printing stage can be covered with two layers of optical diffusion film. The image acquisition device includes, but is not limited to, an industrial camera; for example, the image acquisition device can be a 12-bit monochrome global shutter camera. The image acquisition device is equipped with a lens, which includes, but is not limited to, a factory automation (FA) lens; for example, the lens can be a 25mm FA lens. The image acquisition device is mounted above the working area of ​​the printing stage using a fixture, and images are directed downwards towards the working area. For example, see... Figure 1 , Figure 1 This is a schematic diagram of an image acquisition device for a photopolymerization printing scenario provided in an embodiment of the present invention. The rectangular area of ​​the printing stage is the working area, and two layers of optical diffusion film are laid on the rectangular area.

[0017] Figure 2 This is a flowchart of an image processing method provided in an embodiment of the present invention. This embodiment is applicable to situations where light intensity attenuation correction information corresponding to a light projection device needs to be determined. This method can be executed by an image processing device, which can be implemented in hardware and / or software. The image processing device can be configured in the electronic device provided in this embodiment of the present invention. The electronic device can be a server, computer, or mobile terminal, such as a mobile phone or tablet computer. Figure 2 As shown, the method specifically includes the following steps: S110. In response to the projection of a first projection pattern and a second projection pattern by the optical projection device, a first imaging image corresponding to the first projection pattern and a second imaging image corresponding to the second projection pattern are acquired by the image acquisition device. The grayscale information of the first projection pattern and the grayscale information of the second projection pattern are different. The image content of the first projection pattern and the image content of the second projection pattern are the same. Both the first imaging image and the second imaging image are located in the camera image coordinate system.

[0018] The optical projection device includes, but is not limited to, a digital projection device. Both the first and second projection patterns are pre-set patterns. It should be noted that the grayscale information of the first and second projection patterns are different, but the image content of the first and second projection patterns is the same. For example, the grayscale information of the first projection pattern can be set to 255, and the grayscale information of the second projection pattern can be set to 0. Optionally, the first projection pattern is a pattern with all grayscale values ​​of 255, and the second projection pattern is a pattern with all grayscale values ​​of 0 and the same image content as the first projection pattern.

[0019] Image acquisition devices include, but are not limited to, industrial cameras. An optical projection device projects a first projection pattern and a second projection pattern onto a printing table equipped with a cosine correction device, respectively. The image acquisition device triggers image acquisition logic to acquire a first imaging image corresponding to the first projection pattern and a second imaging image corresponding to the second projection pattern, respectively. The first imaging image is the image acquired by the image acquisition device after the optical projection device projects the first projection pattern onto the printing table, which is covered with two layers of optical diffusion film. The second imaging image is the image acquired by the image acquisition device after the optical projection device projects the second projection pattern onto the printing table equipped with the cosine correction device. It should be noted that both the first and second imaging images are located in the camera image coordinate system. The camera image coordinate system is a two-dimensional planar coordinate system established by the sensor pixel array of the image acquisition device.

[0020] Specifically, the optical projection device projects the first projection pattern and the second projection pattern onto the printing workbench where the cosine correction device is deployed, respectively. The image acquisition device triggers the image acquisition logic to acquire the first imaging image corresponding to the first projection pattern and the second imaging image corresponding to the second projection pattern, respectively, providing comprehensive data support for subsequent analysis and processing.

[0021] S120. Calculate the difference between the grayscale information of the first imaging image and the grayscale information of the second imaging image pixel by pixel to obtain the first image. The grayscale information of each pixel position in the first image represents the deviation between the grayscale information of the corresponding pixel position in the first imaging image and the grayscale information of the corresponding pixel position in the second imaging image.

[0022] The first image is a difference image obtained by performing pixel-by-pixel gray-level difference operations between the first and second imaging images. It should be noted that the gray-level information of each pixel in the first image represents the deviation between the gray-level information of the corresponding pixel in the first and second imaging images. For example, the gray-level information of the corresponding pixel in the first image is obtained by calculating the difference between the gray-level information of each pixel in the first and second imaging images. It should be noted that the first image is located in the camera image coordinate system.

[0023] Specifically, the difference between the grayscale information of each pixel position in the first image and the grayscale information of the corresponding pixel position in the second image is calculated to obtain the grayscale information of the corresponding pixel position in the first image. This achieves accurate calculation of the deviation of grayscale information at different pixel positions, providing accurate data support for subsequent analysis and processing.

[0024] S130. Perform inverse perspective transformation on the first image based on the homography matrix to obtain the second image. The second image is located in the coordinate system of the optical projection device. The homography matrix is ​​pre-calibrated and is used to characterize the transformation relationship between the coordinate system of the optical projection device and the camera image coordinate system.

[0025] The homography matrix is ​​used to characterize the transformation relationship between the coordinate system of the optical projection device and the camera image coordinate system. The second image is obtained by performing an inverse perspective transformation on the first image. It should be noted that the second image is located in the coordinate system of the optical projection device, which is a two-dimensional planar coordinate system established by the digital micromirror pixel array inside the optical projection device.

[0026] Specifically, the homography matrix and the first image are input into the trained inverse perspective transformation model for processing to obtain the second image. The inverse perspective transformation model includes, but is not limited to, neural network models and mathematical models. The inverse perspective transformation model is selected according to the requirements. This invention does not impose any restrictions and achieves accurate transformation from the camera image coordinate system to the coordinate system of the light projection device.

[0027] Optionally, the homography matrix calibration process is as follows: In response to the projection of a preset projection dot matrix by the optical projection device, a dot matrix image corresponding to the projection dot matrix is ​​acquired by the image acquisition device. The projection dot matrix includes multiple marker points and the first position information of each marker point in the coordinate system of the optical projection device; feature extraction is performed on the dot matrix image to obtain the second position information of each projection point in the dot matrix image in the coordinate system of the camera image; the homography matrix is ​​determined based on the first position information and the second position information.

[0028] The projection dot matrix can be pre-set, and its size can be set according to requirements; for example, the size of the projection dot matrix can be 12×21. The projection dot matrix includes multiple marker points, and the shapes of the marker points can be the same or different. The shapes of the marker points include, but are not limited to, rectangles and circles. This invention is illustrated using the example that all marker points are solid circles. For example, the projection dot matrix is ​​a dot matrix with a radius of 40 pixels and a center-to-center distance of 90 pixels between adjacent circles. The first position information is the two-dimensional coordinates of the marker point in the coordinate system of the optical projection device. Optionally, the first position information includes the two-dimensional coordinates of the center of the circle in the coordinate system of the optical projection device.

[0029] The dot matrix image is an image acquired by an image acquisition device after projecting a dot matrix onto a printing worktable equipped with a cosine correction device using a light projection device. The second positional information is the two-dimensional coordinates of the projected points in the camera image coordinate system. Optionally, the second positional information includes the two-dimensional coordinates of the center of a circle in the camera image coordinate system. The second positional information can be determined based on the dot matrix image. For example, the dot matrix image can be input into a trained feature extraction model for feature extraction to obtain the second positional information of each projected point in the dot matrix image in the camera image coordinate system. The feature extraction model includes, but is not limited to, a neural network model; the feature extraction model is selected according to requirements, and this invention is not restrictive.

[0030] The homography matrix can be determined based on first position information and second position information. For example, the first position information and second position information are input into a trained homography matrix determination model for processing to obtain the homography matrix. The homography matrix determination model includes, but is not limited to, a neural network model. The homography matrix determination model is selected according to the requirements, and this invention does not impose any restrictions.

[0031] Specifically, in response to the projection of a preset projection dot matrix by the optical projection device, the image acquisition device acquires the dot matrix image corresponding to the projection dot matrix; the dot matrix image is input into the trained feature extraction model for feature extraction to obtain the second position information of each projection point in the dot matrix image in the camera image coordinate system; the first position information and the second position information are input into the trained homography matrix determination model for processing to obtain the homography matrix, thus achieving accurate calibration of the homography matrix and providing accurate data support for the inverse perspective transformation of the first image.

[0032] Optionally, the second position information includes sub-pixel position information; optionally, feature extraction is performed on the dot matrix image to obtain the second position information of each projection point in the dot matrix image in the camera image coordinate system, including: contour extraction of the projected dot matrix image to obtain contour information corresponding to multiple projection points respectively; fitting the contour information to obtain the initial position information corresponding to the projection points; taking the initial position information as the center, a local window of a preset size is extracted; the gray-level gradient features corresponding to the local window are determined based on the gray-level information of each pixel position in the local window; the initial position information is iteratively corrected based on the preset sub-pixel interpolation model and the gray-level gradient features corresponding to the local window to obtain the sub-pixel position information corresponding to each projection point respectively.

[0033] The sub-pixel position information refers to the coordinates of the marker point in the camera image coordinate system, exceeding integer pixel precision and representing a fraction of a pixel. The contour information is a data set consisting of the pixel boundary contours of the projected points in the raster image. The contour information can be determined based on the projected raster image. For example, the projected raster image can be input into a trained contour extraction model for processing to obtain contour information corresponding to multiple projected points. The contour extraction model includes, but is not limited to, neural network models. The appropriate contour extraction model is selected based on requirements; this invention does not impose any limitations.

[0034] The initial position information consists of pixel coordinates obtained by initially fitting the contour information. It should be noted that the initial position information refers to pixel coordinates in the camera image coordinate system. The initial position information can be obtained by fitting the contour information using the least squares method.

[0035] The gray-level gradient feature is a two-dimensional gradient array consisting of the gray-level changes of each pixel position within a local window centered on the initial position information along the horizontal and vertical directions of the projected dot matrix image. The gray-level gradient feature can be determined based on the gray-level information of each pixel position within the local window. For example, based on the gray-level information of each pixel position within the local window, the gray-level difference along the horizontal and vertical directions of the projected dot matrix image for each pixel position is calculated to obtain the gray-level gradient feature corresponding to the local window.

[0036] The subpixel interpolation model is a numerical fitting model, including but not limited to quadratic surface interpolation and bilinear interpolation models. The subpixel interpolation model is pre-set; the selection of the subpixel interpolation model is based on requirements, and this invention does not impose any restrictions.

[0037] The sub-pixel position information corresponding to each projection point can be determined based on the gray-level gradient features and initial position information of the local window. For example, the gray-level gradient features and initial position information of the local window can be input into a preset sub-pixel interpolation model for processing to obtain the sub-pixel position information corresponding to each projection point.

[0038] Specifically, the projected dot matrix image is input into a trained contour extraction model for processing to obtain contour information corresponding to multiple projection points. The contour information is then fitted using the least squares method to obtain initial position information. A local window of a preset size is extracted centered on the initial position information. Based on the grayscale information of each pixel position in the local window, the grayscale difference along the horizontal and vertical directions of the projected dot matrix image is calculated to obtain the grayscale gradient features corresponding to the local window. The grayscale gradient features corresponding to the local window and the initial position information are input into a preset sub-pixel interpolation model for processing to obtain the sub-pixel position information corresponding to each projection point. This achieves accurate determination of sub-pixel level position information and can significantly improve the coordinate accuracy of the projection points.

[0039] Optionally, determining the homography matrix based on the first position information and the second position information includes: constructing multiple matching point pairs based on the first position information and the second position information, each matching point pair including the first position information of the marker point in the coordinate system of the optical projection device and the second position information of the corresponding projection point in the coordinate system of the camera image; constructing a set of two-dimensional plane projection transformation equations based on the multiple matching point pairs; and solving the set of two-dimensional plane projection transformation equations to obtain the homography matrix.

[0040] In this context, a matching point pair is a combination of positional information corresponding to the same marker point. Each matching point pair includes the first positional information of the marker point in the coordinate system of the optical projection device and the second positional information of the corresponding projected point in the camera image coordinate system. The two-dimensional plane projection transformation equations are a system of multivariate linear equations formed by substituting the positional information of multiple matching point pairs into the homography transformation mathematical relations. The homography matrix can be obtained by solving the two-dimensional plane projection transformation equations.

[0041] Specifically, the first position information of the marked point in the coordinate system of the optical projection device and the second position information of the corresponding projected point in the coordinate system of the camera image are used as matching point pairs; multiple matching point pairs are substituted into the homography transformation mathematical relationship to obtain a set of two-dimensional plane projection transformation equations; the two-dimensional plane projection transformation equations are solved to obtain the homography matrix, realizing the accurate calibration of the homography matrix and providing accurate data support for the inverse perspective transformation of the first image.

[0042] S140. Determine the average grayscale information corresponding to the second image based on the grayscale information of the second image.

[0043] The average grayscale information is the global average grayscale value of the grayscale information of each pixel position in the second image, which represents the average level of light intensity in the second image under the light projection coordinate system.

[0044] Specifically, the arithmetic mean of the grayscale information of each pixel position in the second image is calculated as the average grayscale information of the second image, providing a global reference for subsequent analysis and processing.

[0045] S150. Based on the average grayscale information and the grayscale information of the second image, determine the grayscale compensation coefficients corresponding to each pixel position in the second image.

[0046] The grayscale compensation coefficient is used to correct the output light intensity of the optical projection device. The grayscale compensation coefficient can be determined based on the average grayscale information and the grayscale information of the second image. For example, the ratio of the grayscale information of each pixel position in the second image to the average grayscale information is calculated and used as the grayscale compensation coefficient for each pixel position in the second image.

[0047] Specifically, by calculating the ratio of grayscale information at each pixel location to the average grayscale information in the second image, the ratio of grayscale information at each pixel location to the average grayscale information in the second image is used as the grayscale compensation coefficient corresponding to each pixel location in the second image. This can accurately quantify the light intensity deviation at each pixel location in the light projection coordinate system and achieve pixel-level brightness uniformity correction.

[0048] S160. Determine the light intensity attenuation correction information corresponding to the light projection device based on the grayscale compensation coefficients corresponding to each pixel position in the second image.

[0049] The light intensity attenuation correction information represents the light intensity correction ratio at each pixel position in the light projection coordinate system. The light intensity attenuation correction information can be determined based on the grayscale compensation coefficients corresponding to each pixel position in the second image. For example, the grayscale compensation coefficients corresponding to each pixel position in the second image are input into a trained light intensity attenuation correction information determination model for processing to obtain the light intensity attenuation correction information corresponding to the light projection device. The light intensity attenuation correction information determination model includes, but is not limited to, a neural network model. The appropriate light intensity attenuation correction information determination model can be selected according to requirements; this invention does not impose any limitations.

[0050] Specifically, the grayscale compensation coefficients corresponding to each pixel position in the second image are input into the trained light intensity attenuation correction information determination model for processing, so as to obtain the light intensity attenuation correction information corresponding to the light projection device, thereby realizing the light intensity correction at different pixel positions and improving the accuracy of light intensity correction.

[0051] The technical solution of this embodiment responds to the projection of a first projection pattern and a second projection pattern by a light projection device. Based on an image acquisition device, a first imaging image corresponding to the first projection pattern and a second imaging image corresponding to the second projection pattern are acquired. The grayscale information of the first projection pattern and the second projection pattern are different, but the image content of the first projection pattern and the second projection pattern are the same. Both the first and second imaging images are located in the camera image coordinate system, providing comprehensive data support for subsequent analysis and processing. The difference between the grayscale information of the first and second imaging images is calculated pixel-by-pixel to obtain the first image. The grayscale information at each pixel position in the first image represents the deviation between the grayscale information at the corresponding pixel position in the first imaging image and the grayscale information at the corresponding pixel position in the second imaging image. This achieves accurate calculation of the deviation of grayscale information at different pixel positions, providing accurate data support for subsequent analysis and processing. The system provides accurate data support; it performs inverse perspective transformation on the first image based on the homography matrix to obtain the second image, which is located in the coordinate system of the optical projection device. The homography matrix is ​​pre-calibrated and used to characterize the transformation relationship between the optical projection device coordinate system and the camera image coordinate system, achieving accurate transformation from the camera image coordinate system to the optical projection device coordinate system; it determines the average grayscale information of the second image based on the grayscale information, providing a global reference for subsequent analysis and processing; it determines the grayscale compensation coefficients corresponding to each pixel position in the second image based on the average grayscale information and the grayscale information of the second image, which can accurately quantify the light intensity deviation of each pixel position in the optical projection coordinate system and achieve pixel-level brightness uniformity correction; it determines the light intensity attenuation correction information corresponding to the optical projection device based on the grayscale compensation coefficients corresponding to each pixel position in the second image, realizing light intensity correction at different pixel positions and improving the accuracy of light intensity correction.

[0052] Figure 3 This is a flowchart of another image processing method provided by an embodiment of the present invention. This embodiment is a refinement of the above embodiments. Based on the foregoing embodiments, it provides a detailed explanation of determining the light intensity attenuation correction information corresponding to the light projection device based on the grayscale compensation coefficients corresponding to each pixel position in the second image. For specific implementation details, please refer to the technical solution of this embodiment. Technical terms that are the same as or corresponding to those in the above embodiments will not be repeated here. Figure 3 As shown, the method specifically includes the following steps: S210, in response to the projection of a first projection pattern and a second projection pattern by the optical projection device, a first imaging image corresponding to the first projection pattern and a second imaging image corresponding to the second projection pattern are acquired by the image acquisition device. The grayscale information of the first projection pattern and the grayscale information of the second projection pattern are different. The image content of the first projection pattern and the image content of the second projection pattern are the same. Both the first imaging image and the second imaging image are located in the camera image coordinate system.

[0053] Optionally, before projecting the first projection pattern and the second projection pattern in response to the optical projection device, the method further includes: acquiring a first reference image under the first image acquisition condition and a second reference image under the second image acquisition condition; calculating the difference between the grayscale information of the first reference image and the grayscale information of the second reference image to obtain a third image; determining an imaging correction coefficient based on the grayscale information of the third image; and performing flat-field correction on the image acquisition device based on the imaging correction coefficient to obtain a flat-field corrected image acquisition device.

[0054] The first image acquisition condition refers to the pre-set imaging parameters of the image acquisition device. The first image acquisition condition includes, but is not limited to, exposure time, gain, and gamma correction coefficient. For example, the first image acquisition condition could be: the lens cap of the image acquisition device is open, the exposure time is t, automatic gain is off, and automatic exposure is off. The first reference image is the image acquired by the image acquisition device under the first image acquisition condition. Optionally, the first reference image may include a bright-field image.

[0055] The second image acquisition conditions are the pre-set imaging parameters of the image acquisition device. These conditions include, but are not limited to, exposure time, gain, and gamma correction factor. For example, the second image acquisition conditions could be: the lens cap of the image acquisition device is closed, the exposure time is t, automatic gain is off, and automatic exposure is off. The second reference image is the image acquired by the image acquisition device under the second image acquisition conditions. Optionally, the second reference image may include a dark-field image.

[0056] It should be noted that the first image acquisition conditions and the second image acquisition conditions are different. The first image acquisition conditions and the second image acquisition conditions can be set according to the requirements, and the present invention does not impose any restrictions.

[0057] The third image is a grayscale distribution map after removing dark noise from the image acquisition device and interference from ambient light. The third image can be determined based on the first and second reference images. For example, the third image is obtained by calculating the difference between the grayscale information of each pixel position in the first reference image and the corresponding grayscale information of the second reference image. It should be noted that the grayscale information of each pixel position in the third image represents the deviation between the grayscale information of the corresponding pixel position in the first and second reference images.

[0058] Imaging correction coefficients are correction parameters used to correct grayscale distortion caused by differences in pixel sensitivity in image acquisition devices. Imaging correction coefficients can be determined based on the grayscale information of a third image. For example, the grayscale information of the third image can be input into a trained imaging correction coefficient determination model for processing to obtain the imaging correction coefficients. The imaging correction coefficient determination model includes, but is not limited to, neural network models and mathematical models. The model can be selected according to requirements, and this invention is not restrictive. Another example is calculating the reciprocal of the grayscale information of the third image and performing normalization to obtain the imaging correction coefficients.

[0059] Specifically, an image acquisition device acquires a first reference image under a first image acquisition condition and a second reference image under a second image acquisition condition. The difference between the grayscale information of each pixel position in the first reference image and the grayscale information of the corresponding pixel position in the second reference image is calculated to obtain a third image. The reciprocal of the grayscale information of the third image is calculated and normalized to obtain an imaging correction coefficient. Flat-field correction is performed on the image acquisition device based on the imaging correction coefficient to obtain a flat-field corrected image acquisition device. This achieves flat-field correction of the image acquisition device, compensating for the imaging grayscale distortion caused by uneven light sensitivity of the sensor pixels, unifying the imaging response of different areas of the image acquisition device, and thus improving the realism of the first and second imaging images.

[0060] Optionally, in response to the projection of a first projection pattern and a second projection pattern by the optical projection device, a first imaging image and a second imaging image are acquired based on the image acquisition device after flat field correction.

[0061] Specifically, after the image acquisition device performs flat field correction, in response to the projection of the first projection pattern and the second projection pattern by the optical projection device, the flat field-corrected image acquisition device acquires the first imaging image corresponding to the first projection pattern and the second imaging image corresponding to the second projection pattern, respectively. Based on the first imaging image and the second imaging image acquired by the flat field-corrected image acquisition device, the light intensity attenuation correction information corresponding to the optical projection device is determined, so that the acquired first imaging image and the second imaging image can accurately restore the projection light intensity of the optical projection device.

[0062] S220. Calculate the difference between the grayscale information of the first imaging image and the grayscale information of the second imaging image pixel by pixel to obtain the first image. The grayscale information of each pixel position in the first image represents the deviation between the grayscale information of the corresponding pixel position in the first imaging image and the grayscale information of the corresponding pixel position in the second imaging image.

[0063] S230. Perform inverse perspective transformation on the first image based on the homography matrix to obtain the second image. The second image is located in the coordinate system of the optical projection device. The homography matrix is ​​pre-calibrated and is used to characterize the transformation relationship between the coordinate system of the optical projection device and the camera image coordinate system.

[0064] S240. Determine the average grayscale information corresponding to the second image based on the grayscale information of the second image.

[0065] S250. Based on the average grayscale information and the grayscale information of the second image, determine the grayscale compensation coefficients corresponding to each pixel position in the second image.

[0066] S260. The grayscale compensation coefficients corresponding to each pixel position in the second image are truncated to obtain the truncated grayscale compensation coefficients corresponding to each pixel position in the second image; the truncated grayscale compensation coefficients corresponding to each pixel position in the second image are subjected to grayscale mapping to obtain the light intensity attenuation correction information corresponding to the light projection device.

[0067] The process of truncating the grayscale compensation coefficients corresponding to each pixel position in the second image is as follows: A first grayscale compensation threshold and a second grayscale compensation threshold are pre-set, with the first grayscale compensation threshold being less than the second grayscale compensation threshold. Grayscale compensation coefficients greater than the second grayscale compensation threshold are set to the second grayscale compensation threshold, and grayscale compensation coefficients less than the first grayscale compensation threshold are set to the first grayscale compensation threshold, while other grayscale compensation coefficients remain unchanged. For example, the first grayscale compensation threshold and the second grayscale compensation threshold are 0 and 1, respectively. Grayscale compensation coefficients greater than 1 are set to 1, grayscale compensation coefficients less than 0 are set to 0, and grayscale compensation coefficients greater than or equal to 0 and less than or equal to 1 remain unchanged.

[0068] The truncated grayscale compensation coefficients corresponding to each pixel position in the second image are subjected to grayscale mapping processing. This can be done by mapping the truncated grayscale compensation coefficients to the range of 0 to 255. The data type of the mapped grayscale compensation coefficients is then converted to an unsigned integer, thus obtaining the light intensity attenuation correction information corresponding to the optical projection device. Specifically, by truncating the grayscale compensation coefficient, outliers in the grayscale compensation coefficient can be identified, and the grayscale compensation coefficient can be constrained to a reasonable range. Then, the truncated grayscale compensation coefficient is subjected to grayscale mapping processing and converted into light intensity attenuation correction information adapted to the format of light projection devices, thus realizing pixel-level precise light intensity correction and improving the accuracy of light intensity correction.

[0069] The technical solution of this embodiment responds to the projection of a first projection pattern and a second projection pattern by a light projection device. An image acquisition device acquires a first imaging image corresponding to the first projection pattern and a second imaging image corresponding to the second projection pattern. The grayscale information of the first projection pattern and the second projection pattern are different, but the image content of the first projection pattern and the second projection pattern are the same. Both the first and second imaging images are located in the camera image coordinate system, providing comprehensive data support for subsequent analysis and processing. The difference between the grayscale information of the first and second imaging images is calculated pixel-by-pixel to obtain the first image. The grayscale information of each pixel position in the first image represents the deviation between the grayscale information of the corresponding pixel position in the first imaging image and the grayscale information of the corresponding pixel position in the second imaging image. This achieves accurate calculation of the deviation of grayscale information at different pixel positions, providing accurate data support for subsequent analysis and processing. The second image is obtained by performing an inverse perspective transformation on the first image based on the homography matrix. The second image is located in the coordinate system of the optical projection device. The homography matrix is ​​pre-calibrated and used to characterize the transformation relationship between the optical projection device coordinate system and the camera image coordinate system, realizing accurate transformation from the camera image coordinate system to the optical projection device coordinate system. Based on the grayscale information of the second image, the average grayscale information corresponding to the second image is determined, providing a global reference for subsequent analysis and processing. Based on the average grayscale information and the grayscale information of the second image, the grayscale compensation coefficients corresponding to each pixel position in the second image are determined, which can accurately quantify the light intensity deviation of each pixel position in the optical projection coordinate system and realize pixel-level brightness uniformity correction. The grayscale compensation coefficients corresponding to each pixel position in the second image are truncated to obtain the truncated grayscale compensation coefficients corresponding to each pixel position in the second image. Grayscale mapping processing is performed on the truncated grayscale compensation coefficients corresponding to each pixel position in the second image to obtain the light intensity attenuation correction information corresponding to the optical projection device, realizing pixel-level accurate light intensity correction and improving the accuracy of light intensity correction.

[0070] Figure 4 This is a schematic diagram of the structure of an image processing device provided in an embodiment of the present invention. Figure 4 As shown, the device includes an imaging image acquisition module 310, a first image determination module 320, a second image determination module 330, an average grayscale information determination module 340, a grayscale compensation coefficient determination module 350, and a light intensity attenuation correction information determination module 360.

[0071] The imaging image acquisition module 310 is used to respond to the projection of a first projection pattern and a second projection pattern by the optical projection device, and to acquire a first imaging image corresponding to the first projection pattern and a second imaging image corresponding to the second projection pattern based on the image acquisition device. The grayscale information of the first projection pattern and the grayscale information of the second projection pattern are different, but the image content of the first projection pattern and the image content of the second projection pattern are the same. Both the first imaging image and the second imaging image are located in the camera image coordinate system. The first image determination module 320 is used to calculate the difference between the grayscale information of the first imaging image and the grayscale information of the second imaging image pixel by pixel to obtain a first image. The grayscale information of each pixel position in the first image represents the grayscale information of the corresponding pixel position in the first imaging image and the grayscale information of the corresponding pixel position in the second imaging image. The deviation between the information; the second image determination module 330 is used to perform inverse perspective transformation on the first image based on the homography matrix to obtain the second image, the second image is located in the coordinate system of the optical projection device, and the homography matrix is ​​pre-calibrated and used to characterize the transformation relationship between the coordinate system of the optical projection device and the coordinate system of the camera image; the average grayscale information determination module 340 is used to determine the average grayscale information corresponding to the second image based on the grayscale information of the second image; the grayscale compensation coefficient determination module 350 is used to determine the grayscale compensation coefficient corresponding to each pixel position in the second image based on the average grayscale information and the grayscale information of the second image; the light intensity attenuation correction information determination module 360 ​​is used to determine the light intensity attenuation correction information corresponding to the optical projection device based on the grayscale compensation coefficient corresponding to each pixel position in the second image.

[0072] The technical solution of this embodiment, through an imaging image acquisition module, responds to the projection of a first projection pattern and a second projection pattern by a light projection device. Based on the image acquisition device, a first imaging image corresponding to the first projection pattern and a second imaging image corresponding to the second projection pattern are acquired. The grayscale information of the first projection pattern and the second projection pattern are different, but the image content of the first projection pattern and the second projection pattern are the same. Both the first and second imaging images are located in the camera image coordinate system, providing comprehensive data support for subsequent analysis and processing. Through a first image determination module, the difference between the grayscale information of the first imaging image and the second imaging image is calculated pixel by pixel to obtain the first image. The grayscale information of each pixel position in the first image represents the deviation between the grayscale information of the corresponding pixel position in the first imaging image and the grayscale information of the corresponding pixel position in the second imaging image, achieving accurate calculation of the deviation of grayscale information at different pixel positions, providing accurate data support for subsequent analysis and processing. Through a second image determination module... The system performs an inverse perspective transformation on the first image based on the homography matrix to obtain the second image, which is located in the coordinate system of the optical projection device. The homography matrix is ​​pre-calibrated and used to characterize the transformation relationship between the optical projection device coordinate system and the camera image coordinate system, achieving accurate transformation from the camera image coordinate system to the optical projection device coordinate system. An average grayscale information determination module determines the average grayscale information of the second image based on its grayscale information, providing a global reference for subsequent analysis and processing. A grayscale compensation coefficient determination module determines the grayscale compensation coefficient corresponding to each pixel position in the second image based on the average grayscale information and the grayscale information of the second image, accurately quantifying the light intensity deviation at each pixel position in the optical projection coordinate system and achieving pixel-level light intensity uniformity correction. A light intensity attenuation correction information determination module determines the light intensity attenuation correction information corresponding to the optical projection device based on the grayscale compensation coefficients corresponding to each pixel position in the second image, achieving light intensity correction at different pixel positions and improving the accuracy of light intensity correction.

[0073] Based on the above embodiments, optionally, the light intensity attenuation correction information determination module 360 ​​is further configured to: truncate the grayscale compensation coefficients corresponding to each pixel position in the second image to obtain the truncated grayscale compensation coefficients corresponding to each pixel position in the second image; and perform grayscale mapping processing on the truncated grayscale compensation coefficients corresponding to each pixel position in the second image to obtain the light intensity attenuation correction information corresponding to the light projection device.

[0074] Optionally, the device further includes a homography matrix calibration module, used for: responding to the projection of a preset projection dot matrix by the optical projection device, acquiring a dot matrix image corresponding to the projection dot matrix based on the image acquisition device, wherein the projection dot matrix includes multiple marker points and first position information of each marker point in the coordinate system of the optical projection device; performing feature extraction on the dot matrix image to obtain second position information of each projection point in the dot matrix image in the camera image coordinate system; and determining the homography matrix based on the first position information and the second position information.

[0075] Optionally, the second position information includes sub-pixel position information; the homography matrix calibration module is further configured to: extract contours from the projected dot matrix image to obtain contour information corresponding to multiple projection points; fit the contour information to obtain initial position information corresponding to the projection points; extract a local window of a preset size centered on the initial position information; determine the gray-level gradient features corresponding to the local window based on the gray-level information of each pixel position in the local window; and iteratively correct the initial position information based on the preset sub-pixel interpolation model and the gray-level gradient features corresponding to the local window to obtain sub-pixel position information corresponding to each projection point.

[0076] Optionally, the homography matrix calibration module is also used to: construct multiple matching point pairs based on the first position information and the second position information, each matching point pair including the first position information of the marker point in the coordinate system of the optical projection device and the second position information of the corresponding projection point in the coordinate system of the camera image; construct a set of two-dimensional plane projection transformation equations based on the multiple matching point pairs; solve the set of two-dimensional plane projection transformation equations to obtain the homography matrix.

[0077] Optionally, the device further includes a flat-field correction module, configured to: acquire a first reference image under a first image acquisition condition and a second reference image under a second image acquisition condition, respectively, before responding to the projection of the first projection pattern and the second projection pattern by the optical projection device; calculate the difference between the grayscale information of the first reference image and the grayscale information of the second reference image to obtain a third image; determine the imaging correction coefficient based on the grayscale information of the third image; and perform flat-field correction on the image acquisition device based on the imaging correction coefficient to obtain the flat-field corrected image acquisition device.

[0078] Optionally, the imaging image acquisition module 310 is also used to: acquire the first imaging image and the second imaging image based on the flat-field calibrated image acquisition device in response to the projection of the first projection pattern and the second projection pattern by the optical projection device.

[0079] The image processing apparatus provided in this embodiment of the invention can execute an image processing method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0080] Figure 5This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0081] like Figure 5 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0082] Multiple components in electronic device 10 are connected to input / output (I / O) interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of monitors, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0083] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as an image processing method.

[0084] In some embodiments, an image processing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via read-only memory (ROM) 12 and / or communication unit 19. When the computer program is loaded into random access memory (RAM) 13 and executed by processor 11, one or more steps of an image processing method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform an image processing method by any other suitable means (e.g., by means of firmware).

[0085] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0086] A computer program for implementing an image processing method of the present invention can be written in any combination of one or more programming languages. Such computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer program causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer program can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0087] This invention also provides a computer-readable storage medium storing computer instructions for causing a processor to execute an image processing method, the method comprising: In response to the projection of a first projection pattern and a second projection pattern by the optical projection device, a first imaging image corresponding to the first projection pattern and a second imaging image corresponding to the second projection pattern are acquired by the image acquisition device. The grayscale information of the first projection pattern and the second projection pattern are different, but the image content of the first projection pattern and the second projection pattern are the same. Both the first imaging image and the second imaging image are located in the camera image coordinate system. The difference between the grayscale information of the first imaging image and the grayscale information of the second imaging image is calculated pixel by pixel to obtain the first image. The grayscale information of each pixel position in the first image represents the grayscale information of the corresponding pixel position in the first imaging image. The deviation between grayscale information at corresponding pixel positions in the second image; the inverse perspective transformation of the first image is performed based on the homography matrix to obtain the second image, which is located in the coordinate system of the optical projection device. The homography matrix is ​​pre-calibrated and used to characterize the transformation relationship between the coordinate system of the optical projection device and the camera image coordinate system; the average grayscale information corresponding to the second image is determined based on the grayscale information of the second image; the grayscale compensation coefficients corresponding to each pixel position in the second image are determined based on the average grayscale information and the grayscale information of the second image; the light intensity attenuation correction information corresponding to the optical projection device is determined based on the grayscale compensation coefficients corresponding to each pixel position in the second image.

[0088] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0089] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0090] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0091] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0092] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements an image processing method according to any embodiment of the invention.

[0093] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0094] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0095] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An image processing method, characterized in that, include: In response to the projection of a first projection pattern and a second projection pattern by the optical projection device, a first imaging image corresponding to the first projection pattern and a second imaging image corresponding to the second projection pattern are acquired by the image acquisition device. The grayscale information of the first projection pattern and the grayscale information of the second projection pattern are different. The image content of the first projection pattern and the image content of the second projection pattern are the same. Both the first imaging image and the second imaging image are located in the camera image coordinate system. The difference between the grayscale information of the first imaging image and the grayscale information of the second imaging image is calculated pixel by pixel to obtain the first image. The grayscale information of each pixel position in the first image represents the deviation between the grayscale information of the corresponding pixel position in the first imaging image and the grayscale information of the corresponding pixel position in the second imaging image. The first image is subjected to inverse perspective transformation based on the homography matrix to obtain the second image. The second image is located in the coordinate system of the optical projection device. The homography matrix is ​​pre-calibrated and is used to characterize the transformation relationship between the coordinate system of the optical projection device and the coordinate system of the camera image. The average grayscale information corresponding to the second image is determined based on the grayscale information of the second image; Based on the average grayscale information and the grayscale information of the second image, determine the grayscale compensation coefficients corresponding to each pixel position in the second image; The light intensity attenuation correction information for the light projection device is determined based on the grayscale compensation coefficients corresponding to each pixel position in the second image.

2. The method according to claim 1, characterized in that, The step of determining the light intensity attenuation correction information corresponding to the light projection device based on the grayscale compensation coefficients corresponding to each pixel position in the second image includes: The grayscale compensation coefficients corresponding to each pixel position in the second image are truncated to obtain the truncated grayscale compensation coefficients corresponding to each pixel position in the second image. The grayscale compensation coefficients corresponding to each pixel position in the second image are subjected to grayscale mapping processing to obtain the light intensity attenuation correction information corresponding to the light projection device.

3. The method according to claim 1, characterized in that, The calibration process of the homography matrix is ​​as follows: In response to the optical projection device projecting a preset projection dot matrix, the image acquisition device acquires a dot matrix image corresponding to the projection dot matrix. The projection dot matrix includes multiple marker points and first position information of each marker point in the coordinate system of the optical projection device. Feature extraction is performed on the dot matrix image to obtain the second position information of each projection point in the dot matrix image in the camera image coordinate system; The homography matrix is ​​determined based on the first location information and the second location information.

4. The method according to claim 3, characterized in that, The second location information includes sub-pixel location information; The step of extracting features from the dot matrix image to obtain the second position information of each projection point in the dot matrix image in the camera image coordinate system includes: Contour extraction is performed on the projected dot matrix image to obtain contour information corresponding to each of the multiple projected points; The contour information is fitted to obtain the initial position information corresponding to the projection point; Using the initial position information as the center, a local window of a preset size is captured; The gray-level gradient features corresponding to the local window are determined based on the gray-level information of each pixel position in the local window; The initial position information is iteratively corrected based on the preset sub-pixel interpolation model and the gray-level gradient features corresponding to the local window to obtain the sub-pixel position information corresponding to each projection point.

5. The method according to claim 3, characterized in that, Determining the homography matrix based on the first location information and the second location information includes: Multiple matching point pairs are constructed based on the first location information and the second location information. Each matching point pair includes the first location information of the marker point in the coordinate system of the optical projection device and the second location information of the corresponding projection point in the coordinate system of the camera image. A set of two-dimensional plane projection transformation equations is constructed based on the multiple matching point pairs; Solve the system of equations for the two-dimensional plane projection transformation to obtain the homography matrix.

6. The method according to claim 1, characterized in that, Before responding to the projection of the first projection pattern and the second projection pattern by the optical projection device, the method further includes: Acquire a first reference image under the first image acquisition condition and a second reference image under the second image acquisition condition, respectively; The difference between the grayscale information of the first reference image and the grayscale information of the second reference image is calculated to obtain the third image; The imaging correction coefficients are determined based on the grayscale information of the third image; The image acquisition device is flat-field corrected based on the imaging correction coefficient to obtain the flat-field corrected image acquisition device.

7. The method according to claim 6, characterized in that, The method further includes: In response to the projection of the first projection pattern and the second projection pattern by the optical projection device, the first imaging image and the second imaging image are acquired based on the flat-field calibrated image acquisition device.

8. An image processing apparatus, characterized in that, include: An imaging image acquisition module is used to respond to the projection of a first projection pattern and a second projection pattern by a light projection device, and to acquire a first imaging image corresponding to the first projection pattern and a second imaging image corresponding to the second projection pattern based on the image acquisition device. The grayscale information of the first projection pattern and the grayscale information of the second projection pattern are different, the image content of the first projection pattern and the image content of the second projection pattern are the same, and both the first imaging image and the second imaging image are located in the camera image coordinate system. The first image determination module is used to calculate the difference between the grayscale information of the first imaging image and the grayscale information of the second imaging image pixel by pixel to obtain a first image. The grayscale information of each pixel position in the first image represents the deviation between the grayscale information of the corresponding pixel position in the first imaging image and the grayscale information of the corresponding pixel position in the second imaging image. The second image determination module is used to perform inverse perspective transformation on the first image based on the homography matrix to obtain a second image. The second image is located in the coordinate system of the optical projection device. The homography matrix is ​​pre-calibrated and is used to characterize the transformation relationship between the coordinate system of the optical projection device and the camera image coordinate system. The average grayscale information determination module is used to determine the average grayscale information corresponding to the second image based on the grayscale information of the second image; The grayscale compensation coefficient determination module is used to determine the grayscale compensation coefficient corresponding to each pixel position in the second image based on the average grayscale information and the grayscale information of the second image. The light intensity attenuation correction information determination module is used to determine the light intensity attenuation correction information corresponding to the light projection device based on the grayscale compensation coefficients corresponding to each pixel position in the second image.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the image processing method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the image processing method according to any one of claims 1-7.

Images