Image processing methods

Abstract

The present application relates to an image processing method, comprising: acquiring a two-dimensional image and a three-dimensional image of an object; extracting brightness information and depth information from the three-dimensional image to generate a brightness image and a depth image of the object; gray-scaling the two-dimensional image to obtain a single-channel grayscale image; selecting at least three identical points in the grayscale image and the brightness image; calculating a coordinate transformation matrix between the grayscale image and the brightness image based on the coordinates of the at least three identical points in the grayscale image and the brightness image; and aligning the two-dimensional image with the brightness image and the depth image based on the coordinate transformation matrix to obtain fused image data, wherein the fused image data includes color information of the two-dimensional image and brightness information and depth information of the three-dimensional image. The present application also relates to a non-volatile storage medium.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image processing method and a non-volatile storage medium. Background Art

[0002] During the industrial production process, products may contain defects due to process and equipment issues. Therefore, it is necessary to detect defects to improve product yield. Traditional defect detection methods suffer from low efficiency, especially manual inspection. Visual imaging technology can not only improve inspection efficiency but also significantly reduce labor costs. Current visual inspection methods often use CCD cameras to capture two-dimensional (2D) images, which are then used for inspection. However, this 2D image-based inspection method lacks accuracy, which can easily lead to missed defects and false detections. Summary of the Invention

[0003] In view of the above problems, the present application proposes an image processing method and a non-volatile storage medium.

[0004] To this end, a first aspect of the present invention provides an image processing method, wherein the image processing method includes:

[0005] Acquire two-dimensional and three-dimensional images of an object;

[0006] Extracting brightness information and depth information from the three-dimensional image to generate a brightness image and a depth image of the object;

[0007] Grayscale the two-dimensional image to obtain a single-channel grayscale image;

[0008] Select at least three identical position points in the grayscale image and the brightness image;

[0009] Calculating a coordinate transformation matrix between the grayscale image and the luminance image based on coordinates of at least three identical position points in the grayscale image and the luminance image; and

[0010] Based on the coordinate transformation matrix, the two-dimensional image is aligned with the brightness image and the depth image to obtain fused image data, which includes color information of the two-dimensional image and brightness information and depth information of the three-dimensional image.

[0011] In the embodiments of the present application, a two-dimensional image is grayscaled to form a single-channel grayscale image. Then, a coordinate transformation matrix between the two-dimensional image and the three-dimensional image is solved, and the features of the two-dimensional image and the three-dimensional image are aligned to align the feature data in the two-dimensional image and the three-dimensional image to obtain fused image data, thereby achieving image fusion of the two-dimensional image and the three-dimensional image. This fused image data can provide richer information for subsequent deep learning model analysis, thereby obtaining more accurate detection results.

[0012] In the embodiments of the present application, the purpose of grayscale conversion is to facilitate alignment of features in a two-dimensional image with features in a three-dimensional image. In some embodiments, a weighted average method can be used to grayscale a color RGB three-channel image. Specifically, the grayscale value can be calculated according to the following formula:

[0013] p=w1R+w2G+w3B

[0014] Wherein, p represents the gray value of the coordinate point, R represents the red value of the coordinate point, w1 represents the weight of the red value, G represents the green value of the coordinate point, w2 represents the weight of the green value, B represents the blue value of the coordinate point, and w3 represents the weight of the blue value. Studies have shown that the human eye is most sensitive to green, second to red, and least sensitive to blue. Therefore, in some embodiments, w2>w1>w3 can be set. In some embodiments, w1 can be set to 0.299, w2 to 0.587, and w3 to 0.114. This is because experiments and theories have shown that such weight settings can obtain the psychological grayscale that best conforms to human eye perception. The grayscale value obtained by the weighted average method fully retains the red, green, and blue information of the image. It should be understood that other methods can be conceived for grayscale conversion, such as the component method, the maximum value method, the average value method, and the gamma correction method.

[0015] In some embodiments, the at least three identical points are specific points on the object. These specific points are specific locations in a pattern that can be obtained by analyzing the pattern in the image. Therefore, selecting the at least three identical points can be achieved using an artificial intelligence algorithm.

[0016] In some embodiments, the at least three identically located points include three reference points that are not on the same straight line. When the three reference points are on the same straight line, the coordinate transformation matrix cannot be obtained using the coordinates of the three reference points. Therefore, the at least three identically located points need to include three reference points that are not on the same straight line.

[0017] In some preferred embodiments, the maximum interior angle of the triangle formed by the three reference points does not exceed 160°. The larger the maximum interior angle of the triangle formed by the three reference points, the closer the three reference points are to being aligned. Small sampling errors may cause the three reference points to be aligned, rendering the coordinate transformation matrix calculated based on the coordinates of the three reference points in the grayscale image and the luminance image invalid. Therefore, setting the maximum interior angle of the triangle formed by the three reference points to no more than 160° enhances the reliability of the coordinate transformation matrix.

[0018] In some specific embodiments, the image processing method includes:

[0019] Based on the coordinate transformation matrix, aligning the luminance image and the depth image with the two-dimensional image with the coordinates of the two-dimensional image as a reference; and

[0020] The grayscale information of the grayscale image and the brightness information and depth information aligned to the coordinates of the two-dimensional image are respectively used as information of the three channels to generate a fused image containing the three-channel information.

[0021] It should be understood that aligning the luminance image and depth image obtained from the three-dimensional image with the two-dimensional image based on the coordinates of the two-dimensional image is an embodiment of the present application. Those skilled in the art can think of aligning the two-dimensional image with the luminance image based on the coordinates of the luminance image, that is, aligning it to the three-dimensional image, and also achieving the alignment of the two-dimensional image and the three-dimensional image, completing the data alignment of the two images. However, the resolution of the three-dimensional image is low, so it is preferred to align the luminance image and the depth image to the two-dimensional image. In addition, it should be understood that it is not necessary to synthesize a fused image containing information from three channels. It is only necessary to align the information of the two-dimensional image and the three-dimensional image, and save the color information, luminance information, depth information, etc. of each coordinate in the image.

[0022] In some embodiments, the color information, brightness information, and depth information may be channel-superimposed to obtain an image with multi-channel information. In other embodiments, the color information, brightness information, and depth information may be weighted-fused to obtain an image with single-channel or multi-channel information.

[0023] In some specific embodiments, before aligning the two-dimensional image with the brightness image and the depth image, the depth values ​​in the depth image are normalized to generate a normalized depth image. The depth values ​​in the generated depth image may be particularly large, such as values ​​in the tens of thousands, so the depth values ​​can be limited to between 0 and 1 by normalization, which is beneficial for subsequent calculations. In some embodiments, the depth values ​​in the depth image can be normalized according to the following formula:

[0024]

[0025] Among them, d represents the depth value of the coordinate point in the depth image, d min Represents the minimum depth value in the depth image, d max Represents the maximum depth value in the depth image, d S Represents the normalized depth value of the coordinate point in the depth image.

[0026] In some embodiments, the above-mentioned object is selected from any one of the following items: battery surface, battery top cover weld and battery sealing nail. The embodiment of the present application is applicable to scenarios that require both two-dimensional images and three-dimensional images for defect detection. In the battery production process, the detection of scratches on the battery surface, defects in the battery top cover weld and defects in the sealing nail weld are all important links to ensure the safety of the battery leaving the factory. The two-dimensional image cannot directly reflect the depth information of the relevant parts of the battery, and cannot effectively image the depth-sensitive features, while the resolution of the three-dimensional image is low and cannot cover all target features. Therefore, when detecting scratches on the battery surface, defects in the battery top cover weld and defects in the sealing nail weld, the image processing method of the present application can be used to obtain fused image data for corresponding feature detection.

[0027] In some embodiments, the two-dimensional image and the three-dimensional image may be images collected at different locations on the production line. However, in other embodiments, the two-dimensional image and the three-dimensional image may be images collected at the same location on the production line.

[0028] In some embodiments, the two-dimensional image and the three-dimensional image are taken from the same angle. However, in other embodiments, the two-dimensional image and the three-dimensional image are taken from different angles.

[0029] A second aspect of the present application provides a non-volatile storage medium storing instructions, which, when executed by a processor, causes the processor to execute the image processing method described in the first aspect.

[0030] A third aspect of the present application provides an appearance detection method, the method comprising:

[0031] According to the image processing method of the first aspect above, fused image data of multiple objects is obtained;

[0032] Obtain defect determination results for multiple objects;

[0033] Iteratively training the deep learning algorithm model using the fused image data of the multiple objects and the defect determination results to update the deep learning algorithm model;

[0034] Acquire two-dimensional and three-dimensional images of target objects;

[0035] fusing the two-dimensional image and the three-dimensional image of the target object according to the image processing method of the first aspect to obtain fused image data of the target object; and

[0036] The fused image data of the target object is input into the updated deep learning algorithm model to obtain the defect results of the target object.

[0037] In the embodiments of the present application, the defect results of the target object are obtained by substituting the fused image data of the target object for the trained model. On the one hand, the accuracy of the detection is improved, and the cases of missed defects and false detection of defects (false positives) are reduced. On the other hand, since the fused image data is substituted into one model for computational analysis, there is no need to build two models, nor is there any need to substitute the data of different images into two models for computational analysis, which saves computing resources and improves the detection speed. In addition, using the fused image data, only one model needs to be pre-trained instead of two models.

[0038] In some embodiments, the fused image data of the target object includes grayscale information, brightness information, and depth information of the target object, and the deep learning algorithm model includes a grayscale data input channel, a brightness data input channel, and a depth data input channel. In an embodiment of the present application, the deep learning algorithm model may include three data input channels, and the model detects whether there is a defect based on the input grayscale information, brightness information, and depth information.

[0039] In some embodiments, the deep learning algorithm model includes a red data input channel, a green data input channel, a blue data input channel, a brightness data input channel, and a depth data input channel. In the embodiments of the present application, the deep learning algorithm model may include five data input channels, and the model detects the presence of defects based on the input color information, brightness information, and depth information. It should be understood that in practice, those skilled in the art can select the number of data channels of the model based on actual needs.

[0040] In some embodiments, the acquired two-dimensional image of the target object includes a plurality of two-dimensional images.

[0041] In some embodiments, the acquired three-dimensional image of the target object includes a plurality of three-dimensional images.

[0042] A fourth aspect of the present application provides a non-volatile storage medium storing instructions. When the instructions are executed by a processor, the processor executes the appearance inspection method described in the third aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solution of this application, the following is a brief introduction to the drawings required for use in the embodiments of this application. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on the drawings without inventive work. In the drawings:

[0044] Figure 1A flowchart illustrating an image processing method according to one embodiment of the present application is provided;

[0045] Figure 2 A flowchart illustrating an image processing method according to one embodiment of the present application is provided;

[0046] Figures 3a to 3d A photograph showing a two-dimensional image, a luminance image, a depth image, and a fused image according to one embodiment of the present application; and

[0047] Figure 4 A flow chart illustrating an appearance inspection method according to one embodiment of the present application is shown. DETAILED DESCRIPTION

[0048] The following embodiments of the present application are described in detail with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present application and are therefore only examples and are not intended to limit the scope of protection of the present application.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned figure descriptions are intended to cover non-exclusive inclusions.

[0050] Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present application. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0051] If steps are recited in a sequential order in this specification or claims, this does not necessarily mean that embodiments or aspects are limited to the recited order. Rather, it is conceivable that the steps may be performed in a different order or in parallel with one another, unless one step builds upon another, which absolutely requires that the building step be performed later (however, this will become clear in individual cases). Therefore, the recited order may be a preferred embodiment.

[0052] The inventors of this application have found in practice that the current method of detecting product defects using 2D images cannot fully cover the defect morphology in the product because the 2D images fail to directly reflect the depth information of each part of the product, which can easily lead to missed detection and false detection of defects. In addition, the inventors also realized that the 3D images collected by three-dimensional (3D) cameras have a low resolution and are also unable to fully cover the defect morphology in the product, which can easily lead to missed detection and false detection of defects. The inventors thought that since the resolution of 2D images is sufficient, 2D images can be used to preliminarily detect product defects, and then 3D images can be used to re-inspect the product defects, eliminating falsely detected (false positive) defects and adding missed defects.

[0053] Furthermore, the inventors realized that extracting features from 2D and 3D images and then feeding them into two different neural network models for analysis would require two analyses and the training of two neural network models. Based on this, the inventors came up with the idea of ​​fusing the 2D and 3D images to generate the fused image data, which they then fed into a single neural network model for analysis. This approach reduces the required computing resources, increases analysis speed, and improves efficiency.

[0054] Figure 1 The flowchart of the image processing method 100 according to one embodiment of the present application is illustrated. Figure 1 As shown, in step 102, a two-dimensional image and a three-dimensional image of the object are acquired; then, in step 104, the brightness and depth information in the three-dimensional image are extracted to generate a brightness image and a depth image. In step 106, the two-dimensional image is grayscaled to obtain a single-channel grayscale image to facilitate alignment of features in the two-dimensional image with features in the three-dimensional image. After obtaining the brightness image and grayscale image, at least three identical points are selected in the grayscale image and the brightness image, step 108. In step 110, based on the coordinates of the at least three identical points in the grayscale image and the brightness image, a coordinate transformation matrix between the grayscale image and the brightness image is calculated.

[0055] Those skilled in the art will understand that the same feature point in images captured by two different cameras has different image point coordinates. Since the product shape in images captured by different cameras has the same flatness and parallelism, the coordinate transformation between the image point coordinates of different cameras is an affine transformation, and the matrix transformation equation is as follows:

[0056]

[0057] Here, (px, py) represents the coordinates of the image point of one camera, (qx, qy) represents the coordinates of the image point of the other camera, and a1, b1, c1, a2, b2, and c2 represent the transformation coefficients in the coordinate transformation matrix. This affine transformation has six transformation coefficients, that is, six degrees of freedom; therefore, three sets of points are required to calculate the transformation coefficients in the coordinate transformation matrix.

[0058] After calculating the coordinate transformation matrix, the two-dimensional image is aligned with the brightness image and the depth image based on the coordinate transformation matrix to obtain fused image data, step 112. In the embodiment of the present application, the two-dimensional image is grayed to form a single-channel grayscale image, and then the coordinate transformation matrix of the two-dimensional image and the three-dimensional image is solved, and the feature navel of the two-dimensional image and the three-dimensional image is aligned to obtain fused image data, thereby realizing image fusion of the two-dimensional image and the three-dimensional image. This fused image data can provide richer information for subsequent deep learning model analysis to obtain more accurate detection results.

[0059] Figure 2 1 illustrates a flow chart of an image processing method according to one embodiment of the present application. Figure 2 As shown, in step 202, a three-dimensional image of the object is obtained; in step 206, brightness information and depth information in the three-dimensional image are extracted to generate a brightness image and a depth image, as shown in FIG. Figure 3b and Figure 3c Then, in step 210, the depth values ​​in the depth image are normalized to generate a normalized depth image. In some embodiments, the depth values ​​in the depth image can be normalized according to the following formula:

[0060]

[0061] Among them, d represents the depth value of the coordinate point in the depth image, d min Represents the minimum depth value in the depth image, d max Represents the maximum depth value in the depth image, d S The depth value of the coordinate point in the depth image is normalized. The depth value in the depth image generated in step 206 may be very large, such as tens of thousands. Therefore, the depth value can be limited to between 0 and 1 through normalization, which is beneficial for subsequent calculation and processing.

[0062] In step 204, a two-dimensional image of the object is obtained, such as Figure 3a Then, in step 208, the two-dimensional image of the color RGB three-channel is weighted grayscaled to obtain a single-channel grayscale image. Specifically, the grayscale value can be calculated according to the following formula:

[0063] p=w1R+w2G+w3B

[0064] Where p represents the grayscale value of the coordinate point, R represents the red value of the coordinate point, w1 represents the weight of the red value, G represents the green value of the coordinate point, w2 represents the weight of the green value, B represents the blue value of the coordinate point, and w3 represents the weight of the blue value. Studies have shown that the human eye is most sensitive to green, followed by red, and least sensitive to blue. Therefore, in some embodiments, w2>w1>w3 can be set. In some embodiments, w1 can be set to 0.299, w2 to 0.587, and w3 to 0.114. This is because experiments and theory have shown that such weight settings can obtain the psychological grayscale that best conforms to human eye perception. By using the weighted average method to calculate the grayscale value, on the one hand, the algorithm is easier to implement and the amount of calculation is not large, and on the other hand, the red, green, and blue information of the image is fully retained. It should be understood that other methods can be conceived to perform grayscale conversion, such as the component method, the maximum value method, the average value method, and the gamma correction method.

[0065] It should be understood that Figure 2 As shown, steps 202, 206, and 210 can be performed before, after, or simultaneously with steps 204 and 208. After completing steps 206 and 208, three identical locations with specificity can be selected from the grayscale image and the luminance image, proceeding to step 212. These specific locations are specific locations within a pattern that can be obtained by analyzing the pattern in the image. Therefore, selecting the three identical locations with specificity can be achieved using an artificial intelligence algorithm. Figures 3a to 3d This example illustrates a method for detecting welding defects in lithium battery sealing pins according to the present application. In this example, the two center points of the angle between the linear weld and the circular weld, as well as the circular point of the sealing pin, are selected as three unique, identical locations. These three identical locations are not collinear; this is to satisfy the requirements for calculating the transformation coefficients in the coordinate transformation matrix.

[0066] In some preferred embodiments, the maximum interior angle of the triangle formed by the three identical position points does not exceed 160°. The larger the maximum interior angle of the triangle formed by the three identical position points, the closer the three reference points are to being in a straight line. A smaller sampling error may cause the three identical position points to be close to being in a straight line, causing the coordinate transformation matrix calculated based on the coordinates of the three identical position points in the grayscale image and the luminance image to be invalid. Therefore, the maximum interior angle of the triangle formed by the three identical position points is set to no more than 160°, which enhances the reliability of the coordinate transformation matrix. When the three identical position points cannot meet the requirement of not being in a straight line or the maximum interior angle of the triangle formed does not exceed 160°, more identical position points can be selected, as long as three reference points among the selected identical position points meet the requirement of being in a straight line or the maximum interior angle of the triangle formed does not exceed 160°.

[0067] In step 214, based on the coordinates of the three identical points in the grayscale image and the luminance image, the coordinate transformation matrix between the grayscale image and the luminance image is calculated. Figure 1 The process is similar to step 110 in , and the related contents will not be repeated here.

[0068] After calculating the coordinate transformation matrix, the luminance image and the normalized depth image are aligned to a two-dimensional image based on the coordinate transformation matrix, step 216. This allows data at the same location to be aligned in the dimension of the pixel points of the two-dimensional image, ensuring the effectiveness and accuracy of information fusion.

[0069] Finally, in step 218, the coordinate-transformed brightness image and depth image are stacked with the grayscale image to form a fused image with three-channel information, such as Figure 3d As shown. Furthermore, it should be understood that it is not necessary to synthesize a fused image containing information from all three channels. It is only necessary to align the coordinates of the two-dimensional image and the three-dimensional image and preserve the color, brightness, and depth information at each coordinate in the image. In other embodiments, the color, brightness, and depth information can be weightedly fused to obtain an image containing single-channel or multi-channel information.

[0070] The image processing method of the present application is applicable to scenarios that require both two-dimensional images and three-dimensional images for defect detection. In the battery production process, the detection of scratches on the battery surface, defects in the battery top cover welds, and defects in the sealing nail welds are all important links to ensure the safety of batteries leaving the factory. Two-dimensional images cannot directly reflect the depth information of relevant parts of the battery, and cannot effectively image depth-sensitive features, while the resolution of three-dimensional images is low and cannot cover all target features. Therefore, when detecting scratches on the battery surface, defects in the battery top cover welds, and defects in the sealing nail welds, the image processing method of the present application can be used to obtain fused image data for corresponding feature detection. Figures 3a to 3d The embodiment of the lithium battery sealing nail welding defect detection according to the present application is illustrated. However, it should be understood that the object in the image fusion method of the present application can also be the battery surface, the battery top cover weld, etc. Figures 3a to 3d In the embodiment, Figure 3a The two-dimensional image of Figure 3b and Figure 3c The corresponding three-dimensional images are images acquired at different positions of the production line. However, in other embodiments, the two-dimensional image and the three-dimensional image may be images acquired at the same position of the production line.

[0071] Figure 4 The flow chart of the appearance inspection method according to one embodiment of the present application is illustrated. Figure 4 As shown, in step 402, according to the image processing method of the present application, fused image data of multiple objects are obtained. In step 404, defect determination results of multiple objects are obtained. The defect determination results obtained in step 404 are determined by visual inspectors. Then, in step 406, the fused image data and defect determination results of multiple objects are used to iteratively train the deep learning algorithm model to update the deep learning algorithm model. After training a usable deep learning algorithm model, in step 408, two-dimensional images and three-dimensional images of the target object are collected; in step 410, according to the image fusion method of the present application, the two-dimensional image and the three-dimensional image of the target object are fused to obtain fused image data of the target object. Finally, in step 412, the fused image data of the target object is input into the updated deep learning algorithm model for analysis to obtain the defect results of the target object.

[0072] exist Figure 4In this implementation, defect results for the target object are obtained by substituting the fused image data of the target object for the trained model. This not only improves detection accuracy and reduces missed defects and false positives, but also eliminates the need to construct two models or to separately substituting data from different images for analysis, thus saving computing resources and increasing detection speed. Furthermore, using fused image data only requires pre-training one model, rather than two.

[0073] In some embodiments, the fused image data of the target object includes grayscale information, brightness information, and depth information of the target object, and the deep learning algorithm model includes a grayscale data input channel, a brightness data input channel, and a depth data input channel. In such an embodiment, the deep learning algorithm model may include three data input channels, and the model detects the presence of defects based on the input grayscale information, brightness information, and depth information.

[0074] In some embodiments, the deep learning algorithm model includes a red data input channel, a green data input channel, a blue data input channel, a brightness data input channel, and a depth data input channel. In this embodiment, the deep learning algorithm model may include five data input channels, and the model detects defects based on the input color information, brightness information, and depth information. It should be understood that in practice, those skilled in the art may select the number of data channels of the model based on actual needs.

[0075] While the present invention has been described with reference to preferred embodiments, various modifications may be made and equivalent components may be substituted without departing from the scope of the present invention. In particular, the various technical features described in the various embodiments may be combined in any manner, provided no structural conflicts exist. The present invention is not limited to the specific embodiments disclosed herein, but encompasses all technical solutions within the scope of the claims.

Claims

1. An image processing method, characterized in that: The image processing method comprises: Acquire a two-dimensional image and a three-dimensional image of an object, wherein the object is selected from any one of the following items: a battery surface, a battery top cover weld, and a battery sealing nail; Extracting brightness information and depth information from the three-dimensional image to generate a brightness image and a depth image of the object; Gray-scaling the two-dimensional image to obtain a single-channel grayscale image; Selecting at least three identical position points in the grayscale image and the brightness image; Calculating a coordinate transformation matrix between the grayscale image and the luminance image based on the coordinates of the at least three identical position points in the grayscale image and the luminance image; and aligning the two-dimensional image with the luminance image and the depth image based on the coordinate transformation matrix to obtain fused image data, where the fused image data includes color information of the two-dimensional image and luminance information and depth information of the three-dimensional image; The fused image data is input into a trained deep learning model to obtain the defect result of the object.

2. The image processing method according to claim 1, wherein: The at least three identical position points are specific points of the object.

3. The image processing method according to claim 1 or 2, characterized in that: The at least three identical position points include three reference points that are not on the same straight line.

4. The image processing method according to claim 3, wherein: The maximum internal angle of the triangle formed by the three reference points does not exceed 160°.

5. The image processing method according to claim 1, wherein: The image processing method comprises: Based on the coordinate transformation matrix, aligning the luminance image and the depth image with the two-dimensional image with the coordinates of the two-dimensional image as a reference; and The grayscale information of the grayscale image and the brightness information and depth information aligned to the coordinates of the two-dimensional image are respectively used as information of three channels to generate a fused image containing the three-channel information.

6. The image processing method according to claim 1, wherein: Before aligning the two-dimensional image with the brightness image and the depth image, the depth values ​​in the depth image are normalized to generate a normalized depth image.

7. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores instructions, and when the instructions are executed by a processor, the instructions cause the processor to execute the image processing method according to any one of claims 1 to 6.

Images