Image registration method, device, optical system and medium

Abstract

The present invention discloses an image registration method, device, optical system and medium, including: obtaining a calibration plate image, the calibration plate image is obtained by shooting the calibration plate at different positions based on a shooting device in a multi-channel optical system, and includes a 2D calibration plate image and a 3D calibration plate image, the position of the calibration plate is preset, based on a preset 2D dimensional image algorithm and a 3D dimensional image algorithm, the 2D calibration plate image and the 3D calibration plate image are evaluated respectively to obtain a 2D dimensional evaluation result and a 3D dimensional evaluation result, and image registration is performed on the calibration plate image according to the 2D dimensional evaluation result and / or the 3D dimensional evaluation result. It can be seen that the present application evaluates the calibration plate image by using a preset dimensional image algorithm to obtain a 2D dimensional evaluation result and a 3D dimensional evaluation result, and then performs accurate image registration on the calibration plate image according to the dimensional evaluation result, thereby improving the registration accuracy of the multi-channel image.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image registration method, device, optical system and medium. Background Art

[0002] Multi-channel optical imaging systems are widely used in microscopic imaging and automated optical inspection, such as semiconductor inspection, display semiconductor inspection, and biological inspection, where the simultaneous acquisition of rich optical information and the need for high-throughput production are crucial. However, when multi-channel imaging is used in applications requiring higher precision, the information from multiple channels must be integrated and mapped one-to-one. Traditional methods that rely solely on subjective image observation cannot achieve the most perfect assembly guidance.

[0003] Currently, traditional image registration methods have certain limitations, especially in scenarios where multi-channel imaging systems need to be quickly built. Each channel has its own aberrations, and image registration accuracy is significantly affected by the cumulative tolerances of optical and mechanical setups and hardware structures. This makes it impossible to perfectly align images between channels, seriously affecting detection results and resulting in low image registration accuracy. Therefore, improving the registration accuracy of multi-channel images is a technical problem that needs to be solved urgently. Summary of the Invention

[0004] Based on this, it is necessary to address the above technical problems. The embodiments of the present invention provide an image registration method, device, optical system and medium, which can accurately register the calibration plate image and improve the registration accuracy of multi-channel images.

[0005] A first aspect of an embodiment of the present application provides an image registration method, the image registration method comprising:

[0006] Acquire a calibration plate image, wherein the calibration plate image is obtained by photographing the calibration plate at different positions using a photographing device in the multi-channel optical system, the calibration plate image includes a 2D calibration plate image and a 3D calibration plate image, and the position of the calibration plate is preset;

[0007] The 2D calibration plate image is evaluated using a preset 2D dimensional image algorithm to obtain a 2D dimensional evaluation result, and the 3D calibration plate image is evaluated using a preset 3D dimensional image algorithm to obtain a 3D dimensional evaluation result, wherein both the 2D dimensional evaluation result and the 3D dimensional evaluation result include a camera status dimension and an image imaging status dimension;

[0008] Image registration is performed on the calibration plate image according to the 2D dimension evaluation result and / or the 3D dimension evaluation result.

[0009] A second aspect of the embodiments of the present application provides an image registration device, the image registration device comprising:

[0010] an acquisition module, configured to acquire a calibration plate image, wherein the calibration plate image is obtained by photographing the calibration plate at different positions using a photographing device in a multi-channel optical system, the calibration plate image includes a 2D calibration plate image and a 3D calibration plate image, and the position of the calibration plate is preset;

[0011] an evaluation module, configured to evaluate the 2D calibration plate image using a preset 2D dimensional image algorithm to obtain a 2D dimensional evaluation result, and to evaluate the 3D calibration plate image using a preset 3D dimensional image algorithm to obtain a 3D dimensional evaluation result, wherein both the 2D dimensional evaluation result and the 3D dimensional evaluation result include a camera status dimension and an image imaging status dimension;

[0012] A registration module is used to perform image registration on the calibration plate image according to the 2D dimension evaluation result and / or the 3D dimension evaluation result.

[0013] A third aspect of the embodiments of the present application provides a multi-channel optical system, the multi-channel optical system comprising a shooting device, a calibration plate, and a control module;

[0014] The control module is used to obtain a calibration plate image, wherein the calibration plate image is obtained by photographing the calibration plate at different positions based on a photographing device in a multi-channel optical system, and the calibration plate image includes a 2D calibration plate image and a 3D calibration plate image, and the position of the calibration plate is pre-set; the 2D calibration plate image is evaluated using a preset 2D dimensional image algorithm to obtain a 2D dimensional evaluation result, and the 3D calibration plate image is evaluated using a preset 3D dimensional image algorithm to obtain a 3D dimensional evaluation result, wherein the 2D dimensional evaluation result and the 3D dimensional evaluation result both include a photographing device status dimension and an image imaging status dimension; and image registration is performed on the calibration plate image according to the 2D dimensional evaluation result and / or the 3D dimensional evaluation result.

[0015] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image registration method as described in the first aspect is implemented.

[0016] In summary, the present invention provides an image registration method, device, optical system and medium to obtain a calibration plate image, wherein the calibration plate image is obtained by photographing the calibration plate at different positions based on a shooting device in a multi-channel optical system, and the calibration plate image includes a 2D calibration plate image and a 3D calibration plate image, and the position of the calibration plate is pre-set. The 2D calibration plate image is evaluated using a preset 2D dimensional image algorithm to obtain a 2D dimensional evaluation result, and the 3D calibration plate image is evaluated using a preset 3D dimensional image algorithm to obtain a 3D dimensional evaluation result, wherein the 2D dimensional evaluation result and the 3D dimensional evaluation result both include a shooting device status dimension and an image imaging status dimension, and image registration is performed on the calibration plate image based on the 2D dimensional evaluation result and / or the 3D dimensional evaluation result. It can be seen that this application evaluates the acquired 2D calibration plate image and 3D calibration plate image based on the preset 2D dimensional image algorithm and 3D dimensional image algorithm, respectively, and obtains 2D dimensional evaluation results and 3D dimensional evaluation results. Then, based on the 2D dimensional evaluation results and / or the 3D dimensional evaluation results, the calibration plate image can be accurately aligned, thereby improving the alignment accuracy of the multi-channel image. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0018] Figure 1 is a schematic structural diagram of a multi-channel optical system provided by one embodiment of the present invention;

[0019] Figure 2-3 2D and 3D calibration plates according to an embodiment of the present invention;

[0020] Figure 4 is a schematic structural diagram of another multi-channel optical system provided by one embodiment of the present invention;

[0021] Figure 5 is a flow chart of an image registration method provided by one embodiment of the present invention;

[0022] Figure 6 It is a structural diagram of an image registration device provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0023] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of the present invention.

[0024] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0025] It will also be understood that the term "and / or" used in the present description and appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0026] As used in the present specification and the appended claims, the term “if” may be interpreted as “when” or “upon” or “in response to determining”, depending on the context. Similarly, the phrase “if it is determined” or “if compared to [described condition or event]” may be interpreted as meaning “upon determination” or “in response to determination” or “upon comparison to [described condition or event]” or “in response to comparison to [described condition or event]”, depending on the context.

[0027] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0028] References to "one embodiment" or "some embodiments" in the present specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present invention. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0029] It should be understood that the order of execution of the steps in the following embodiments does not necessarily mean the order in which they are executed. The order in which each process is executed should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0030] The present invention provides an image registration method, device, optical system and medium, which specifically relate to the field of multi-channel image registration technology, and are particularly suitable for scenes requiring high-precision pixel-level fusion such as microscopic imaging and semiconductor detection, and are used to align multi-channel images, eliminate aberrations between channels, and accumulate mechanical tolerance differences to support image registration of multi-channel images. In terms of hardware, the flexibility of the optical element layout is retained, and by combining a shooting device and a 2D or 3D calibration plate, it is convenient to flexibly configure according to different detection requirements; in terms of software, through the image registration method of the present invention, the images of each channel can be made one-to-one corresponding, thereby improving the image registration accuracy; at the same time, the imaging efficiency of the optical system can be retained, ensuring the efficient conduct of the detection work, and can be applied to the calibration and adjustment specifications of various indicators of the optical system.

[0031] In order to illustrate the technical solution of the present invention, specific embodiments are provided below.

[0032] See also Figure 1 , Figure 1 This is a structural diagram of a multi-channel optical system provided by one embodiment of the present invention. The multi-channel optical system includes a shooting device, a calibration board and a control module. The shooting device and the control module can be directly or indirectly connected through wired or wireless communication, which is not specifically limited in this application.

[0033] The camera is any device with a camera function. Each camera is equipped with a six-dimensional adjustment mechanism to ensure sufficient adjustment freedom. It is used to capture the calibration plate image at different positions and send the captured calibration plate image to a control module (the control module can be set at an external server, computer, or the camera). The control module then adjusts the dimensions of the camera based on the 2D dimensional evaluation results and / or 3D dimensional evaluation results calculated by the image algorithm, thereby achieving image registration of the calibration plate image. It should be noted that the number of the camera can be one or more, and this application does not specifically limit this.

[0034] The calibration plate includes 2D calibration plate and 3D calibration plate, such as Figure 2-3As shown, the 2D calibration plate contains N*N partitions (depending on the registration accuracy requirements), each partition has a marker point, which is evenly distributed and consistent. The number of partitions depends on the registration accuracy requirements. The higher the accuracy requirements, the more partitions there are. The marker point is a single graphic within the partition. Here, a cross is used as a feature mark as an example. It should be noted that the marker point is not limited to square, circle and other patterns; the 3D calibration plate contains N*N partitions (depending on the registration accuracy requirements), each partition is composed of microstructures of different heights (different colors represent inconsistent heights in the area), and high consistency can be ensured through precision processing. The surface of the calibration plate is covered with a high reflectivity coating to enhance image contrast.

[0035] The control module is used to obtain a calibration plate image, which includes a 2D calibration plate image and a 3D calibration plate image. After obtaining the calibration plate image, the 2D calibration plate image is evaluated by using a preset 2D dimensional image algorithm to obtain a 2D dimensional evaluation result, and the 3D calibration plate image is evaluated by using a preset 3D dimensional image algorithm to obtain a 3D dimensional evaluation result, wherein the 2D dimensional evaluation result and the 3D dimensional evaluation result both include the camera status dimension and the image imaging status dimension, and then the dimension of the camera is adjusted according to the 2D dimensional evaluation result and / or the 3D dimensional evaluation result, thereby achieving image registration of the 2D calibration plate image and / or the 3D calibration plate image. It can be seen that by combining the camera device and the 2D and 3D calibration plates, as well as combining a detailed image algorithm, the 2D calibration plate image and / or the 3D calibration plate image are image registered, thereby improving the registration accuracy of the multi-channel image, and ultimately achieving efficient alignment of different channel images at the same hardware level, thereby improving the accuracy and efficiency of optical detection.

[0036] It can be understood that the structural diagram of the multi-channel optical system described in the embodiment of the present application is for more clearly illustrating the technical solution of the embodiment of the present application, and does not constitute a limitation on the technical solution provided in the embodiment of the present application. A person skilled in the art can know that with the evolution of the system structure and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.

[0037] See also Figure 4 , Figure 42 is a schematic structural diagram of another multi-channel optical system provided by an embodiment of the present invention. The multi-channel optical system has two or more imaging devices, each of which is equipped with a six-dimensional adjustment mechanism to ensure sufficient adjustment freedom. The distance between the lower end face of the objective lens of the multi-channel optical system and the sample surface is fixed and fixed, which is defined as the working distance. The calibration process is as follows: a calibration plate is placed at the working distance position, i.e., the object focal plane, and then fixed. A reference channel is pre-set. For example, when using imaging device 1, several dimensions of imaging device 1 are first adjusted to the optimal state so that imaging device 1 can produce clear images. Then, using imaging devices 2 / 3, the calibration plate is focused to achieve a clear state. At this time, the two channels may not yet achieve global clarity, so adjustments need to be made based on the feedback from the calibration plate. At this time, the channels are not perfectly aligned with imaging device 1. Then, by adjusting the six axes of imaging devices 2 / 3, each adjustment is performed by capturing images from imaging devices 1 / 2 / 3, i.e., each capture results in three images. The three calibration plate images are fused or pseudo-colored, and the current registration quality is evaluated. The above steps are repeated until the acquired image achieves the optimal effect.

[0038] It can be seen that the above Figure 1 and Figure 4 The system architecture shown is only an example and is not specifically limited in the embodiments of the present application.

[0039] See also Figure 5 , is a flow chart of an image registration method provided by an embodiment of the present invention, such as Figure 5 As shown, the image registration method can be implemented through the following steps.

[0040] S501: Acquire a calibration plate image, wherein the calibration plate image is obtained by photographing the calibration plate at different positions using a photographing device in a multi-channel optical system, the calibration plate image includes a 2D calibration plate image and a 3D calibration plate image, and the position of the calibration plate is pre-set.

[0041] In step S501, the number of the photographing devices is not limited, and each photographing device is equipped with a six-dimensional adjustment mechanism to ensure sufficient degree of freedom in adjustment. The calibration plate includes a 2D calibration plate and a 3D calibration plate, such as Figure 2-3As shown, a 2D calibration plate contains N*N partitions (depending on the required registration accuracy), each with a uniformly distributed and consistent marker. The number of partitions depends on the required registration accuracy; higher accuracy requires more partitions. A marker is a single pattern within a partition. Here, a cross is used as a characteristic marker, but it should be noted that markers are not limited to squares, circles, or other patterns. A 3D calibration plate contains N*N partitions (depending on the required registration accuracy), each composed of microstructures of varying heights (different colors represent areas of varying height). High consistency can be achieved through precision machining, and the surface of the calibration plate is coated with a high-reflectivity coating to enhance image contrast. The calibration plate's position is pre-set within the imaging range of the multi-channel optical system. Using a positioning device, the calibration plate is precisely placed on the sample holder, below the objective lens near the imaging focal plane, and the plate is held in place to ensure its stable position. A camera captures images of the calibration plate at each pre-set position, ensuring consistent lighting conditions and avoiding interference from shadows and reflections. For 3D calibration plates, images can also be captured from different angles to obtain more images.

[0042] In one embodiment of the invention, obtaining a calibration plate image includes:

[0043] Acquire an initial calibration plate image obtained by photographing the calibration plate at multiple positions by the photographing device, wherein there are multiple initial calibration plate images;

[0044] Determining the focus position of the camera according to the clarity of each of the initial calibration plate images;

[0045] Acquire a calibration plate image obtained by photographing the calibration plate by the photographing device at the focus position and at multiple defocus positions within a defocus area, wherein the defocus area is centered on the focus position, and a distance between an edge position of the defocus area and the focus position is preset.

[0046] Specifically, the camera acquires initial calibration plate images from multiple locations, and then evaluates the clarity of each initial calibration plate image. Common clarity evaluation metrics include grayscale variance and the Laplace operator. The image with the highest clarity is selected as the focused image at that location, and the location where it was captured is recorded as the focused position. This means that when the camera is in the focused position, the image has the highest clarity. The closer the camera is to the focused position, the clearer the image; the farther away from the focused position, the blurrier the image.

[0047] Furthermore, by using the clarity of each image in the initial calibration plate image as the vertical coordinate and the position of the camera as the horizontal coordinate, when the image clarity is the highest, the position of the camera corresponding to the camera is the focus position. For example, the camera moves along the Z axis with a step size of 1mm, and captures one calibration plate image at each position z=1, 2, ..., 5mm where the camera is located. The corresponding clarity is 280, 450, 680, 620, 510, respectively. Then, the position z=3 where the camera is located is the focus position. With the focus position as the center, a defocus area is set. The distance between the edge position of the defocus area and the focus position can be pre-set according to actual needs. Within the defocus area, multiple defocus positions are selected, and these positions should be evenly distributed to cover different parts of the defocus area. With the focus position as the center, the camera is controlled to photograph the calibration plate at multiple positions within the defocus area, excluding the focus position, to obtain calibration plate images obtained by photographing the calibration plate at the focus position and at multiple defocus positions within the defocus area. The light spot in the image photographed at the focus position is clear, while the light spot in the image photographed at multiple defocus positions is blurred. The range of the defocus area is determined based on the performance parameters of the camera, i.e., the range of the defocus area of ​​a multi-channel optical system including different camera devices is different. By taking the calibration plate images at the focus position and the defocus position through the above steps, the defocus characteristics of the camera can be further analyzed, which helps to improve image quality and reduce problems such as image distortion and chromatic aberration.

[0048] In the embodiment of the present application, by acquiring the calibration plate image, the calibration plate image information can be more comprehensively understood, and the accuracy and stability of image acquisition by the multi-channel optical system can be improved, so that the image registration accuracy and efficiency of the calibration plate image can be subsequently improved.

[0049] S502: Use a preset 2D dimensional image algorithm to evaluate the 2D calibration plate image to obtain a 2D dimensional evaluation result, and use a preset 3D dimensional image algorithm to evaluate the 3D calibration plate image to obtain a 3D dimensional evaluation result, wherein the 2D dimensional evaluation result and the 3D dimensional evaluation result both include a shooting device status dimension and an image imaging status dimension.

[0050] In step S502, the camera status dimension may include an attitude angle sub-dimension and / or an installation position sub-dimension, wherein the attitude angle of the camera can represent the relationship between the coordinate system of the camera body and the geographic coordinate system. In some cases, the attitude angle may also be called the Euler angle or other names, which are not limited in the embodiments of the present application. The installation position sub-dimension includes the longitude information, latitude information, and altitude information of the camera; the image imaging status dimension may include the distribution sub-dimension of feature points, the accuracy sub-dimension of depth information, and the image clarity sub-dimension. The present application evaluates the 2D calibration plate image and the 3D calibration plate image respectively by using a preset image algorithm to obtain a 2D dimension evaluation result and a 3D dimension evaluation result.

[0051] In one embodiment of the invention, the 2D calibration plate image includes a reference channel image and other channel images. The 2D calibration plate image is evaluated using a preset 2D dimension image algorithm to obtain a 2D dimension evaluation result, including:

[0052] Performing image fusion on the reference channel image and the other channel images respectively to obtain a plurality of three-dimensional color image matrices, wherein each of the three-dimensional color image matrices corresponds to a respective channel image;

[0053] Analyzing and calculating each pixel in each of the three-dimensional color image matrices to obtain the balance of the RGB value of each pixel;

[0054] performing histogram statistics on each of the three-dimensional color image matrices according to the balance of the RGB values ​​of each pixel to obtain a plurality of histogram distribution information;

[0055] A 2D dimension evaluation result is determined based on the histogram distribution information.

[0056] Specifically, the 2D calibration plate image includes a reference channel image and other channel images, and the reference channel image generally refers to a channel image used as a reference or benchmark, and other channel images refer to other channel images except the reference channel image. It should be noted that the reference channel image can be the channel image with the highest clarity or the channel image with the highest resolution, and this application does not impose any restrictions on this. Each channel contains multiple images, and the reference channel image and other channel images are fused separately. Commonly used image fusion methods include pyramid fusion, Poisson fusion, etc. These methods can be selected according to specific needs, and this application does not impose any restrictions on this. By fusing images of multiple channels, multiple three-dimensional color image matrices containing different information combinations can be obtained. Each three-dimensional color image matrix is ​​a data cube containing three RGB channels, and each pixel has a corresponding RGB value. By extracting the RGB value of each pixel in each three-dimensional color image matrix and analyzing it, the balance of the RGB value of each pixel is obtained. Then, based on the balance of the RGB value of each pixel, each three-dimensional color image matrix is ​​histogrammed to obtain multiple histogram distribution information. The histogram can display the distribution of the RGB value of each pixel, helping to understand the overall color characteristics and balance of the image. Then, by observing and analyzing the distribution information of the histogram, including peak values, valley values, distribution shape, etc., it is determined whether the 2D dimension evaluation results meet the preset 2D dimension evaluation conditions.

[0057] After obtaining the reference channel image and other channel images (all channel images are grayscale images), each channel image is fused separately. Here, taking three channel images as an example, the reference channel image is recorded as Gr, and the other channel images are recorded as Gg and Gb. The image size corresponding to each partition is m×n (i.e., there are m rows and n columns). Then, a color image is generated, represented as a three-dimensional matrix C with dimensions of m×n×3. For each pixel (i, j), the RGB value of the color image can be expressed as:

[0058] C[i, j, 0] = Gr[i, j] (red channel - first channel)

[0059] C[i, j, 1] = Gg[i, j] (green channel - second channel)

[0060] C[i, j, 2] = Gb[i, j] (blue channel - third channel)

[0061] Where (i, j) represents the pixel position on the image. Positions with channel index 0 take the corresponding pixel value in the reference channel image Gr, positions with channel index 1 take the corresponding pixel value in the other channel image Gg, and positions with channel index 2 take the corresponding pixel value in the other channel image Gb. For example, if a pixel has a value of 200 in Gr, 50 in Gg, and 30 in Gb, then the RGB value of that pixel is (200, 50, 30). The 3D matrix corresponding to the color image is then analyzed. For each pixel (i, j), the balance of each pixel's RGB value is calculated. Based on the balance of each pixel's RGB value, the 3D color image matrices are histogrammed to generate multiple histogram distribution information, reflecting the distribution of pixels with different levels of balance within the image. Through these steps, the balance and overall quality of each channel in a multi-channel image can be quickly and accurately assessed, providing a more comprehensive understanding of the image's features and details.

[0062] In one embodiment of the invention, analyzing and calculating each pixel in each of the three-dimensional color image matrices to obtain the balance of the RGB values ​​of each pixel includes:

[0063] Calculating the difference between the maximum RGB value and the minimum RGB value of each pixel in the three-dimensional color image matrix, and determining whether the difference is less than a preset difference threshold;

[0064] If the difference is less than a preset difference threshold, determining that the RGB components of the pixel are balanced;

[0065] Calculating the R channel deviation, G channel deviation, and B channel deviation of each pixel in each of the three-dimensional color image matrices, and determining whether the R channel deviation, G channel deviation, and B channel deviation are all less than a preset deviation threshold;

[0066] If the R channel deviation, the G channel deviation, and the B channel deviation are all less than the preset deviation threshold, determining the RGB color balance of the pixel;

[0067] Normalizing the RGB value of each pixel in each of the three-dimensional color image matrices to obtain a standardized RGB value of each pixel;

[0068] Determine whether the normalized RGB value of each pixel is close to a preset RGB threshold;

[0069] If the normalized RGB value of each pixel is close to a preset RGB threshold, determining that the RGB color difference of the pixel is balanced;

[0070] The balance of the RGB value of each pixel is determined based on the RGB component balance of the pixel, the RGB color balance of the pixel, and the RGB color difference balance of the pixel.

[0071] Specifically, for each pixel in each three-dimensional color image matrix, the RGB value of each pixel is extracted respectively, and the difference between the maximum and minimum values ​​of each pixel RGB value is calculated. The difference formula is: Range = max(R, G, B) − min(R, G, B). By judging whether this difference is less than the preset difference threshold, if it is less than, the RGB component balance of the pixel is determined. For each pixel, the mean of the three channels R, G, and B of each pixel in each three-dimensional color image matrix is ​​first calculated. The mean formula is: Mean = (R+G+B) / 3. This mean is used as the reference value for color balance. Then, the deviation of the red (R), green (G), and blue (B) channel values ​​in each pixel from the above mean is calculated respectively, that is, the R channel deviation: , G channel deviation and B channel deviation , and performs a threshold comparison using a preset deviation threshold T: When ΔR < T, ΔG < T, and ΔB < T, meaning the R, G, and B channel deviations are all less than the preset deviation thresholds, the pixel's RGB color balance is determined. If any channel deviation exceeds the preset deviation threshold, the pixel exhibits color imbalance. Each pixel's RGB value in each three-dimensional color image matrix is ​​normalized to determine whether the normalized RGB value is close to the preset RGB threshold. If the normalized RGB value is close to the preset RGB threshold, the pixel's RGB color difference balance is determined. The balance of each pixel's RGB value is then comprehensively determined based on the pixel's RGB component balance, RGB color balance, and RGB color difference balance. By calculating the difference between the maximum and minimum RGB values, the deviations of the three R, G, and B channels, and the normalized RGB values, the balance of each pixel's RGB value can be comprehensively and accurately assessed, enabling timely detection and correction of color issues in the image, improving overall image quality and registration accuracy.

[0072] It should be noted that the preset difference threshold, the preset deviation threshold and the preset RGB threshold can be set according to actual conditions, and this application does not impose any limitation on this.

[0073] In one embodiment of the invention, the 3D calibration plate image is evaluated using a preset 3D dimensional image algorithm to obtain a 3D dimensional evaluation result, including:

[0074] Processing the 3D calibration plate image using a focus measurement function to generate a plurality of depth maps;

[0075] Calculating the depth deviation of the same pixel marker point in each of the depth maps;

[0076] Determining a root mean square error (RMS) of all pixel markers in each depth map based on a depth deviation of the same pixel marker in each depth map;

[0077] A 3D dimension evaluation result is determined according to a root mean square error of all pixel marker points in each of the depth maps.

[0078] Specifically, the image clarity is evaluated by selecting a suitable focus measurement function (such as gradient function, Laplace operator, etc.), and the focus measurement function is applied to the 3D calibration plate image of each channel to calculate the clarity value of each pixel. For each pixel, the position with the highest clarity in the 3D calibration plate image is found, which is the best focus position of the pixel. According to the best focus position of each pixel, the best focus position is divided into different depth levels, and each pixel is divided into the corresponding depth level, thereby forming multiple depth maps. The depth map values ​​represent the distance or depth from the pixel to the camera. Pixel markers are selected from each depth map. These are typically prominent features or artificially marked points in the image. For each pixel marker, the depth deviation between its depth values ​​in each depth map is calculated. For all selected pixel markers, the squared sum of their depth deviations is calculated. This sum of squared depth deviations is divided by the number of pixel markers to obtain the root mean square error (RMSE). The RMSE is used to assess overall registration accuracy and reflects the degree of consistency between the depth maps. A preset RMSE threshold is then set. If the calculated RMSE is less than the preset RMSE threshold, the depth maps are considered highly consistent and the 3D evaluation results meet the preset 3D evaluation criteria. Otherwise, the consistency is considered poor and the 3D evaluation results do not meet the preset 3D evaluation criteria, requiring further optimization or adjustment. By calculating the depth deviation and RMSE for the same pixel marker in each depth map, the consistency between multiple depth maps can be assessed, helping to identify and correct potential errors, thereby improving image registration efficiency.

[0079] In this embodiment, by using a preset image algorithm to evaluate the 2D calibration plate image and the 3D calibration plate image respectively, the 2D dimensional evaluation results and the 3D dimensional evaluation results are obtained, which can more comprehensively reflect the multi-channel image imaging status, so that the shooting device can be dimensionally adjusted according to the 2D dimensional evaluation results and the 3D dimensional evaluation results, so that the images of each channel can correspond one to one, thereby improving the image registration accuracy and stability, and ensuring the imaging quality and performance.

[0080] S503: Perform image registration on the calibration plate image according to the 2D dimension evaluation result and / or the 3D dimension evaluation result.

[0081] In step S503, since both the 2D dimension evaluation results and the 3D dimension evaluation results include the camera status dimension and the image imaging status dimension, and the camera status dimension may include the attitude angle sub-dimension and / or the installation position sub-dimension, wherein the attitude angle of the camera can represent the relationship between the coordinate system of the camera body and the geographic coordinate system. In some cases, the attitude angle may also be called the Euler angle or other names, which is not limited in the embodiment of the present application. The installation position sub-dimension includes the longitude information of the camera, the latitude information of the camera, and the height information of the camera; the image imaging status dimension may include the distribution sub-dimension of feature points, the accuracy sub-dimension of depth information, and the image clarity sub-dimension. By using the 2D dimension evaluation results and / or the 3D dimension evaluation results, areas with poor imaging quality or obvious errors are identified. In 2D images, these areas may appear blurred, distorted, or have sparse feature points. In 3D images, these areas may appear as inaccurate depth information or deviations in 3D model reconstruction.

[0082] For 2D or 3D image regions with poor imaging quality, image registration can be optimized by adjusting the transformation matrix or the dimensions of the camera. For example, a depth correction factor can be introduced to modify the transformation matrix, or a more complex 3D registration algorithm can be used to process depth information. The specially processed images are then re-registered, and a new dimensionality evaluation result is calculated. Based on this new dimensionality evaluation result, the registration parameters and strategy are further adjusted until satisfactory registration accuracy is achieved.

[0083] In one embodiment of the invention, performing image registration on the calibration plate image according to the 2D dimension evaluation result includes:

[0084] Determine whether the 2D dimension evaluation result meets the preset 2D dimension evaluation conditions;

[0085] If the 2D dimension evaluation result does not meet the preset 2D dimension evaluation conditions, the X-axis position and / or Y-axis position of the shooting device in the multi-channel optical system is adjusted, and the calibration plate image is aligned using the preset rigid mapping algorithm and non-rigid mapping algorithm.

[0086] Specifically, the preset 2D dimension evaluation conditions are set according to application requirements, and this application does not impose any restrictions on this, and are used to determine whether the 2D dimension evaluation results meet specific standards. These conditions may include image clarity, uniformity of feature point distribution, accuracy of depth information, and so on. For example, it can be set that the image clarity must be higher than a certain threshold, the feature point distribution must meet certain uniformity requirements, the depth information error must be less than a certain preset value, etc. The 2D dimension evaluation result is compared with the preset 2D dimension evaluation conditions. If the 2D dimension evaluation result meets the preset 2D dimension evaluation conditions, no further adjustment is required; if the 2D dimension evaluation result does not meet the preset 2D dimension evaluation conditions, then the X-axis position and / or Y-axis position of the shooting device in the multi-channel optical system are adjusted according to the shooting device status dimension. For example, if the image clarity is insufficient, it may be necessary to adjust the focal length or aperture of the lens; if the feature points are unevenly distributed, it may be necessary to adjust the shooting angle or position, etc. The calibration plate images are then registered using pre-set rigid mapping and non-rigid mapping algorithms. Rigid mapping algorithms handle rigid transformations such as translation, rotation, and scaling between images, while non-rigid mapping algorithms handle local deformation and non-rigid transformations between images. This applies when low precision is required. If the distortion of the multiple channels in a multi-channel optical system is not linear, a non-rigid mapping algorithm is required. Rigid mapping algorithms align the calibration plate images through rotation and translation transformations, preserving the image shape and size. Non-rigid mapping algorithms use B-spline transformations, thin plate splines, or optical flow methods to adjust the shape or generate smooth deformations using control points. This can achieve smooth shape changes, which can be achieved using more complex transformation models such as elastic deformation or free-form shape transformations. Thus, by determining the 2D dimensionality evaluation results and adjusting the X- and / or Y-axis positions of the camera, image quality can be ensured to meet registration requirements, thereby improving image registration accuracy and efficiency and enhancing the robustness of the multi-channel optical system.

[0087] In one embodiment of the invention, performing image registration on the calibration plate image according to the 3D dimension evaluation result includes:

[0088] Determining whether the 3D dimension evaluation result meets the preset 3D dimension evaluation conditions;

[0089] If the 3D dimensional evaluation result does not meet the preset 3D dimensional evaluation conditions, the Z-axis position and / or objective lens spacing of the shooting device in the multi-channel optical system are adjusted, and the calibration plate image is aligned using the preset chromatic aberration compensation algorithm and distortion correction algorithm.

[0090] Specifically, the preset 3D dimension evaluation conditions are set according to specific application requirements, and this application does not impose any restrictions on this. It is used to determine whether the 3D dimension evaluation results meet specific standards. These conditions may include image clarity, uniformity of feature point distribution, accuracy of depth information, etc. For example, the standard deviation of the depth information must be less than a certain value, etc. The 3D dimension evaluation results are compared with the preset 3D dimension evaluation conditions. If the 3D dimension evaluation results meet the preset 3D dimension evaluation conditions, no further adjustment is required. If the 3D dimension evaluation results do not meet the preset 3D dimension evaluation conditions, the Z-axis position and / or objective lens spacing of the shooting device in the multi-channel optical system are adjusted according to the status dimension of the shooting device. For example, if the focus quality of a local area of ​​a channel is significantly lower than the benchmark (determined by the proportion of Fourier high-frequency energy), it may be necessary to adjust the Z-axis position or objective lens spacing of the shooting device to change the focal length. The preset chromatic aberration compensation algorithm and distortion correction algorithm are used to perform image registration on the calibration plate image. Among them, the chromatic aberration compensation algorithm is used to deal with the image color distortion problem caused by chromatic aberration of the multi-channel optical system and improve the color consistency of the image; the distortion correction algorithm is used to deal with the image deformation problem caused by lens distortion and restore the true shape of the image. For the focus offset of different wavelength channels, the axial chromatic aberration compensation algorithm can be used to introduce the chromatic aberration compensation factor , optimize the Z-axis adjustment amount to complete the image registration of the calibration plate image. The calculation formula is:

[0091]

[0092] in, is the Z-axis correction amount, is the chromatic aberration compensation factor, is the empirical coefficient, is the Z-axis adjustment amount of the channel to be adjusted, is the difference between the theoretical reference channel and the channel to be adjusted. For depth-related distortions (such as field curvature), a nonlinear distortion correction algorithm can be used. By using B-spline surface fitting to correct for these distortions, the control point spacing is matched to the calibration plate partition density to complete image registration of the calibration plate image. This shows that by determining the 3D dimensionality evaluation results and adjusting the Z-axis position or objective lens spacing of the camera, the quality of depth information acquisition can be improved, and the accuracy of depth information can be enhanced. This, in turn, optimizes the imaging performance of the multi-channel optical system, improves the accuracy and efficiency of image registration, and enhances the robustness of the multi-channel optical system.

[0093] In another embodiment, image registration is performed on the calibration plate image based on the 2D and / or 3D dimensional evaluation results. After adjusting to the optimal result, any discrepancies from the ideal result are re-evaluated to ensure that the registration result meets the application requirements. If the registration result does not meet the application requirements, an affine transformation matrix is ​​used to align the multi-channel images, and the image parameters are iteratively optimized by minimizing the reprojection error (Levenberg-Marquardt algorithm) until the registration result meets the application requirements. This re-evaluation of the registered image allows for timely detection and correction of errors that may occur during the registration process, thereby ensuring the accuracy and efficiency of the registration result and improving the robustness and adaptability of the multi-channel optical system.

[0094] In this embodiment, the transformation relationship between images can be determined more accurately based on the 2D dimension evaluation results and / or the 3D dimension evaluation results, thereby improving the registration accuracy of multi-channel imaging images and achieving efficient alignment of images of different channels at the same hardware level, thereby improving the accuracy and efficiency of optical detection.

[0095] In summary, the present invention provides an image registration method, device, optical system and medium to obtain a calibration plate image, wherein the calibration plate image is obtained by photographing the calibration plate at different positions based on a shooting device in a multi-channel optical system, and the calibration plate image includes a 2D calibration plate image and a 3D calibration plate image, and the position of the calibration plate is pre-set. The 2D calibration plate image is evaluated using a preset 2D dimensional image algorithm to obtain a 2D dimensional evaluation result, and the 3D calibration plate image is evaluated using a preset 3D dimensional image algorithm to obtain a 3D dimensional evaluation result, wherein the 2D dimensional evaluation result and the 3D dimensional evaluation result both include a shooting device status dimension and an image imaging status dimension, and image registration is performed on the calibration plate image based on the 2D dimensional evaluation result and / or the 3D dimensional evaluation result. It can be seen that this application evaluates the acquired 2D calibration plate image and 3D calibration plate image based on the preset 2D dimensional image algorithm and 3D dimensional image algorithm, respectively, and obtains 2D dimensional evaluation results and 3D dimensional evaluation results. Then, based on the 2D dimensional evaluation results and / or the 3D dimensional evaluation results, the calibration plate image can be accurately aligned, thereby improving the alignment accuracy of the multi-channel image.

[0096] See also Figure 6 , Figure 6 Schematic diagram of the structure of the image registration device provided by the embodiment of the present invention. The image registration device corresponds to the image registration method in the above embodiment. Figure 5 as well as Figure 5 For the convenience of explanation, only the parts related to this embodiment are shown. Figure 6The image registration device 60 includes: an acquisition module 61 , an evaluation module 62 , and a registration module 63 .

[0097] An acquisition module 61 is configured to acquire a calibration plate image, wherein the calibration plate image is obtained by photographing the calibration plate at different positions using a photographing device in a multi-channel optical system. The calibration plate image includes a 2D calibration plate image and a 3D calibration plate image, and the position of the calibration plate is pre-set.

[0098] An evaluation module 62 is configured to evaluate the 2D calibration plate image using a preset 2D dimensional image algorithm to obtain a 2D dimensional evaluation result, and to evaluate the 3D calibration plate image using a preset 3D dimensional image algorithm to obtain a 3D dimensional evaluation result, wherein both the 2D dimensional evaluation result and the 3D dimensional evaluation result include a camera status dimension and an image imaging status dimension;

[0099] The registration module 63 is configured to perform image registration on the calibration plate image according to the 2D dimension evaluation result and / or the 3D dimension evaluation result.

[0100] Optionally, the acquisition module 61 is specifically configured to:

[0101] Acquiring an initial calibration plate image obtained by photographing the calibration plate at multiple positions by the photographing device;

[0102] Determining a focus position of the camera according to the clarity of the initial calibration plate image;

[0103] Acquire a calibration plate image obtained by photographing the calibration plate by the photographing device at the focus position and at multiple defocus positions within a defocus area, wherein the defocus area is centered on the focus position, and a distance between an edge position of the defocus area and the focus position is preset.

[0104] Optionally, the evaluation module 62 is specifically configured to:

[0105] Performing image fusion on the reference channel image and the other channel images respectively to obtain a plurality of three-dimensional color image matrices, wherein each of the three-dimensional color image matrices corresponds to a respective channel image;

[0106] Analyzing and calculating each pixel in each of the three-dimensional color image matrices to obtain the balance of the RGB value of each pixel;

[0107] performing histogram statistics on each of the three-dimensional color image matrices according to the balance of the RGB values ​​of each pixel to obtain a plurality of histogram distribution information;

[0108] A 2D dimension evaluation result is determined based on the histogram distribution information.

[0109] Optionally, the evaluation module 62 is further configured to:

[0110] Calculating the difference between the maximum RGB value and the minimum RGB value of each pixel in the three-dimensional color image matrix, and determining whether the difference is less than a preset difference threshold;

[0111] If the difference is less than a preset difference threshold, determining that the RGB components of the pixel are balanced;

[0112] Calculating the R channel deviation, G channel deviation, and B channel deviation of each pixel in each of the three-dimensional color image matrices, and determining whether the R channel deviation, G channel deviation, and B channel deviation are all less than a preset deviation threshold;

[0113] If the R channel deviation, the G channel deviation, and the B channel deviation are all less than the preset deviation threshold, determining the RGB color balance of the pixel;

[0114] Normalizing the RGB value of each pixel in each of the three-dimensional color image matrices to obtain a standardized RGB value of each pixel;

[0115] Determine whether the normalized RGB value of each pixel is close to a preset RGB threshold;

[0116] If the normalized RGB value of each pixel is close to a preset RGB threshold, determining that the RGB color difference of the pixel is balanced;

[0117] The balance of the RGB value of each pixel is determined based on the RGB component balance of the pixel, the RGB color balance of the pixel, and the RGB color difference balance of the pixel.

[0118] Optionally, the evaluation module 62 is further configured to:

[0119] Processing the 3D calibration plate image using a focus measurement function to generate a plurality of depth maps;

[0120] Calculating the depth deviation of the same pixel marker point in each of the depth maps;

[0121] Determining a root mean square error (RMS) of all pixel markers in each depth map based on a depth deviation of the same pixel marker in each depth map;

[0122] A 3D dimension evaluation result is determined according to a root mean square error of all pixel marker points in each of the depth maps.

[0123] Optionally, the registration module 63 is specifically configured to:

[0124] Determine whether the 2D dimension evaluation result meets the preset 2D dimension evaluation conditions;

[0125] If the 2D dimension evaluation result does not meet the preset 2D dimension evaluation conditions, the X-axis position and / or Y-axis position of the shooting device in the multi-channel optical system is adjusted, and the calibration plate image is aligned using the preset rigid mapping algorithm and non-rigid mapping algorithm.

[0126] Optionally, the registration module 63 is further configured to:

[0127] Determining whether the 3D dimension evaluation result meets the preset 3D dimension evaluation conditions;

[0128] If the 3D dimensional evaluation result does not meet the preset 3D dimensional evaluation conditions, the Z-axis position and / or objective lens spacing of the shooting device in the multi-channel optical system are adjusted, and the calibration plate image is aligned using the preset chromatic aberration compensation algorithm and distortion correction algorithm.

[0129] It should be noted that the information interaction, execution process and other contents between the above-mentioned units are based on the same concept as the embodiment of the method of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0130] In one embodiment, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor in an electronic optical system, the electronic optical system is enabled to perform the steps of any embodiment of an image registration method disclosed herein, which will not be repeated here. The computer-readable storage medium may be non-volatile or volatile.

[0131] The processor may be a CPU, other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0132] Memory includes readable storage media, internal memory, and the like. The internal memory can be the internal memory of the electron-optical system, providing an environment for the operation of the operating system and computer-readable instructions stored in the readable storage medium. The readable storage medium can be the electron-optical system's hard drive. In other embodiments, it can also be an external storage system for the electron-optical system, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped with the electron-optical system. Furthermore, the memory can include both the internal storage unit of the electron-optical system and the external storage system. The memory is used to store the operating system, associated applications, boot loaders, data, and other programs, such as the program code of computer programs. The memory can also be used to temporarily store data that has been output or is about to be output.

[0133] It is understood by those skilled in the art that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0134] Those skilled in the art can clearly understand that for the sake of convenience and brevity in description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0135] The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, persons skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. An image registration method, characterized in that: include: Acquire a calibration plate image, wherein the calibration plate image is obtained by photographing the calibration plate at different positions using a photographing device in the multi-channel optical system, the calibration plate image includes a 2D calibration plate image and a 3D calibration plate image, and the position of the calibration plate is preset; The 2D calibration plate image is evaluated using a preset 2D dimensional image algorithm to obtain a 2D dimensional evaluation result, and the 3D calibration plate image is evaluated using a preset 3D dimensional image algorithm to obtain a 3D dimensional evaluation result, wherein both the 2D dimensional evaluation result and the 3D dimensional evaluation result include a camera status dimension and an image imaging status dimension; Performing image registration on the calibration plate image according to the 2D dimension evaluation result and / or the 3D dimension evaluation result; The 2D calibration plate image includes a reference channel image and other channel images, and the 2D calibration plate image is evaluated using a preset 2D dimension image algorithm to obtain a 2D dimension evaluation result, including: Performing image fusion on the reference channel image and the other channel images respectively to obtain a plurality of three-dimensional color image matrices, wherein each of the three-dimensional color image matrices corresponds to a respective channel image; Analyzing and calculating each pixel in each of the three-dimensional color image matrices to obtain the balance of the RGB value of each pixel; performing histogram statistics on each of the three-dimensional color image matrices according to the balance of the RGB values ​​of each pixel to obtain a plurality of histogram distribution information; A 2D dimension evaluation result is determined based on the histogram distribution information.

2. The image registration method according to claim 1, wherein: The analyzing and calculating each pixel in each of the three-dimensional color image matrices to obtain the balance of the RGB value of each pixel includes: Calculating the difference between the maximum RGB value and the minimum RGB value of each pixel in the three-dimensional color image matrix, and determining whether the difference is less than a preset difference threshold; If the difference is less than a preset difference threshold, determining that the RGB components of the pixel are balanced; Calculating the R channel deviation, G channel deviation, and B channel deviation of each pixel in each of the three-dimensional color image matrices, and determining whether the R channel deviation, G channel deviation, and B channel deviation are all less than a preset deviation threshold; If the R channel deviation, the G channel deviation, and the B channel deviation are all less than the preset deviation threshold, determining the RGB color balance of the pixel; Normalizing the RGB value of each pixel in each of the three-dimensional color image matrices to obtain a standardized RGB value of each pixel; Determine whether the normalized RGB value of each pixel is close to a preset RGB threshold; If the normalized RGB value of each pixel is close to a preset RGB threshold, determining that the RGB color difference of the pixel is balanced; The balance of the RGB value of each pixel is determined based on the RGB component balance of the pixel, the RGB color balance of the pixel, and the RGB color difference balance of the pixel.

3. The image registration method according to claim 1, wherein: The 3D calibration plate image is evaluated by using a preset 3D dimension image algorithm to obtain a 3D dimension evaluation result, including: Processing the 3D calibration plate image using a focus measurement function to generate a plurality of depth maps; Calculating the depth deviation of the same pixel marker point in each of the depth maps; Determining a root mean square error (RMS) of all pixel markers in each depth map based on a depth deviation of the same pixel marker in each depth map; A 3D dimension evaluation result is determined according to a root mean square error of all pixel marker points in each of the depth maps.

4. The image registration method according to claim 1, wherein: The performing image registration on the calibration plate image according to the 2D dimension evaluation result includes: Determine whether the 2D dimension evaluation result meets the preset 2D dimension evaluation conditions; If the 2D dimension evaluation result does not meet the preset 2D dimension evaluation conditions, the X-axis position and / or Y-axis position of the shooting device in the multi-channel optical system is adjusted, and the calibration plate image is aligned using the preset rigid mapping algorithm and non-rigid mapping algorithm.

5. The image registration method according to claim 1, wherein: The performing image registration on the calibration plate image according to the 3D dimension evaluation result includes: Determining whether the 3D dimension evaluation result meets the preset 3D dimension evaluation conditions; If the 3D dimensional evaluation result does not meet the preset 3D dimensional evaluation conditions, the Z-axis position and / or objective lens spacing of the shooting device in the multi-channel optical system are adjusted, and the calibration plate image is aligned using the preset chromatic aberration compensation algorithm and distortion correction algorithm.

6. The image registration method according to claim 1, wherein: The obtaining of the calibration plate image comprises: Acquire an initial calibration plate image obtained by photographing the calibration plate at multiple positions by the photographing device, wherein there are multiple initial calibration plate images; Determining a focus position of the camera according to the clarity of the initial calibration plate image; Acquire a calibration plate image obtained by photographing the calibration plate by the photographing device at the focus position and at multiple defocus positions within a defocus area, wherein the defocus area is centered on the focus position, and a distance between an edge position of the defocus area and the focus position is preset.

7. An image registration device, characterized in that: include: an acquisition module, configured to acquire a calibration plate image, wherein the calibration plate image is obtained by photographing the calibration plate at different positions using a photographing device in a multi-channel optical system, the calibration plate image includes a 2D calibration plate image and a 3D calibration plate image, and the position of the calibration plate is preset; an evaluation module, configured to evaluate the 2D calibration plate image using a preset 2D dimensional image algorithm to obtain a 2D dimensional evaluation result, and to evaluate the 3D calibration plate image using a preset 3D dimensional image algorithm to obtain a 3D dimensional evaluation result, wherein both the 2D dimensional evaluation result and the 3D dimensional evaluation result include a camera status dimension and an image imaging status dimension; a registration module, configured to perform image registration on the calibration plate image according to the 2D dimension evaluation result and / or the 3D dimension evaluation result; The 2D calibration plate image includes a reference channel image and other channel images, and the 2D calibration plate image is evaluated using a preset 2D dimension image algorithm to obtain a 2D dimension evaluation result, including: Performing image fusion on the reference channel image and the other channel images respectively to obtain a plurality of three-dimensional color image matrices, wherein each of the three-dimensional color image matrices corresponds to a respective channel image; Analyzing and calculating each pixel in each of the three-dimensional color image matrices to obtain the balance of the RGB value of each pixel; performing histogram statistics on each of the three-dimensional color image matrices according to the balance of the RGB values ​​of each pixel to obtain a plurality of histogram distribution information; A 2D dimension evaluation result is determined based on the histogram distribution information.

8. A multi-channel optical system, characterized in that: The multi-channel optical system includes a shooting device, a calibration plate and a control module; The control module is configured to acquire a calibration plate image, wherein the calibration plate image is obtained by photographing the calibration plate at different positions based on a photographing device in a multi-channel optical system, the calibration plate image includes a 2D calibration plate image and a 3D calibration plate image, and the position of the calibration plate is preset; the 2D calibration plate image is evaluated using a preset 2D dimensional image algorithm to obtain a 2D dimensional evaluation result, and the 3D calibration plate image is evaluated using a preset 3D dimensional image algorithm to obtain a 3D dimensional evaluation result, wherein both the 2D dimensional evaluation result and the 3D dimensional evaluation result include a photographing device status dimension and an image imaging status dimension; and image registration is performed on the calibration plate image according to the 2D dimensional evaluation result and / or the 3D dimensional evaluation result; The 2D calibration plate image includes a reference channel image and other channel images, and the 2D calibration plate image is evaluated using a preset 2D dimension image algorithm to obtain a 2D dimension evaluation result, including: Performing image fusion on the reference channel image and the other channel images respectively to obtain a plurality of three-dimensional color image matrices, wherein each of the three-dimensional color image matrices corresponds to a respective channel image; Analyzing and calculating each pixel in each of the three-dimensional color image matrices to obtain the balance of the RGB value of each pixel; performing histogram statistics on each of the three-dimensional color image matrices according to the balance of the RGB values ​​of each pixel to obtain a plurality of histogram distribution information; A 2D dimension evaluation result is determined based on the histogram distribution information.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the image registration method according to any one of claims 1 to 6 is implemented.

Images