Image correction methods, electronic systems and computer program products

Abstract

This invention provides an image correction method, electronic system, and computer program product, relating to the field of image processing technology. The method includes acquiring a first image and a second image of the same target; extracting feature point pairs from the first and second images; determining the distortion coefficient of the secondary camera and the rotation parameters of the camera coordinate system between the primary and secondary cameras based on the feature point pairs and the intrinsic parameters of the primary and secondary cameras; and correcting the second image based on the distortion coefficient, the rotation parameters, and the intrinsic parameters of the primary camera to obtain a target corrected image of the second image. Embodiments of this invention can provide better stereoscopic image correction results at a lower cost.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image correction method, electronic system, and computer program product. Background Technology

[0002] Stereo image correction refers to performing a planar projective transformation on two images so that the epipolar lines of the two images are in the same horizontal direction, while the epipolar points are mapped to infinity. This makes the two images have only horizontal parallax, thereby reducing the stereo matching problem from two dimensions to one dimension and improving the matching speed.

[0003] The bokeh effect of dual cameras is one application of stereoscopic image correction. To achieve better correction results, dual cameras typically need to be calibrated during the mobile phone manufacturing process. This requires setting up a calibration and verification station on the production line. While calibrated dual cameras offer good stereoscopic image correction, setting up a calibration station increases costs and production time. Alternatively, if camera calibration is not performed on the production line, and stereoscopic image correction is applied directly to the captured image during shooting, station costs and production time can be reduced. However, the correction effect of this method is affected by the camera's shooting quality. Since the quality of various mobile phone cameras varies, distortion during camera manufacturing can lead to poor image quality. Applying stereoscopic image correction directly to poor-quality images results in poor correction effects. In conclusion, existing stereoscopic image correction methods cannot simultaneously achieve both cost and effectiveness. There is a need for a method that offers both good correction results and low cost. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide an image correction method, electronic system and computer program product that can provide better stereoscopic image correction effect at a lower cost.

[0005] In a first aspect, embodiments of the present invention provide an image correction method applied to an uncalibrated multi-view camera, the multi-view camera including a main camera and a secondary camera; the method includes: acquiring a first image and a second image targeting the same target; wherein the first image is acquired by the main camera and the second image is acquired by the secondary camera; extracting feature point pairs from the first image and the second image; wherein the feature point pair includes a first feature point in the first image and a second feature point in the second image; the first feature point and the second feature point correspond to the same world coordinates; and correcting the second image according to the feature point pair and the intrinsic parameters of the main camera and the secondary camera to obtain a target corrected image of the second image.

[0006] In a preferred embodiment of the present invention, the step of correcting the second image based on the feature point pair and the intrinsic parameters of the main camera and the secondary camera to obtain a target corrected image of the second image includes: determining the distortion coefficient of the secondary camera and the rotation parameter of the camera coordinate system of the main camera and the secondary camera based on the feature point pair and the intrinsic parameters of the main camera and the secondary camera; wherein the rotation parameter is used to adjust the first image and the second image so that the camera coordinate systems corresponding to the adjusted first image and the second image are coaxial; and correcting the second image based on the distortion coefficient, the rotation parameter and the intrinsic parameters of the main camera to obtain a target corrected image of the second image.

[0007] In a preferred embodiment of the present invention, the step of correcting the second image according to the distortion coefficient, the rotation parameter and the intrinsic parameter of the main camera to obtain a target corrected image of the second image includes: performing distortion correction on the second image according to the distortion coefficient to obtain an initial corrected image of the second image; and performing stereo correction on the initial corrected image according to the rotation parameter and the intrinsic parameter of the main camera to obtain a target corrected image of the second image.

[0008] In a preferred embodiment of the present invention, the step of extracting feature point pairs from the first image and the second image includes: preprocessing the first image and the second image to obtain a first preprocessed image and a second preprocessed image; wherein the first preprocessed image and the second preprocessed image have the same resolution, and the camera coordinate systems corresponding to the first preprocessed image and the second preprocessed image are coaxial; dividing the first preprocessed image equally according to a preset first grid density to obtain a plurality of first image blocks; dividing the second preprocessed image equally according to a preset second grid density to obtain a plurality of second image blocks; wherein the first grid density is greater than the second grid density; and extracting feature point pairs from the first image blocks and the second image blocks.

[0009] In a preferred embodiment of the present invention, the step of extracting feature point pairs based on the first image block and the second image block includes: traversing each first image block in the first image and performing the following processing on the first image block to obtain multiple feature point pairs: determining a target image block corresponding to the first image block from the multiple second image blocks; the target image block being the image block among the multiple second image blocks that is closest to the centroid of the first image block; extracting multiple first feature points from the first image block and extracting multiple second feature points from the target image block; and combining the multiple first feature points with the feature point closest to the multiple second feature points to determine the feature point pair of the first image block.

[0010] In a preferred embodiment of the present invention, after traversing each first image block in the first image and performing the following processing on the first image block to obtain multiple feature point pairs, the method further includes: using the RANSAC algorithm of the fundamental matrix and homography matrix to remove mismatched feature point pairs from the multiple feature point pairs.

[0011] In a preferred embodiment of the present invention, the step of preprocessing the first image and the second image to obtain a first preprocessed image and a second preprocessed image includes: performing image compression processing on the first image and the second image respectively to obtain a first intermediate image and a second intermediate image with the same resolution; rotating the first intermediate image and the second intermediate image to a preset camera coordinate system to obtain the first preprocessed image and the second preprocessed image; wherein the preset camera coordinate system has the optical center of the secondary camera as the origin, the line connecting the optical center of the primary camera to the optical center of the secondary camera as the X-axis, and the straight line passing through the origin and perpendicular to the X-axis downward as the Y-axis.

[0012] In a preferred embodiment of the present invention, the step of determining the distortion coefficient of the secondary camera and the rotation parameters of the camera coordinate system of the primary camera and the secondary camera based on the feature point pair and the intrinsic parameters of the primary camera and the secondary camera includes: constructing the intrinsic parameter matrix of the primary camera and the secondary camera; setting an objective function to minimize the difference between the ordinate of the corrected point of the second feature point and the ordinate of the first feature point; wherein the corrected point of the second feature point is obtained by: correcting the second feature point according to the distortion coefficient of the secondary camera to obtain an intermediate corrected point of the second feature point; correcting the intermediate corrected point according to the intrinsic parameter matrix of the primary camera and the intrinsic parameter matrix of the secondary camera, and the rotation parameters of the camera coordinate system of the primary camera and the secondary camera to obtain a target corrected point of the second feature point; and calculating the distortion coefficient and the rotation parameters based on the feature point pair and the objective function.

[0013] In a preferred embodiment of the present invention, the step of calculating the distortion coefficient and the rotation parameter based on the feature point pair and the objective function includes: optimizing the objective function using the Levenberg-Marquardt optimization algorithm based on the feature point pair to obtain the distortion coefficient; performing distortion correction on the second feature point in the feature point pair according to the distortion coefficient to obtain the corresponding intermediate correction point; removing point pairs in the feature point pair where the difference between the ordinate of the first feature point and the corresponding intermediate correction point exceeds a preset threshold to obtain an optimized feature point pair; and calculating the rotation parameter based on the optimized feature point pair and the objective function.

[0014] In a preferred embodiment of the present invention, the step of constructing the intrinsic parameter matrix of the main camera and the secondary camera includes: constructing the intrinsic parameter matrix of the main camera and the secondary camera based on the first focal length and the first pixel size of the main camera, the second focal length and the second pixel size of the secondary camera, and the resolution of the first image and the second image.

[0015] In a preferred embodiment of the present invention, the step of minimizing the difference between the correction point of the second feature point and the ordinate of the first feature point includes: if the feature point pair includes multiple pairs, for each pair of feature points, calculating the difference between the correction point of the second feature point and the ordinate of the first feature point in the pair; and setting the objective function to minimize the sum of the differences in the ordinates corresponding to multiple pairs of feature points.

[0016] In a preferred embodiment of the present invention, the step of correcting the second feature point according to the distortion coefficient of the secondary camera to obtain the intermediate correction point of the second feature point includes: correcting the second feature point using a preset distortion correction model to obtain the intermediate correction point of the second feature point; wherein, the distortion correction model is: In the formula, (x,y) are the coordinates of the intermediate correction point of the second feature point, (x d ,y d ) represents the coordinates of the second feature point. k1 and k2 are the distortion coefficients.

[0017] Secondly, embodiments of the present invention also provide an electronic system, the electronic system comprising: a processing device and a storage device; the storage device storing a computer program, the computer program executing the above-described image correction method when run by the processing device.

[0018] Thirdly, embodiments of the present invention also provide a computer-readable storage medium storing computer-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the above-described image correction method.

[0019] Fourthly, embodiments of the present invention also provide a computer program product, including computer program instructions, which are read and executed by a processor to perform the above-described image correction method.

[0020] The embodiments of the present invention bring the following beneficial effects:

[0021] This invention provides an image correction method, electronic system, and computer program product, which acquires a first image and a second image of the same shooting target; wherein the first image is acquired by a main camera and the second image is acquired by a secondary camera; feature point pairs are extracted from the first image and the second image; wherein the feature point pair includes a first feature point in the first image and a second feature point in the second image; the first feature point and the second feature point correspond to the same world coordinates; and the second image is corrected according to the feature point pair and the intrinsic parameters of the main camera and the secondary camera to obtain a target corrected image of the second image. In this method, matching feature point pairs are extracted from the images captured by the main and secondary cameras. Based on the first image captured by the main camera, the second image captured by the secondary camera is aligned with the first image captured by the main camera. For example, the distortion coefficient of the secondary camera and the rotation parameters for transforming the camera coordinate system of the main camera to the camera coordinate system of the secondary camera can be solved based on the extracted feature point pairs. Then, distortion correction and stereo correction can be performed on the second image captured by the secondary camera based on the solved parameters. This method does not require setting up a calibration and verification step for the camera module in the production process, thus saving costs. Furthermore, since distortion correction is performed before stereo correction, the problem of poor stereo correction effect caused by the distortion of the image captured by the camera itself is alleviated, thereby improving the image correction effect.

[0022] Other features and advantages of this disclosure will be set forth in the following description, or some features and advantages may be inferred from the description or determined without doubt, or may be learned by practicing the techniques described above.

[0023] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0024] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of an image stereoscopic correction structure provided in an embodiment of the present invention;

[0026] Figure 2 A simplified model for image stereo correction provided in this embodiment of the invention;

[0027] Figure 3A schematic diagram of the structure of an electronic system provided in an embodiment of the present invention;

[0028] Figure 4 This is a flowchart illustrating an image correction method provided in an embodiment of the present invention;

[0029] Figure 5 This is a schematic diagram of a non-coaxial camera provided in an embodiment of the present invention;

[0030] Figure 6 This is a schematic diagram illustrating a camera rotated to the coaxial position according to an embodiment of the present invention;

[0031] Figure 7 This is a schematic diagram of a process for extracting feature point pairs according to an embodiment of the present invention;

[0032] Figure 8 This is a schematic flowchart illustrating the determination of distortion coefficients and rotation parameters according to an embodiment of the present invention.

[0033] Figure 9a , Figure 9b This is a schematic diagram of an image before image correction provided in an embodiment of the present invention;

[0034] Figure 10a , Figure 10b This is a schematic diagram of an image after image correction provided in an embodiment of the present invention;

[0035] Figure 11 This is a schematic diagram of the structure of an image correction device provided in an embodiment of the present invention.

[0036] Icons: 100 - Electronic system; 102 - Processing equipment; 104 - Storage device; 106 - Input device; 108 - Output device; 110 - Image acquisition equipment; 112 - Bus system; 91 - Image acquisition module; 92 - Feature point pair extraction module; 93 - Image correction module. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] In related technologies, image stereo correction can reduce stereo matching search from two dimensions to one dimension, meaning the image satisfies row alignment constraints. In practical applications, neither the camera manufacturing precision nor the module installation requirements allow for absolute row alignment; therefore, algorithms are needed to achieve row alignment between the main and secondary cameras. Specifically, for example... Figure 1 The diagram shown illustrates stereoscopic image correction, where c l and c r The optical centers of the left and right cameras are π respectively. l and π r These are images captured by the left and right cameras, respectively. w is a point in 3D space, and m is the result of perspective projection. l and m r e represents the image points in the images captured by the left and right cameras, respectively. l and e r These are the intersection points of the line connecting the optical centers of the left and right cameras and the two images, respectively; they can also be called the poles. l With e l The connection of m r With e r The line connecting these points can be called the epipolar line, corresponding to the epipolar line in the diagram. After stereoscopic image correction, π... l and π r The two image planes are transformed into π respectively. vl and π vr Two new virtual image planes, corresponding to the virtual parallel plane in the figure; at this time, the image coordinates of the 3D space point w in the virtual image of the left camera are: The image coordinates in the virtual image of the right camera are: After stereoscopic image correction, the final result is and With the same vertical coordinate, the stereoscopic correction of the image is completed.

[0039] The image correction process described above can be based on the same three-dimensional space, changing the pose of the original camera according to a certain relationship, so that the two newly obtained cameras are in the same pose at a fixed base distance. Therefore, it can be... Figure 1 The stereoscopic correction model shown is simplified as follows: Figure 2 The image shown is a simplified model for stereoscopic correction. Figure 2 The image correction process involves the original poses of the two cameras (left and right) and then performing stereo correction to ensure that the cameras are at the same horizontal level with identical poses and parallel optical axes. Currently, many algorithms exist for image correction. One such algorithm is cylindrical projection, which projects the image onto a common cylindrical surface; however, this algorithm is computationally complex. Another approach is to divide the image correction process into projective and affine transformations. However, projective transformations require nonlinear solutions and cannot guarantee stability.

[0040] In addition, in the actual application of dual cameras in mobile phones, the dual camera module can achieve high accuracy after calibration at the module factory. However, the cost is high because a calibration and verification station needs to be set up. If the camera is not calibrated, although the cost can be reduced, it may cause distortion in the image captured by the camera, thereby reducing the effect of image stereo correction.

[0041] Based on this, the present invention provides an image correction method, electronic system, and computer program product. This technology can be applied to various devices with cameras, such as security equipment, computers, mobile phones, cameras, tablet computers, and vehicle terminal equipment. This technology can be implemented using both software and hardware. To facilitate understanding of this embodiment, the image correction method disclosed in this invention will first be described in detail.

[0042] Example 1:

[0043] First, refer to Figure 3 An example electronic system 100 is described for implementing the image correction method, electronic system, and computer program product of embodiments of the present invention.

[0044] like Figure 3 The diagram illustrates the structure of an electronic system 100. The electronic system 100 includes one or more processing devices 102, one or more storage devices 104, input devices 106, and output devices 108. It may also include one or more image acquisition devices 110. These components are interconnected via a bus system 112 and / or other forms of connection mechanisms (not shown). It should be noted that... Figure 3 The components and structure of the electronic system 100 shown are merely exemplary and not limiting; the electronic system may have other components and structures as needed.

[0045] The processing device 102 may be a gateway, a smart terminal, or a device containing a central processing unit (CPU) or other forms of processing unit with data processing capabilities and / or instruction execution capabilities. It can process data from other components in the electronic system 100 and control other components in the electronic system 100 to perform desired functions.

[0046] Storage device 104 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and processing device 102 may execute the program instructions to implement the client functions (implemented by the processing device) in the embodiments of the present invention described below, and / or other desired functions. Various applications and various data, such as various data used and / or generated by the applications, may also be stored in the computer-readable storage medium.

[0047] The input device 106 may be a device used by a user to input commands, and may include one or more of a keyboard, mouse, microphone, and touch screen.

[0048] The output device 108 can output various information (e.g., images or sounds) to the outside (e.g., a user) and may include one or more of a display, a speaker, etc.

[0049] The image acquisition device 110 can acquire preview video frames or image data (such as images to be corrected or recognized), and store the acquired preview video frames or image data in the storage device 104 for use by other components.

[0050] Exemplary examples show that the components in an example electronic system for implementing the image correction method, apparatus, and electronic system according to embodiments of the present invention can be integrated or distributed. For instance, the processing device 102, storage device 104, input device 106, and output device 108 can be integrated into one unit, while the image acquisition device 110 is positioned at a designated location where images can be acquired. When the components in the above-described electronic system are integrated, the electronic system can be implemented as a smart terminal such as a camera, smartphone, tablet computer, computer, vehicle terminal, or camcorder.

[0051] Example 2:

[0052] This embodiment provides an image stereo correction method, which is applied to an uncalibrated multi-view camera, including a main camera and a secondary camera; for example... Figure 4 The diagram shown is a flowchart of the method. Figure 4 As can be seen, the method includes the following steps:

[0053] Step S402: Acquire a first image and a second image of the same shooting target; wherein the first image is acquired by the main camera and the second image is acquired by the secondary camera.

[0054] Here, the first and second images mentioned above can be original images taken by the main camera and the secondary camera targeting the same subject. That is, the content of the first and second images can be the same, and they contain the same subject, which can be a person, landscape, object, etc. Typically, the field of view of the main camera is less than or equal to that of the secondary camera, which also means that the first and second images are not in the same horizontal or vertical direction.

[0055] Step S404: Extract feature point pairs from the first image and the second image; wherein the feature point pair includes a first feature point in the first image and a second feature point in the second image; the first feature point and the second feature point correspond to the same world coordinates.

[0056] The first and second feature points mentioned above can be representative parts of the image. The information of these feature points can include their position coordinates, size, and orientation. Since the camera can be placed anywhere in the environment, a reference coordinate system can be chosen to describe the camera's position and the position of any object in the environment. This reference coordinate system can be called the world coordinate system. Here, the relationship between the camera coordinate system and the world coordinate system can be described using a rotation matrix and a translation vector.

[0057] Specifically, feature point extraction methods, such as SIFT (Scale-Invariant Features Transform) and SURF (Speeded Up Robust Features), can be used to extract the first feature point of the first image and the second feature point of the second image. Furthermore, feature point matching methods, such as FLANN (Fast Library for Approximate Nearest Neighbors), SURF, and ORB (Oriented FAST and Rotated BRIEF), can be used to obtain matching feature point pairs between the first and second images, where each first feature point in the first image corresponds to a second feature point in the second image.

[0058] Furthermore, for each feature point pair, the first and second feature points in the pair correspond to the same world coordinates; that is, the first and second feature points represent features of objects at the same coordinates in the world coordinate system. In practice, after extracting the feature point pairs, reliable feature point pairs can be selected from the extracted pairs using data filtering methods.

[0059] Step S406: Based on the feature point pair and the intrinsic parameters of the main camera and the secondary camera, the second image is corrected to obtain the target corrected image of the second image.

[0060] In one possible implementation, the target corrected image of the second image can be obtained through the following steps 11-12:

[0061] (11) Based on the feature point pair and the intrinsic parameters of the main camera and the secondary camera, determine the distortion coefficient of the secondary camera and the rotation parameters of the camera coordinate system of the main camera and the secondary camera; wherein, the rotation parameters are used to adjust the first image and the second image so that the camera coordinate systems corresponding to the adjusted first image and the second image are coaxial.

[0062] (12) The second image is corrected according to the distortion coefficient, the rotation parameter and the intrinsic parameter of the main camera to obtain the target corrected image of the second image.

[0063] The intrinsic parameters of the main camera and the secondary camera are determined by the camera itself. For example, they can be an intrinsic parameter matrix, which can be calculated based on the camera's focal length, pixel size, and image resolution.

[0064] like Figure 5 The diagram shows a non-coaxial camera. The camera on the left represents the main camera, and the camera on the right represents the secondary camera. C L C represents the optical center of the main camera. R π represents the optical center of the secondary camera. L π represents the image captured by the main camera. R This represents the image captured by the secondary camera. For example... Figure 6 The diagram illustrates a camera rotation to coaxial alignment. A new coordinate system is established with the optical center of the secondary camera as the origin, the line connecting the optical centers of the primary and secondary cameras as the new X-axis (positive direction to the right), and the line passing through the origin and perpendicular to the X-axis as the Y-axis (positive direction downwards). In this coordinate system, the primary and secondary cameras are coaxial. Figure 5 The camera coordinate systems of the main camera and the secondary camera, which were originally non-coaxial, are rotated according to the rotation parameters. Figure 6 The new coordinate system shown below has a coaxial relationship.

[0065] The rotation parameter can be a 3×3 matrix. Based on the rotation matrix and the pixel coordinates of the first and second images, the first and second images are adjusted using a preset operation method, such as rotation or translation, so that the main camera corresponding to the first image and the secondary camera corresponding to the second image are set coaxially in the coordinate system after the adjustment. For example, they can be in the same horizontal direction or in the same vertical direction.

[0066] In practice, the first image captured by the main camera can be used as a basis, and the second image captured by the secondary camera can be approximated to the first image. That is, for the extracted feature point pairs, the pixel position coordinates corresponding to the first feature point can be used as the "target". The pixel position coordinates of the second feature point are obtained after distortion correction based on distortion coefficients and stereo correction based on camera intrinsic parameters and rotation parameters. Theoretically, these coordinates should approach the pixel position coordinates of the first feature point. In this way, based on the extracted feature point pairs and the intrinsic parameters of the main and secondary cameras, the aforementioned distortion coefficients and rotation parameters can be optimized and obtained.

[0067] When the second image is corrected based on the distortion coefficient, the rotation parameter, and the intrinsic parameters of the main camera, in one possible implementation, the second image can first be distorted based on the distortion coefficient to obtain an initial corrected image of the second image; then, the initial corrected image can be stereoscopically corrected based on the rotation parameter and the intrinsic parameters of the main camera to obtain a target corrected image of the second image.

[0068] Compared to calibrated image stereo correction methods, the image correction method provided in this embodiment does not require a calibration and verification step for the camera module in the production process, thus saving costs. Compared to methods that directly perform stereo correction on images captured by the camera module, the image correction method in this embodiment also performs distortion correction on the image before stereo correction, thereby alleviating the problem of poor stereo correction effect caused by distortion in the images captured by the camera itself, and thus improving the image correction effect.

[0069] The image correction method provided in this invention acquires a first image and a second image of the same target. The first image is acquired by a main camera, and the second image is acquired by a secondary camera. Feature point pairs are extracted from the first and second images. Each feature point pair includes a first feature point in the first image and a second feature point in the second image. The first and second feature points correspond to the same world coordinates. Based on the feature point pairs and the intrinsic parameters of the main and secondary cameras, the distortion coefficient of the secondary camera and the rotation parameters of the camera coordinate systems of the main and secondary cameras are determined. These rotation parameters are used to adjust the first and second images so that the camera coordinate systems of the adjusted first and second images are coaxial. The second image is corrected based on the distortion coefficient, the rotation parameters, and the intrinsic parameters of the main camera to obtain a target corrected image of the second image. This image correction method can be applied to devices equipped with calibrated or uncalibrated cameras, which not only saves equipment production costs but also improves the image correction effect.

[0070] Example 3:

[0071] exist Figure 4 Based on the image correction method shown, this embodiment focuses on describing the specific implementation process of step S404 (extracting feature point pairs from the first image and the second image) in the above embodiment. For example... Figure 7 The diagram shown illustrates a process for extracting feature point pairs, which includes the following steps:

[0072] Step S702: Preprocess the first image and the second image to obtain a first preprocessed image and a second preprocessed image; wherein the first preprocessed image and the second preprocessed image have the same resolution, and the camera coordinate systems corresponding to the first preprocessed image and the second preprocessed image are coaxial.

[0073] In this embodiment, image preprocessing is performed through the following steps 21-22:

[0074] (21) Perform image compression processing on the first image and the second image respectively to obtain a first intermediate image and a second intermediate image with the same resolution.

[0075] Assuming the resolution of the first image is 1500×1200 and the resolution of the second image is 1200×800, the first and second images can be adjusted to the same resolution through image compression, for example, both can be compressed to a resolution of 1000×750.

[0076] In practice, if the difference between the field of view (FOV) of the second image and that of the first image exceeds 8°, the second image needs to be cropped before image compression to ensure the difference is within 8°. For example, if the field of view of the first image is 75° and that of the second image is 110°, the second image's field of view is much larger than that of the first image. In this case, the second image needs to be cropped to approximately 83° from the center.

[0077] (22) Rotate the first intermediate image and the second intermediate image to a preset camera coordinate system to obtain a first preprocessed image and a second preprocessed image; wherein, the preset camera coordinate system takes the optical center of the secondary camera as the origin, the line connecting the optical center of the main camera to the optical center of the secondary camera as the X-axis, and the straight line passing through the origin and perpendicular to the X-axis downward as the Y-axis.

[0078] Figure 6 The coordinate system X'OY' is an example of the aforementioned preset camera coordinate system. In the figure, the optical center of the secondary camera is the origin, the line connecting the optical center of the primary camera to the optical center of the secondary camera is the X-axis, the line passing through the origin and perpendicular to the X-axis downwards is the Y-axis, and the line passing through the origin and perpendicularly forwards is the Z-axis. Thus, after the above image cropping, image compression, and image rotation processes, the coordinate systems corresponding to the first and second preprocessed images are coaxial, which can be coaxial in the horizontal direction or coaxial in the vertical direction.

[0079] Step S704: Divide the first preprocessed image equally according to a preset first grid density to obtain multiple first image blocks; divide the second preprocessed image equally according to a preset second grid density to obtain multiple second image blocks; wherein the first grid density is greater than the second grid density.

[0080] For example, if the first grid density is 20×20 and the second grid density is 16×16, then the first grid density is greater than the second grid density, and the first image block obtained after dividing the image into equal parts will be smaller than the second image block.

[0081] Step S706: Extract feature point pairs based on the first image block and the second image block.

[0082] In at least one possible implementation, each first image block in the first image can be traversed, and the first image block can be processed according to steps 31-33 below to obtain multiple feature point pairs:

[0083] (31) Determine the target image block corresponding to the first image block from the plurality of second image blocks; the target image block is the image block among the plurality of second image blocks that is closest to the centroid of the first image block.

[0084] The coordinates of the centroid of the first image block are determined based on the coordinates of the four corner points of the first image block. From the multiple second image blocks that are divided equally, the image block that is closest to the coordinates of the centroid is found and determined as the second image block that matches the first image block, which is the target image block.

[0085] (32) Extract multiple first feature points from the first image block and extract multiple second feature points from the target image block.

[0086] For example, multiple first feature points can be extracted from the first image patch and multiple second feature points can be extracted from the target image patch using the ORB feature extraction algorithm.

[0087] (33) The feature point pair of the first feature point and the second feature point is determined as the feature point pair of the first image block.

[0088] A feature point combination includes a first feature point and a second feature point. By calculating the distance between the feature points, the feature point combination with the smallest distance is determined as the feature point pair for the first image block. In this way, through the above steps, each first image block corresponds to a unique feature point pair.

[0089] In practice, when extracting matching feature point pairs using algorithms such as SIFT and ORB, mismatches can easily occur if there are significant differences in the viewpoints between the first and second images, or if the second image contains regions not present in the first image. To eliminate the impact of these mismatched feature point pairs on subsequent correction, after extracting the aforementioned multiple feature point pairs, the Random Sample Consensus (RANSAC) algorithm for the fundamental matrix and homography matrix can be used to remove mismatched feature point pairs. RANSAC is a simple and effective method for removing noise. Unlike ordinary denoising algorithms, RANSAC uses as few points as possible to estimate model parameters and then maximizes the influence range of the obtained model parameters.

[0090] The image correction method provided in this embodiment preprocesses the first image captured by the main camera and the second image captured by the secondary camera, and then divides them into equal parts. The feature point pairs are extracted in units of gridded image blocks, which makes the extracted feature point pairs evenly distributed. Therefore, the distortion coefficients and rotation parameters calculated based on the extracted feature point pairs are more accurate, further alleviating the problem of overcorrection.

[0091] Example 4:

[0092] exist Figure 4Based on the image correction method shown, this embodiment focuses on describing the specific implementation process of step S406 (determining the distortion coefficient of the secondary camera and the rotation parameters of the camera coordinate system of the primary and secondary cameras based on the feature point pair and the intrinsic parameters of the primary and secondary cameras) in the above embodiment. Figure 8 As shown, this is a flowchart illustrating the determination of distortion coefficients and rotation parameters, wherein the process includes the following steps:

[0093] Step S802: Construct the intrinsic parameter matrices of the main camera and the secondary camera.

[0094] In one approach, the intrinsic parameter matrices of the main camera and the secondary camera can be constructed based on the first focal length and first pixel size of the main camera, the second focal length and second pixel size of the secondary camera, and the resolutions of the first and second images. Here, the pixel size refers to the value of the camera's charge-coupled device (CCD).

[0095] For example, the intrinsic parameter matrix of the main camera and the intrinsic parameter matrix of the secondary camera can be represented as:

[0096]

[0097]

[0098] In the above formula, fl represents the ratio of the first focal length of the main camera to the first pixel size, fr represents the ratio of the second focal length of the secondary camera to the second pixel size; (w, h) represents the resolution of the first and second images after image compression; K L K represents the intrinsic parameter matrix of the main camera. R This represents the intrinsic parameter matrix of the secondary camera.

[0099] Step S804: Set an objective function to minimize the difference between the corrected point of the second feature point and the ordinate of the first feature point; wherein, the corrected point of the second feature point is obtained by: correcting the second feature point according to the distortion coefficient of the secondary camera to obtain the intermediate corrected point of the second feature point; correcting the intermediate corrected point according to the intrinsic parameter matrix of the main camera and the intrinsic parameter matrix of the secondary camera, and the rotation parameters of the camera coordinate system of the main camera and the secondary camera to obtain the target corrected point of the second feature point.

[0100] If the feature point pair includes multiple pairs, then for each feature point pair, calculate the difference between the correction point of the second feature point and the ordinate of the first feature point; and set the objective function to minimize the sum of the ordinate differences corresponding to multiple feature points.

[0101] The second feature point can be corrected using a preset distortion correction model to obtain the intermediate correction point of the second feature point; in this embodiment, the distortion correction model is:

[0102]

[0103] In the formula, (x,y) are the coordinates of the intermediate correction point of the second feature point, (x d ,y d ) represents the coordinates of the second feature point. k1 and k2 are the distortion coefficients.

[0104] In one implementation, the objective function described above can be:

[0105]

[0106] In the formula, error() is the difference between the corrected point of the second feature point and the ordinate of the first feature point; f l u is the first focal length of the main camera. l The horizontal resolution of the main camera is half that of the main camera, v l Let (θ1, θ2, θ3) be half the vertical resolution of the main camera, (θ1, θ2, θ3) be the rotation angles of the coordinate axes of the camera coordinate system from the main camera to the secondary camera, and k1 and k2 be the distortion coefficients. K is the ordinate of the first feature point in the i-th feature point pair. l K is the intrinsic parameter of the main camera. r These are the intrinsic parameters of the secondary camera. R is the inverse of the intrinsic parameter matrix of the secondary camera. lr The rotation matrix from the main camera to the secondary camera. U is the inverse of the rotation matrix from the main camera to the secondary camera. r Let d[(K) be the intrinsic parameter of the secondary camera. r (k1, k2)U r The symbol indicates that distortion correction is performed on the second feature point. The ordinate represents the ordinate of the correction point of the second feature point in the i-th feature point pair, and n is the number of feature point pairs.

[0107] In the above objective function, after the second image is processed by the distortion correction model, it undergoes a second correction process using the image stereo correction model. The formula for aligning the second image to the first image is:

[0108]

[0109] In the formula, LU rThis represents the point after distortion correction and stereo correction of the second feature point, dU. r This represents the point after distortion correction for the second feature point. This indicates that stereoscopic image correction processing is being performed.

[0110] Step S806: Based on the feature point pair and the objective function, calculate the distortion coefficient and the rotation parameter.

[0111] In one embodiment, the distortion coefficient and the rotation parameter can be calculated through the following steps 41-44:

[0112] (41) Based on the feature point pair, the objective function is optimized by the Levenberg-Marquardt optimization algorithm to obtain the distortion coefficient.

[0113] (42) Based on the distortion coefficient, the second feature point in the feature point pair is distorted to obtain the corresponding intermediate correction point.

[0114] (43) Remove the feature point pairs in the feature point pair where the difference between the ordinate of the first feature point and the corresponding intermediate correction point exceeds the preset threshold, and obtain the optimized feature point pairs.

[0115] Here, a new feature point pair is formed by the intermediate correction point of the second feature point and the corresponding first feature point. The difference in ordinate between the first feature point and the corresponding intermediate correction point is calculated based on the objective function. If the difference in ordinate is greater than a preset threshold, for example, if the difference in ordinate is greater than 1 pixel, then the feature point pair is considered to have an error that does not meet the requirements and is discarded. In this way, through this optimization process, an optimized feature point pair is obtained.

[0116] (44) Calculate the rotation parameter based on the optimized feature point pair and the objective function.

[0117] Based on the optimized feature point pairs described above, and combined with the objective function, the rotation parameters are calculated. In this embodiment, the rotation matrix R from the main camera to the secondary camera is obtained. lr .

[0118] Thus, based on the distortion coefficients and rotation parameters obtained from the above steps, the second image can be first subjected to distortion correction processing to obtain the initial corrected image of the second image; then, based on the rotation parameters and the intrinsic parameters of the main camera, the initial corrected image can be stereoscopically corrected to obtain the target corrected image of the second image.

[0119] To verify the effectiveness of this image correction method, please refer to the before-and-after image diagrams shown in Figures 9 and 10. Figure 9a and Figure 10a Corresponding to the first image and the first corrected image, Figure 9b and Figure 10b This corresponds to the second image and the second corrected image, ultimately achieving row alignment between the first corrected image and the second corrected image, with zero horizontal parallax.

[0120] The image correction method provided in this embodiment, by setting an objective function, aligns the second image captured by the secondary camera with the first image captured by the primary camera. With the first and second images aligned, the method minimizes the error between the ordinate values ​​of the pixels in the second image after distortion correction and stereo correction and the corresponding pixels in the first image, thereby optimizing the distortion coefficients and rotation parameters in the objective function. This image correction method combines a distortion correction model and a stereo correction model, performing distortion correction on the second image captured by the secondary camera first, followed by stereo correction, which improves the image correction effect.

[0121] Corresponding to Figure 4 The image correction method shown in the present invention also provides an image correction device, such as... Figure 11 The diagram shown is a structural schematic of an image correction device, consisting of... Figure 11 As can be seen, the device includes an image acquisition module 91, a feature point pair extraction module 92, and an image correction module 93 connected in sequence. The functions of each module are as follows:

[0122] The image acquisition module 91 is used to acquire a first image and a second image of the same shooting target; wherein the first image is acquired by the main camera and the second image is acquired by the secondary camera;

[0123] The feature point pair extraction module 92 is used to extract feature point pairs from the first image and the second image; wherein, the feature point pair includes a first feature point in the first image and a second feature point in the second image; the first feature point and the second feature point correspond to the same world coordinates;

[0124] The image correction module 93 is used to correct the second image based on the feature point pair and the intrinsic parameters of the main camera and the secondary camera to obtain the target corrected image of the second image.

[0125] The image correction device provided in this embodiment of the invention acquires a first image and a second image targeting the same target; wherein the first image is acquired by a main camera and the second image is acquired by a secondary camera; feature point pairs are extracted from the first image and the second image; wherein the feature point pair includes a first feature point in the first image and a second feature point in the second image; the first feature point and the second feature point correspond to the same world coordinates; and the second image is corrected according to the feature point pair and the intrinsic parameters of the main camera and the secondary camera to obtain a target corrected image of the second image. In this device, matching feature point pairs are extracted from images captured by the main and secondary cameras. Based on the first image captured by the main camera, the second image captured by the secondary camera is aligned with the first image captured by the main camera. The distortion coefficient of the secondary camera and the rotation parameters for transforming the camera coordinate system of the main camera to the camera coordinate system of the secondary camera are calculated based on the extracted feature point pairs. Then, distortion correction and stereo correction can be performed on the second image captured by the secondary camera based on the solved parameters. This method does not require the calibration and verification of the camera module in the production process, thus saving costs. Furthermore, since distortion correction is performed before stereo correction, the problem of poor stereo correction effect caused by the distortion of the image captured by the camera itself is alleviated, thereby improving the image correction effect.

[0126] In one possible implementation, the image correction module 93 is further configured to: determine the distortion coefficient of the secondary camera and the rotation parameters of the camera coordinate systems of the primary camera and the secondary camera based on the feature point pair and the intrinsic parameters of the primary camera and the secondary camera; wherein the rotation parameters are used to adjust the first image and the second image so that the camera coordinate systems corresponding to the adjusted first image and the second image are coaxial; and correct the second image based on the distortion coefficient, the rotation parameters and the intrinsic parameters of the primary camera to obtain a target corrected image of the second image.

[0127] In one possible implementation, the image correction module 93 is further configured to: perform distortion correction on the second image according to the distortion coefficient to obtain an initial corrected image of the second image; and perform stereo correction on the initial corrected image according to the rotation parameter and the intrinsic parameter of the main camera to obtain a target corrected image of the second image.

[0128] In one possible implementation, the feature point extraction module 92 is further configured to: preprocess the first image and the second image to obtain a first preprocessed image and a second preprocessed image; wherein the first preprocessed image and the second preprocessed image have the same resolution, and the camera coordinate systems corresponding to the first preprocessed image and the second preprocessed image are coaxial; divide the first preprocessed image equally according to a preset first grid density to obtain a plurality of first image blocks; divide the second preprocessed image equally according to a preset second grid density to obtain a plurality of second image blocks; wherein the first grid density is greater than the second grid density; and extract feature point pairs based on the first image blocks and the second image blocks.

[0129] In one possible implementation, the feature point extraction module 92 is further configured to: traverse each first image block in the first image, and perform the following processing on the first image block to obtain multiple feature point pairs: determine a target image block corresponding to the first image block from the multiple second image blocks; the target image block is the image block among the multiple second image blocks that is closest to the centroid of the first image block; extract multiple first feature points from the first image block, and extract multiple second feature points from the target image block; and combine the multiple first feature points with the feature point closest to the multiple second feature points to determine the feature point pair of the first image block.

[0130] In one possible implementation, the feature point extraction module 92 is further configured to: eliminate mismatched feature point pairs from the plurality of feature point pairs using the RANSAC algorithm of the fundamental matrix and homography matrix.

[0131] In one possible implementation, the feature point extraction module 92 is further configured to: perform image compression processing on the first image and the second image respectively to obtain a first intermediate image and a second intermediate image with the same resolution; rotate the first intermediate image and the second intermediate image to a preset camera coordinate system to obtain a first preprocessed image and a second preprocessed image; wherein the preset camera coordinate system has the optical center of the secondary camera as the origin, the line connecting the optical center of the main camera to the optical center of the secondary camera as the X-axis, and the straight line passing through the origin and perpendicular to the X-axis downward as the Y-axis.

[0132] In one possible implementation, the image correction module 93 is further configured to: construct the intrinsic parameter matrix of the main camera and the secondary camera; set an objective function to minimize the difference between the corrected point of the second feature point and the ordinate of the first feature point; wherein the corrected point of the second feature point is obtained by: correcting the second feature point according to the distortion coefficient of the secondary camera to obtain an intermediate corrected point of the second feature point; correcting the intermediate corrected point according to the intrinsic parameter matrix of the main camera and the intrinsic parameter matrix of the secondary camera, and the rotation parameters of the camera coordinate system of the main camera and the secondary camera to obtain a target corrected point of the second feature point; and calculating the distortion coefficient and the rotation parameter based on the feature point pair and the objective function.

[0133] In one possible implementation, the image correction module 93 is further configured to: optimize the objective function based on the feature point pair using the Levenberg-Marquardt optimization algorithm to obtain the distortion coefficient; perform distortion correction on the second feature point in the feature point pair according to the distortion coefficient to obtain the corresponding intermediate correction point; remove point pairs in the feature point pair where the difference between the ordinates of the first feature point and the corresponding intermediate correction point exceeds a preset threshold to obtain an optimized feature point pair; and calculate the rotation parameter based on the optimized feature point pair and the objective function.

[0134] In one possible implementation, the image correction module 93 is further configured to: construct the intrinsic parameter matrix of the main camera and the secondary camera based on the first focal length and first pixel size of the main camera, the second focal length and second pixel size of the secondary camera, and the resolution of the first image and the second image.

[0135] In one possible implementation, the image correction module 93 is further configured to: if the feature point pair includes multiple pairs, calculate the difference in ordinate between the correction point of the second feature point and the first feature point for each pair of feature points; and set the objective function to minimize the sum of the differences in ordinates corresponding to multiple pairs of feature points.

[0136] In one possible implementation, the image correction module 93 is further configured to: correct the second feature point using a preset distortion correction model to obtain an intermediate correction point of the second feature point; wherein the distortion correction model is: In the formula, (x,y) are the coordinates of the intermediate correction point of the second feature point, (x d ,y d ) represents the coordinates of the second feature point. k1 and k2 are the distortion coefficients.

[0137] The image correction device provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned image correction method embodiment. For the sake of brevity, any parts not mentioned in the embodiment of the image correction device can be referred to the corresponding content in the aforementioned image correction method embodiment.

[0138] This invention provides an electronic system comprising: an image acquisition device, a processing device, and a storage device; the image acquisition device is used to acquire preview video frames or image data; the storage device stores a computer program, which executes the image correction method described above, or the steps of the image correction method described above, when the processing device is running.

[0139] This invention also provides a computer program product, including computer program instructions, which are read and executed by a processor to perform the above-described image correction method.

[0140] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the electronic system described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0141] This invention also provides a machine-readable storage medium storing machine-executable instructions. When these machine-executable instructions are called and executed by a processor, they cause the processor to implement the above-described image correction method. For specific implementation details, please refer to the foregoing method embodiments, which will not be repeated here.

[0142] The image correction method, electronic system, and computer program product provided in the embodiments of the present invention include a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the image correction method described in the preceding method embodiments. For specific implementation, please refer to the method embodiments, which will not be repeated here.

[0143] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0144] Furthermore, in the description of the embodiments of the present invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the present invention based on the specific circumstances.

[0145] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0146] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An image correction method, characterized in that, The method is applied to an uncalibrated multi-view camera, wherein the multi-view camera includes a main camera and a secondary camera; the method includes: Acquire a first image and a second image of the same target; wherein the first image is acquired by the main camera and the second image is acquired by the secondary camera; Feature point pairs are extracted from the first image and the second image; wherein, the feature point pair includes a first feature point in the first image and a second feature point in the second image; the first feature point and the second feature point correspond to the same world coordinates; Based on the feature point pairs and the intrinsic parameters of the main camera and the secondary camera, the second image is corrected to obtain the target corrected image of the second image; The step of correcting the second image based on the feature point pairs and the intrinsic parameters of the main camera and the secondary camera to obtain the target corrected image of the second image includes: Based on the feature point pairs and the intrinsic parameters of the main camera and the secondary camera, the distortion coefficient of the secondary camera and the rotation parameters of the camera coordinate systems of the main camera and the secondary camera are determined; wherein, the rotation parameters are used to adjust the first image and the second image so that the camera coordinate systems corresponding to the adjusted first image and the second image are coaxial; The second image is corrected based on the distortion coefficient, the rotation parameter, and the intrinsic parameters of the main camera to obtain the target corrected image of the second image.

2. The method according to claim 1, characterized in that, The step of correcting the second image based on the distortion coefficient, the rotation parameter, and the intrinsic parameters of the main camera to obtain the target corrected image of the second image includes: The distortion is corrected based on the distortion coefficients to obtain the initial corrected image of the second image; The initial corrected image is stereoscopically corrected based on the rotation parameters and the intrinsic parameters of the main camera to obtain the target corrected image of the second image.

3. The method according to claim 1, characterized in that, The step of extracting feature point pairs from the first image and the second image includes: The first image and the second image are preprocessed to obtain a first preprocessed image and a second preprocessed image respectively; wherein the first preprocessed image and the second preprocessed image have the same resolution, and the camera coordinate systems corresponding to the first preprocessed image and the second preprocessed image are coaxial. The first preprocessed image is divided into multiple first image blocks according to a preset first grid density; the second preprocessed image is divided into multiple second image blocks according to a preset second grid density; wherein the first grid density is greater than the second grid density; Based on the first image block and the second image block, feature point pairs are extracted.

4. The method according to claim 3, characterized in that, The step of extracting feature point pairs based on the first image patch and the second image patch includes: Traverse each first image block in the first image and perform the following processing on the first image block to obtain multiple feature point pairs: A target image block corresponding to the first image block is determined from the plurality of second image blocks; the target image block is the image block among the plurality of second image blocks that is closest to the centroid of the first image block; Multiple first feature points are extracted from the first image block, and multiple second feature points are extracted from the target image block; The feature point pair of the first image block is determined by combining the nearest feature point among the plurality of first feature points and the plurality of second feature points.

5. The method according to claim 4, characterized in that, After traversing each first image block in the first image and performing the following processing on that first image block to obtain multiple feature point pairs, the method further includes: The RANSAC algorithm, based on the fundamental matrix and homography matrix, is used to eliminate mismatched feature point pairs from the multiple feature point pairs.

6. The method according to claim 3, characterized in that, The steps of preprocessing the first image and the second image to obtain a first preprocessed image and a second preprocessed image include: Image compression processing is performed on the first image and the second image respectively to obtain a first intermediate image and a second intermediate image with the same resolution. The first intermediate image and the second intermediate image are rotated to a preset camera coordinate system to obtain a first preprocessed image and a second preprocessed image; wherein, the preset camera coordinate system has the optical center of the secondary camera as the origin, the line connecting the optical center of the primary camera to the optical center of the secondary camera as the X-axis, and the straight line passing through the origin and perpendicular to the X-axis downward as the Y-axis.

7. The method according to any one of claims 1-6, characterized in that, The steps of determining the distortion coefficient of the secondary camera and the rotation parameters of the camera coordinate systems of the primary camera and the secondary camera based on the feature point pairs and the intrinsic parameters of the primary camera and the secondary camera include: Construct the intrinsic parameter matrices of the main camera and the secondary camera; A target function is set to minimize the difference between the corrected point of the second feature point and the ordinate of the first feature point; wherein, the corrected point of the second feature point is obtained by: correcting the second feature point according to the distortion coefficient of the secondary camera to obtain the intermediate corrected point of the second feature point; and correcting the intermediate corrected point according to the intrinsic parameter matrix of the main camera and the intrinsic parameter matrix of the secondary camera, and the rotation parameters of the camera coordinate system of the main camera and the secondary camera to obtain the target corrected point of the second feature point. Based on the feature point pairs and the objective function, the distortion coefficients and the rotation parameters are calculated.

8. The method according to claim 7, characterized in that, The steps of calculating the distortion coefficients and rotation parameters based on the feature point pairs and the objective function include: Based on the feature point pairs, the objective function is optimized using the Levenberg-Marquardt optimization algorithm to obtain the distortion coefficients; Based on the distortion coefficient, the second feature point in the feature point pair is distorted to obtain the corresponding intermediate correction point; Remove the feature point pairs from the list where the difference between the ordinate of the first feature point and the corresponding intermediate correction point exceeds a preset threshold to obtain optimized feature point pairs. The rotation parameters are calculated based on the optimized feature point pairs and the objective function.

9. The method according to claim 7, characterized in that, The steps for constructing the intrinsic parameter matrices of the main camera and the secondary camera include: Based on the first focal length and first pixel size of the main camera, the second focal length and second pixel size of the secondary camera, and the resolutions of the first and second images, the intrinsic parameter matrices of the main camera and the secondary camera are constructed.

10. The method according to claim 7, characterized in that, The step of setting an objective function to minimize the difference between the corrected point of the second feature point and the ordinate of the first feature point includes: If the feature point pair includes multiple pairs, for each feature point pair, calculate the difference in ordinate between the correction point of the second feature point and the first feature point in that feature point pair; The objective function is set to minimize the sum of the differences in the ordinates of multiple pairs of feature points.

11. The method according to claim 7, characterized in that, The step of correcting the second feature point based on the distortion coefficient of the secondary camera to obtain the intermediate correction point of the second feature point includes: The second feature point is corrected using a preset distortion correction model to obtain the intermediate correction point of the second feature point; wherein, the distortion correction model is: In the formula, ( , ) represents the coordinates of the intermediate correction point of the second feature point. , () represents the coordinates of the second feature point. , and The distortion coefficient is denoted as .

12. An electronic system, characterized in that, The electronic system includes: a processing device and a storage device; The storage device stores a computer program that, when executed by the processing device, performs the image correction method as described in any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the image correction method according to any one of claims 1 to 11.

14. A computer program product, characterized in that, It includes computer program instructions, which, when read and executed by a processor, perform the method as described in any one of claims 1 to 11.

Images