A high-precision image stitching and correction method based on multi-camera

Abstract

The present invention relates to the field of image processing technology, and provides a high-precision image stitching and correction method based on a multi-camera, which identifies potential distortion areas in an image according to multiple viewing angle images, and outputs the distortion areas; extracts detailed features of the distortion area, and combines the geometric consistency of adjacent areas to deduce the distortion type of each distortion area, and generates local correction parameters; uses adaptive nonlinear transformation, combines the mapping relationship between pixel position and distortion degree, accurately corrects each pixel, and obtains a distortion-corrected image; after the distortion-corrected images are stitched, the edge transition effect is optimized in combination with multi-scale region fusion technology. The present invention can eliminate visual unnaturalness at the seams, and improves the accuracy and visual experience of overall image stitching.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a high-precision image stitching and correction method based on multi-cameras. Background Art

[0002] With the continuous development of image processing technology and computer vision technology, image stitching and correction have become important research topics in many fields, especially in the application of multi-camera systems. Multi-camera systems collect images from different perspectives through multiple cameras, providing a rich source of data for applications such as high-precision image reconstruction, three-dimensional reconstruction and visual navigation. With the maturity of deep learning technology, the accuracy and automation of image stitching and correction have been significantly improved. Traditional stitching algorithms mainly rely on feature matching and geometric transformation, but these methods often have problems of inaccurate stitching or unnatural distortion in the case of image distortion, illumination changes, and mismatch between images from different perspectives. In recent years, the application of deep learning, especially convolutional neural networks (CNNs) in image processing, has promoted the progress of image stitching and correction methods, and the accuracy of image feature extraction, distortion detection and correction using deep learning models has been significantly improved. However, although the existing technology has solved these problems to a certain extent, in high-precision image stitching, especially when processing images from multi-camera systems, there are still technical bottlenecks such as inaccurate distortion correction and unnatural transition of stitching edges.

[0003] Although the existing image stitching and correction technology has made some progress, it still has many limitations in practical applications. For example, the images in the multi-camera system are often severely distorted after stitching due to factors such as viewing angle differences, perspective transformation, and camera distortion. Especially at the edge of the image, the transition effect of the stitching seam is often not ideal. In addition, most of the existing distortion correction methods are based on global models, ignoring the local features in the image, and it is difficult to adapt to the distortion characteristics of different regions, resulting in inaccurate correction effects. Especially in complex scenes, the correction of local distortion is still a technical problem. Summary of the invention

[0004] The embodiments of the present invention provide a high-precision image stitching and correction method based on a multi-camera, which can at least to a certain extent solve the problems of inaccurate distortion correction and unnatural stitching edges that are still faced when processing multi-camera images.

[0005] Other features and advantages of the present invention will become apparent from the following detailed description, or may be learned in part by practice of the present invention.

[0006] According to one aspect of the present invention, a high-precision image stitching and correction method based on a multi-camera is provided, comprising: based on multiple perspective images provided by the multi-camera, combining a deep learning algorithm to identify potential distortion areas in the image, and intelligently partitioning each original image; and combining the distortion features of each area to output the distortion area; based on the extracted distortion area detail features and combined with the geometric consistency of adjacent areas, deriving the distortion type of each distortion area, and generating local correction parameters by dynamically adjusting the correction strength; when performing distortion correction in the distortion area, using an adaptive nonlinear transformation, by combining the mapping relationship between the pixel position and the degree of distortion, accurately correcting each pixel, and obtaining a distortion-corrected image; after the distortion-corrected images are stitched, combining the multi-scale area fusion technology to optimize the edge transition effect, and eliminate the unnatural seams and transition phenomena.

[0007] In the present invention, based on the aforementioned scheme, the deep learning algorithm is combined to identify potential distortion areas in the image, including: performing preliminary texture feature extraction on multiple perspective images, identifying areas with complex textures, and preliminarily marking them as potential distortion areas; through geometric analysis of the image, the geometric shapes in the image are classified, and attention should be paid to the areas preliminarily marked as potential distortion areas.

[0008] In the present invention, based on the above scheme, the geometric forms in the image are classified, including: using the Hough transform method to detect the straight lines in the image, that is, the straight lines in the distorted area are usually stretched or bent, deviating from the original straight line form; fitting the straight line or curve using the least squares method to analyze the degree of deviation; judging by calculating the difference between the straight line in the image and the actual geometric form: if the difference , it is determined to be a distorted area, where is the angular difference between the straight line in the image and the ideal geometric shape, To set the threshold.

[0009] In the present invention, based on the above-mentioned solution, the derivation of the distortion type of each distorted area includes: extracting the detail features of each area based on the received distorted area data; dynamically deriving the distortion type of each distorted area by considering the context information of the distorted area and combining the geometric relationship of the adjacent areas, including using the local geometric features and the geometric consistency of the adjacent areas to determine the type of distortion: wherein the geometric consistency measurement formula is as follows:

[0010]

[0011] in, For Region The geometric consistency measure of and Respectively, the regions and Region The geometric parameters of is the maximum value of the geometric morphology parameter; according to the derived geometric relationship, the classification algorithm is used to classify the distorted area and output the distortion type.

[0012] In the present invention, based on the above-mentioned solution, the local correction parameter is generated by dynamically adjusting the correction strength, including: combining the feature information and geometric consistency measurement of the distorted area, dynamically adjusting the correction strength of each area, wherein the correction strength adjustment formula is:

[0013]

[0014] in, For Region The correction strength, is the adjustment factor, is the maximum value of the geometric consistency metric; based on the correction strength, the local correction parameters of the generated area are:

[0015]

[0016] in, For Region The local correction parameter of is the correction coefficient corresponding to different distortion types.

[0017] In the present invention, based on the above-mentioned scheme, the adaptive nonlinear transformation is used to accurately correct each pixel by combining the mapping relationship between the pixel position and the degree of distortion, including: for each distorted area, calculating the degree of distortion of each pixel in the area, and establishing a mapping relationship between the pixel position and the degree of distortion; wherein the mapping relationship takes into account the spatial position, distortion type and distortion intensity in the image; and according to the geometric differences between multiple perspectives and the degree of distortion of each pixel, using an adaptive nonlinear transformation method to locally correct each pixel.

[0018] In the present invention, based on the above-mentioned scheme, the multi-scale region fusion technology is combined to optimize the transition effect of the edge and eliminate the unnatural seams and transition phenomena, including: according to the corrected image, identifying and determining the stitching area; using the multi-scale region fusion algorithm to weightedly fuse the stitching area to ensure the natural transition of the image; in the stitched image, for the situation where there are still incoherent details or inconsistent textures, the detail enhancement technology is applied to further eliminate the abruptness caused by the seams by enhancing the texture details and local contrast of the stitching area; through multi-scale region fusion and detail enhancement, an optimized stitching image is obtained.

[0019] According to one aspect of the present invention, there is provided a computer-readable medium having a computer program stored thereon, and when the computer program is executed by a processor, it implements the high-precision image stitching and correction method based on a multi-camera as described in the above embodiments.

[0020] According to one aspect of the present invention, there is provided an electronic device, including: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the high-precision image stitching and correction method based on a multi-camera as described in the above embodiments.

[0021] According to one aspect of the present invention, there is provided a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the high-precision image stitching and correction method provided in the above various optional implementation manners based on a multi-camera.

[0022] In the technical solution of the present invention, a deep learning algorithm is combined with a local distortion model and an adaptive non-linear transformation to intelligently identify the distorted areas in an image and perform precise correction. By considering the context information of the image and dynamically adjusting the distortion correction parameters, the present invention can correct local distortions more precisely, especially significantly improving the correction effect in the edge areas. The present invention also combines a multi-scale region fusion technology to optimize the transition effect of the stitching edge, can eliminate the visual unnatural phenomenon at the seam, and improves the overall accuracy and visual experience of image stitching.

[0023] Therefore, the problems of inaccurate distortion correction and unnatural seam transition in the prior art are solved.

[0024] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present invention, and are used together with the specification to explain the principles of the present invention. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.

[0026] Figure 1 Schematically shows a flowchart of the high-precision image stitching and correction method based on a multi-camera in an embodiment of the present invention.

[0027] Figure 2 The flowchart of identifying potential distortion areas in an image in one embodiment of the present invention is schematically shown.

[0028] Figure 3 The flowchart of deriving the distortion type of each distortion region in one embodiment of the present invention is schematically shown.

[0029] Figure 4 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0030] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the present invention will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0031] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present invention. However, those skilled in the art will appreciate that the technical solution of the present invention can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present invention.

[0032] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0033] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0034] The implementation details of the technical solution of the present invention are described in detail below:

[0035] Figure 1 A flowchart of a high-precision image stitching and correction method based on multi-cameras according to an embodiment of the present invention is shown. Figure 1As shown, the high-precision image stitching and correction method based on multi-cameras includes at least steps S1 to S4, which are described in detail as follows:

[0036] S1: Based on the multiple perspective images provided by the multi-view cameras, the deep learning algorithm is combined to identify the potential distortion areas in the image, and each original image is intelligently partitioned, such as complex geometric areas or areas with drastic texture changes. The distortion features of each area are combined to output the distorted area.

[0037] S1.1: Perform preliminary texture feature extraction on multiple viewpoint images to identify areas with complex textures, especially parts of the image that may be distorted, and preliminarily mark them as potential distorted areas.

[0038] Image gradient information and texture distribution features are used as input to preliminarily mark potential distortion areas.

[0039] In a multi-camera system, since there is a certain geometric relationship between the overlapping areas of multiple perspectives, the potential distortion areas can be better identified by fusing the texture information of different perspectives.

[0040] Specifically, we first use a convolutional neural network (CNN) to capture the texture change characteristics of the image. CNN can effectively identify the texture and edge information in the image by learning local features;

[0041] Image gradient (such as Sobel operator) and texture distribution features (such as gray level co-occurrence matrix, local binary pattern, etc.) are used as input, where the texture distribution features are calculated as follows:

[0042]

[0043] in, is the gray-level co-occurrence matrix, which indicates the distance between the image and and angle The grayscale contrast at is the Dirac delta function, which indicates the gray value matching of the image at that position. For images The original pixel , and Represents the gray level of the pixel value respectively.

[0044] S1.2: Through geometric analysis of images, e.g. Figure 2 As shown, the geometric forms in the image are classified, with special attention paid to the areas that are initially marked as potential distortion areas. The geometric forms (such as straight lines, angles, etc.) and depth information are combined to further classify and mark these areas as distortion areas.

[0045] Preferably, first, geometric features in the image are used to identify potential distortion areas.

[0046] Geometric shapes usually include features such as straight lines, curves, angles, and edges. These features often show unnatural changes in the distorted area. You can use methods such as Hough Transform to detect straight lines in the image. That is, the straight lines in the distorted area are usually stretched or bent, deviating from the original straight line shape.

[0047] In order to identify these changes, the least squares method can be used to fit a straight line or curve and analyze its degree of deviation.

[0048] The judgment can be made by calculating the difference between the straight line in the image and the actual geometric shape. For example, setting a threshold , if the difference , it is determined to be a distorted area, where is the angular difference between the straight line in the image and the ideal geometry.

[0049] S1.3: Based on texture and geometric morphology analysis, adaptively partition each marked distorted region.

[0050] Among them, for each distorted area, adaptive partitioning is performed according to the complexity of its texture changes and geometric shapes. Areas with drastic texture changes and obvious geometric distortion will be divided into high-priority areas, while areas with slight distortion will be divided into low-priority areas. For example, the priority of the area can be determined by comparing the weighted calculation results of the comprehensive texture changes and geometric distortion with the set grading threshold.

[0051] Different distortion correction strategies are used for areas of different priorities: a more sophisticated correction algorithm is used for high-priority areas, and a simpler correction algorithm is used for low-priority areas.

[0052] S2: Based on the extracted detailed features of the distorted area and combined with the geometric consistency of adjacent areas, the distortion type of each distorted area is derived, and the local correction parameters are generated by dynamically adjusting the correction strength to achieve a natural transition of the image.

[0053] S2.1: Based on the received distorted area data, extract the detailed features (including texture, contour, curve, etc.) of each area.

[0054] The detailed features will be used to construct a preliminary distortion model for each distorted area.

[0055] S2.2: By considering the contextual information of the distorted region and combining the geometric relationship of adjacent regions (such as symmetry and geometric continuity of adjacent regions), the distortion type and correction parameters of each distorted region are dynamically derived.

[0056] This process is based on the geometric consistency of the characteristics of the local area of ​​the image and the adjacent areas to ensure a natural transition of distortion correction.

[0057] like Figure 3 As shown, preferably, the geometric information (such as straight lines, angles, etc.) extracted in step S1 is combined to use the local geometric features and the geometric consistency of the adjacent areas to determine the type of distortion, wherein the geometric consistency measurement formula is as follows:

[0058]

[0059] in, For Region The geometric consistency measure of and Respectively, the regions and Region The geometric parameters of is the maximum value of the geometric parameters.

[0060] Furthermore, based on the derived geometric relationship, a classification algorithm (such as SVM, decision tree, etc.) is used to classify the distorted area and output the distortion type. , such as "bending distortion", "magnification distortion", etc., for example, When represents bending distortion, etc.

[0061] S2.3: Dynamically adjust distortion correction parameters based on the derived geometric relationships.

[0062] Preferably, correction parameters of different distortion types are initialized according to the derived distortion types. For example, bending distortion may require curve correction, while stretching distortion may require proportional adjustment.

[0063] Furthermore, the correction strength of each region is dynamically adjusted by combining the feature information of the distorted region and the geometric consistency metric.

[0064] Specifically, the correction strength is related to the degree of distortion and geometric consistency. The more severe the distortion, the greater the correction strength. The correction strength adjustment formula is:

[0065]

[0066] in, For Region The correction strength, is the adjustment factor, is the maximum value of the geometric consistency measure.

[0067] Furthermore, based on the correction strength, a local correction parameter of the region is generated, wherein the correction parameter can be calculated by the following formula:

[0068]

[0069] in, For Region The local correction parameter of is the correction coefficient corresponding to different distortion types.

[0070] It can be seen that in this step, the distortion type and correction parameters of each distorted area are derived by extracting the detailed features of the distorted area and analyzing the geometric consistency with the adjacent areas.

[0071] By dynamically adjusting the correction strength, a highly adaptable local correction model is generated to achieve natural transition and distortion correction of the image, which provides accurate correction parameters and framework for subsequent pixel-level correction.

[0072] S3: When performing distortion correction in the distorted area, an adaptive nonlinear transformation is used to accurately correct each pixel by combining a mapping relationship between a pixel position and a distortion degree, and a distortion-corrected image is obtained.

[0073] Especially in high distortion areas, a more sophisticated pixel-level transformation algorithm is used to avoid obvious errors in details.

[0074] Specifically, first, for each distorted region, the distortion degree of each pixel in the region is calculated, and a mapping relationship between the pixel position and the distortion degree is established.

[0075] Among them, the mapping relationship takes into account the spatial position in the image, the distortion type (such as stretching, compression, etc.), and the distortion intensity.

[0076] Furthermore, an adaptive nonlinear transformation method is used to perform local correction on each pixel.

[0077] Taking into account the multiple perspective data obtained by multi-view cameras, an adaptive nonlinear transformation method can be used for distortion correction based on the geometric differences between multiple perspectives and the degree of distortion of each pixel.

[0078] By integrating information from multiple perspectives, the distortion correction error caused by a single perspective can be effectively avoided, especially in areas where the image details are rich and vary dramatically.

[0079] Specifically, for areas with severe distortion, a more sophisticated nonlinear transformation algorithm is used to repair details, where the nonlinear transformation formula is:

[0080]

[0081] in, is the corrected pixel value, Pixel The degree of distortion at is the adjustment coefficient of the transformation, is the weight coefficient obtained by integrating multiple perspective information.

[0082] It can be seen that this step enables the correction process to be dynamically adjusted according to the distortion type and correction strength of different areas, making the correction of each pixel more precise and adaptive, ensuring the natural transition and accuracy of the details, and avoiding obvious errors or seams in high-distortion areas.

[0083] Furthermore, advanced correction algorithms are used in high distortion areas, such as Bezier curve or radial basis function (RBF) based methods. The specific calculation formula is as follows:

[0084]

[0085] in, is the pixel value after fine correction, is the adjustment coefficient for fine correction, is the power index.

[0086] This fine correction can prevent distortion of details or obvious seams in high-distortion areas. By optimizing the repair of detail areas during the correction process, it reduces obvious flaws in the transition parts, making the image correction more natural and realistic.

[0087] Finally, based on the aforementioned correction process, the corrected image is output and the correction parameters of each pixel (such as correction intensity, type, etc.) are recorded.

[0088] It can be seen that step S3 further refines the correction process to each pixel level based on step S2, and uses the adaptive nonlinear transformation method for accurate correction. By integrating information from multiple perspectives, the errors that may be caused by a single perspective are avoided. Especially in areas with severe distortion, a more sophisticated correction algorithm is used to ensure the fidelity of image details and natural transition, thereby obtaining a high-quality distortion-corrected image.

[0089] S4: After the distortion-corrected images are stitched, the multi-scale region fusion technology is used to optimize the edge transition effect and eliminate the unnatural seams and transitions.

[0090] It should be noted that image fusion algorithms of different scales can adapt to the stitching requirements of areas with different resolutions.

[0091] S4.1: According to the corrected image of S3, identify and determine the stitching area.

[0092] Specifically, the features of the splicing area are extracted through multi-scale analysis, including texture contrast, edge features, detail levels, etc. On this basis, the splicing area is divided into different scale levels.

[0093] It should be noted that dividing the stitching area into different scale levels may include generating images at multiple scale levels by constructing an image pyramid.

[0094] S4.2: Use multi-scale region fusion algorithm to perform weighted fusion on the spliced ​​regions.

[0095] During the stitching process, weighted processing is performed on features of different scales to ensure that the transition of the image is natural and the seams are not obvious, especially in the edge areas, and the areas with low contrast are enhanced to make the stitching transition smoother.

[0096] Preferably, the multi-scale weighted fusion formula is as follows:

[0097]

[0098]

[0099] in, is the pixel value of the final image after stitching, is the number of scale layers, For scale The weighting function under For scale The corrected image pixel value under ; is the normalization constant, is the pixel value The distance to the edge of the stitching area, For scale The standard deviation below.

[0100] It should be noted that the multi-camera system provides images from multiple perspectives, and can use a multi-scale region fusion algorithm to weightedly process image data from different perspectives to ensure a natural transition in the stitching area and unclear seams, especially in the edge areas, enhancing areas with low contrast to make the stitching transition smoother.

[0101] S4.3: In the stitched image, especially in the seam area, there may still be incoherent details or inconsistent textures. Detail enhancement technology can be applied to further eliminate the abruptness caused by the seam by enhancing the texture details and local contrast of the stitched area.

[0102] Optionally, this step can also be performed by repairing the texture of the seam area so that it blends better with the surrounding area.

[0103] S4.4: Finally, the optimized stitched image is obtained through multi-scale region fusion and detail enhancement.

[0104] This image has been finely fused and restored, with natural seams and rich details. This optimized image will be used as the final output and provided to subsequent applications (such as display, transmission, etc.) to ensure that the quality of the image after stitching is optimal.

[0105] Furthermore, during the image stitching process, a real-time feedback mechanism can be established to dynamically adjust the distortion correction and stitching strategies through sensor data or additional visual input.

[0106] For example, when the angle and position of the multi-view camera changes, the stitching parameters can be immediately readjusted based on the feedback information to maintain the stitching accuracy and quality.

[0107] In the technical solution of the present invention, a deep learning algorithm is used in combination with a local distortion model and an adaptive nonlinear transformation to intelligently identify the distorted area in the image and perform precise correction. By considering the contextual information of the image and dynamically adjusting the distortion correction parameters, the present invention can more accurately correct local distortion, especially in the edge area. The correction effect is significantly improved. The present invention also combines multi-scale regional fusion technology to optimize the transition effect of the splicing edge, which can eliminate the visual unnatural phenomenon at the seam and improve the accuracy and visual experience of the overall image splicing.

[0108] Therefore, the problems of inaccurate correction of image splicing distortion and unnatural seam transition in the prior art are solved.

[0109] Figure 4 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present invention is shown.

[0110] It should be noted that the computer system of the electronic device in this embodiment is only an example and should not bring any limitation to the functions and scope of use of the embodiment of the present invention.

[0111] In this embodiment, the computer system includes a central processing unit 401, which can perform various appropriate actions and processes according to the program stored in the read-only memory 402 or the program loaded from the storage part 408 to the random access memory 403, such as executing the high-precision image stitching and correction method based on the multi-eye camera described in the above embodiment. In the random access memory 403, various programs and data required for system operation are also stored. The central processing unit 401, the read-only memory 402 and the random access memory 403 are connected to each other via a bus 404. The input / output interface 405 is also connected to the bus 404.

[0112] The following components are connected to the input / output interface 405: an input section 406 including a keyboard, a mouse, etc.; an output section 407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the input / output interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed so that a computer program read therefrom is installed into the storage section 408 as needed.

[0113] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 409, and / or installed from the removable medium 411. When the computer program is executed by the central processing unit 401, various functions defined in the system of the present invention are executed.

[0114] It should be noted that the computer-readable medium shown in the embodiment of the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable computer program. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0115] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. Among them, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0116] The units involved in the embodiments of the present invention may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, limit the units themselves.

[0117] According to one aspect of the present invention, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above-mentioned various optional implementations.

[0118] As another aspect, the present invention further provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the high-precision image stitching and correction method based on multi-eye cameras described in the above embodiment.

[0119] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0120] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation of the present invention.

[0121] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the embodiments disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary technical means in the art that are not disclosed in the present invention.

[0122] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A high-precision image stitching and correction method based on multi-cameras, characterized in that: include: Based on the multiple perspective images provided by the multi-camera, combined with the deep learning algorithm, the potential distortion areas in the image are identified, each original image is intelligently partitioned, and the distortion areas are output; Based on the extracted detailed features of the distorted area and combined with the geometric consistency of adjacent areas, the distortion type of each distorted area is derived, and the local correction parameters are generated by dynamically adjusting the correction strength; When performing distortion correction in the distorted area, an adaptive nonlinear transformation is used to accurately correct each pixel by combining a mapping relationship between a pixel position and a distortion degree, and a distortion-corrected image is obtained; After the distortion-corrected images are stitched together, the edge transition effect is optimized by combining the multi-scale region fusion technology to eliminate the unnatural seams and transitions; The distortion type of each distorted area is derived, including: extracting detailed features of each area based on the received distorted area data; dynamically deriving the distortion type of each distorted area by considering the context information of the distorted area and combining the geometric relationship of adjacent areas, including: using local geometric features and geometric consistency of adjacent areas to determine the type of distortion; using a classification algorithm to classify the distorted area according to the derived geometric relationship and output the distortion type; the adaptive nonlinear transformation is used to accurately correct each pixel by combining the mapping relationship between pixel position and distortion degree, including: for each distorted area, calculating the distortion degree of each pixel in the area, and establishing a mapping relationship between pixel position and distortion degree; wherein the mapping relationship considers the spatial position, distortion type and distortion intensity in the image; and using an adaptive nonlinear transformation method to locally correct each pixel according to the geometric differences between multiple perspectives and the distortion degree of each pixel.

2. The high-precision image stitching and correction method based on multi-camera according to claim 1, characterized in that: The method of combining a deep learning algorithm to identify potential distortion areas in an image includes: Perform preliminary texture feature extraction on multiple-view images, identify areas with complex textures, and preliminarily mark them as potential distortion areas; Through geometric analysis of the image, the geometric shapes in the image are classified, and attention should be paid to the areas that are initially marked as potential distortions.

3. The high-precision image stitching and correction method based on multi-camera according to claim 2, characterized in that: The classifying of geometric forms in the image comprises: Use the Hough transform method to detect straight lines in the image, that is, the straight lines in the distorted area will be stretched or bent, deviating from the original straight line shape; Use the least squares method to fit a straight line or curve and analyze the degree of deviation; The judgment is made by calculating the difference between the straight line in the image and the actual geometric shape: If the difference , it is determined to be a distorted area, where is the angular difference between the straight line in the image and the ideal geometric shape, To set the threshold.

4. The high-precision image stitching and correction method based on multi-camera according to claim 1, characterized in that: The geometric consistency measurement formula is as follows: in, For Region The geometric consistency measure of and Respectively, the regions and Region The geometric parameters of is the maximum value of the geometric parameters.

5. The high-precision image stitching and correction method based on multi-camera according to claim 1, characterized in that: Generate local correction parameters by dynamically adjusting the correction strength, including: Combined with the feature information and geometric consistency measurement of the distorted area, the correction strength of each area is dynamically adjusted, where the correction strength adjustment formula is: in, For Region The correction strength, is the adjustment factor, is the maximum value of the geometric consistency measure; Based on the correction strength, the local correction parameters of the generated area are: in, For Region The local correction parameter of is the correction coefficient corresponding to different distortion types.

6. The high-precision image stitching and correction method based on multi-camera according to claim 1, characterized in that: The multi-scale region fusion technology is combined to optimize the edge transition effect and eliminate the unnatural seams and transition phenomena, including: According to the corrected image, identify and determine the stitching area; A multi-scale regional fusion algorithm is used to perform weighted fusion of the stitching areas to ensure a natural transition of the image; In the stitched image, if there are still incoherent details or inconsistent textures, the detail enhancement technology is applied to further eliminate the abruptness caused by the seams by enhancing the texture details and local contrast of the stitched area; Through multi-scale region fusion and detail enhancement, an optimized stitched image is obtained.

Images