An image stitching method, device, equipment and medium

Abstract

An embodiment of the present application provides an image stitching method, device, equipment and medium, which relates to the field of computer technology, and includes: obtaining a first to-be-stitched view and a second to-be-stitched view collected by a first objective lens and a second objective lens; mapping the second to-be-stitched view according to a second target mapping relationship to obtain a second target image; stitching the first to-be-stitched view and the second target image; the second target mapping relationship is generated by the following method: determining matching pixel points in the first objective view and the second objective view; using the position correspondence relationship between the first pixel point and the second pixel point in each matching pair to obtain an initial mapping relationship; calculating a compensation value according to the focal length, image width, camera spacing, sensor width, first distance, and second distance of the multi-camera; compensating the initial mapping relationship with the compensation value to obtain the second target mapping relationship. Applying the solution provided by the embodiment of the present application can improve the quality of the stitched image.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to an image stitching method, apparatus, device, and medium. Background Art

[0002] In the field of monitoring, in order to obtain a broader monitoring view, multi-eye cameras are usually used for image acquisition, and then the images collected by each eye camera in the multi-eye cameras are stitched.

[0003] In the related art, when performing image stitching, the overlapping area of the images collected by each eye camera is usually determined, and then the views collected by different cameras are merged according to the overlapping area to stitch the images collected by different cameras together.

[0004] Although the above solution can achieve image stitching, due to the different parameters of different cameras, there may be a problem of misalignment after directly stitching the images collected by different cameras according to the overlapping area, resulting in poor quality of the stitched image. Summary of the Invention

[0005] The purpose of the embodiments of this application is to provide an image stitching method, apparatus, device, and medium to improve the quality of the stitched image. The specific technical solutions are as follows:

[0006] In a first aspect, the embodiments of this application provide an image stitching method, and the method includes:

[0007] Obtain a first view to be stitched collected by a first eye lens, and obtain a second view to be stitched collected by a second eye lens, where the first eye lens and the second eye lens are two adjacent lenses in the same multi-eye camera, and there is an overlapping area in the first view to be stitched and the second view to be stitched;

[0008] Perform mapping processing on each second pixel point in the second view to be stitched according to a second target mapping relationship to obtain a second target image;

[0009] Stitch the overlapping areas in the first view to be stitched and the second target image;

[0010] Among them, the second target mapping relationship is generated by the following method:

[0011] Obtain a first view of the first eye lens, and obtain a second view of the second eye lens, where the first view and the second view are views collected by the multi-eye camera in the mapping relationship generation scenario, and there is an overlapping area in the first view and the second view;

[0012] Determine the matching pixel points in the first view and the second view to obtain a plurality of matching pairs, where each matching pair includes a first pixel point located in the overlapping area in the first view and a second pixel point located in the overlapping area in the second view;

[0013] Utilize the position correspondence relationship between the first pixel point and the second pixel point in each matching pair to determine the position conversion relationship from each pixel point in the second view to each pixel point in the first view, and obtain the initial mapping relationship for mapping the second view to the first view;

[0014] According to the focal length, image width, camera spacing, sensor width, first distance, and second distance of the multi-view camera, calculate the compensation value of the initial mapping relationship, where the image width is: the width of the image collected by the multi-view camera, the camera spacing is: the spacing between the first view lens and the second view lens, the sensor width is: the width of the sensor of each view camera in the multi-view camera, the first distance is: the distance of the object in the overlapping area relative to the multi-view camera, and the second distance is: the distance of the target in the actual application scenario of the multi-view camera relative to the multi-view camera;

[0015] Utilize the calculated compensation value to compensate the initial mapping relationship to obtain the second target mapping relationship.

[0016] In a second aspect, an embodiment of the present application provides an image stitching device, and the device includes:

[0017] A view to be stitched obtaining module, configured to obtain a first view to be stitched collected by a first view lens and obtain a second view to be stitched collected by a second view lens, where the first view lens and the second view lens are two adjacent lenses in the same multi-view camera, and there is an overlapping area in the first view to be stitched and the second view to be stitched;

[0018] A first mapping module, configured to perform mapping processing on each second pixel point in the second view to be stitched according to the second target mapping relationship to obtain a second target image;

[0019] An image stitching module, configured to stitch the overlapping areas in the first view to be stitched and the second target image;

[0020] Among them, the second target mapping relationship is generated by the following module:

[0021] A view acquisition module, configured to acquire a first view captured by the first objective lens and a second view captured by the second objective lens, where the first view and the second view are views captured by the multi-lens camera in a mapping relationship generation scenario, and there is an overlapping area in the first view and the second view;

[0022] A matching pair determination module, configured to determine matching pixel points in the first view and the second view to obtain a plurality of matching pairs, where each matching pair includes a first pixel point located in the overlapping area in the first view and a second pixel point located in the overlapping area in the second view;

[0023] An initial mapping relationship acquisition module, configured to use the position correspondence relationship between the first pixel point and the second pixel point in each matching pair to determine the position conversion relationship from each pixel point in the second view to each pixel point in the first view, and obtain an initial mapping relationship for mapping the second view to the first view;

[0024] A compensation value calculation module, configured to calculate a compensation value for the initial mapping relationship according to the focal length, image width, camera spacing, sensor width, first distance, and second distance of the multi-lens camera, where the image width is the width of the image captured by the multi-lens camera, the camera spacing is the spacing between the first objective lens and the second objective lens, the sensor width is the width of the sensor of each camera in the multi-lens camera, the first distance is the distance of an object in the overlapping area relative to the multi-lens camera, and the second distance is the distance of a target in the actual application scenario of the multi-lens camera relative to the multi-lens camera;

[0025] A second target acquisition module, configured to compensate the initial mapping relationship using the calculated compensation value to obtain a second target mapping relationship.

[0026] In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0027] The memory is used to store a computer program;

[0028] The processor, when executing the program stored in the memory, implements the method steps of any one of the first aspect.

[0029] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method steps of any one of the first aspect are implemented.

[0030] An embodiment of the present application further provides a computer program product including instructions, which when running on a computer, causes the computer to execute the image stitching method described in any one of the above.

[0031] Advantageous effects of the embodiments of the present application:

[0032] In the solution provided by the embodiment of the present application, a first view to be stitched collected by a first objective lens can be obtained, and a second view to be stitched collected by a second objective lens can be obtained, where the first objective lens and the second objective lens are two adjacent lenses in the same multi-objective camera, and there is an overlapping area in the first view to be stitched and the second view to be stitched; perform mapping processing on each second pixel point in the second view to be stitched according to the second target mapping relationship to obtain a second target image; stitch the overlapping areas in the first view to be stitched and the second target image; where the second target mapping relationship is generated by the following method: obtain a first objective view collected by the first objective lens, and obtain a second objective view collected by the second objective lens, where the first objective view and the second objective view are views collected by the multi-objective camera in the mapping relationship generation scenario, and there is an overlapping area in the first objective view and the second objective view; determine the matching pixel points in the first objective view and the second objective view to obtain a plurality of matching pairs, where each matching pair includes a first pixel point located in the overlapping area in the first objective view and a second pixel point located in the overlapping area in the second objective view; use the position correspondence relationship between the first pixel point and the second pixel point in each matching pair to determine the position conversion relationship between each pixel point in the second objective view and each pixel point in the first objective view, and obtain an initial mapping relationship for mapping the second objective view to the first objective view; calculate a compensation value for the initial mapping relationship according to the focal length, image width, camera spacing, sensor width, first distance, and second distance of the multi-objective camera, where the image width is: the width of the image collected by the multi-objective camera, the camera spacing is: the spacing between the first objective lens and the second objective lens, the sensor width is: the width of the sensor of each objective camera in the multi-objective camera, the first distance is: the distance of the object in the overlapping area relative to the multi-objective camera, and the second distance is: the distance of the target relative to the multi-objective camera in the actual application scenario of the multi-objective camera; use the calculated compensation value to compensate the initial mapping relationship to obtain the second target mapping relationship. In this way, in the actual application scenario, the above second target mapping relationship can be used to perform mapping processing on the pixel points in the second view to be stitched collected by the second objective lens. Since the above second target mapping relationship can reflect the position conversion relationship between each pixel point in the second objective view and each pixel point in the first objective view collected by the first objective lens, the pixel points in the mapped second view to be stitched correspond to the pixel points in the first view to be stitched collected by the first objective lens, and thus the pixel points in different views can be prevented from being misaligned during stitching. It can be seen that applying the solution provided by the embodiment of the present application can improve the quality of the stitched image. Description of the Drawings

[0033] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for describing the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other embodiments can also be obtained based on these drawings.

[0034] Figure 1 It is a schematic flowchart of an image stitching method provided by an embodiment of the present application;

[0035] Figure 2 It is a schematic flowchart of a mapping relationship generation method provided by an embodiment of the present application;

[0036] Figure 3 It is a schematic flowchart of a method for obtaining an initial mapping relationship provided by an embodiment of the present application;

[0037] Figure 4 It is a schematic diagram of a mapping relationship generation process provided by an embodiment of the present application;

[0038] Figure 5 It is a schematic flowchart of a color difference adjustment method provided by an embodiment of the present application;

[0039] Figure 6 It is a schematic diagram of a process for obtaining a target color gain provided by an embodiment of the present application;

[0040] Figure 7 It is a schematic flowchart of a pixel fusion method provided by an embodiment of the present application;

[0041] Figure 8 It is a schematic diagram of calculating the mean value using a sliding window provided by an embodiment of the present application;

[0042] Figure 9 It is a schematic diagram of a pixel fusion process provided by an embodiment of the present application;

[0043] Figure 10 It is a schematic structural diagram of an image stitching device provided by an embodiment of the present application;

[0044] Figure 11 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0045] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art based on the present application belong to the scope of protection of the present application.

[0046] To improve the accuracy of image stitching, the embodiments of the present application provide an image stitching method, device, equipment and medium, which will be introduced in detail below.

[0047] The embodiments of the present application provide an image stitching method, which can be applied to electronic devices such as computers, servers, mobile phones, and multi-camera cameras. The method includes:

[0048] Obtain a first view to be stitched collected by a first camera lens, and obtain a second view to be stitched collected by a second camera lens, where the first camera lens and the second camera lens are two adjacent lenses in the same multi-camera camera, and there is an overlapping area in the first view to be stitched and the second view to be stitched;

[0049] Perform mapping processing on each second pixel point in the second view to be stitched according to the second target mapping relationship to obtain a second target image;

[0050] Stitch the overlapping areas in the first view to be stitched and the second target image;

[0051] Among them, the second target mapping relationship is generated in the following manner:

[0052] Obtain a first view collected by the first camera lens, and obtain a second view collected by the second camera lens, where the first view and the second view are views collected by the multi-camera camera in the mapping relationship generation scenario, and there is an overlapping area in the first view and the second view;

[0053] Determine the matching pixel points in the first view and the second view to obtain a plurality of matching pairs, where each matching pair includes a first pixel point located in the overlapping area in the first view and a second pixel point located in the overlapping area in the second view;

[0054] Utilize the position correspondence relationship between the first pixel point and the second pixel point in each matching pair to determine the position conversion relationship between each pixel point in the second view and each pixel point in the first view, and obtain an initial mapping relationship for mapping the second view to the first view;

[0055] Calculate the compensation value of the initial mapping relationship according to the focal length, image width, camera spacing, sensor width, first distance, and second distance of the multi-camera, where the image width is the width of the image collected by the multi-camera, the camera spacing is the distance between the first camera lens and the second camera lens, the sensor width is the width of the sensor of each camera in the multi-camera, the first distance is the distance of the object in the overlapping area relative to the multi-camera, and the second distance is the distance of the target in the actual application scenario of the multi-camera relative to the multi-camera;

[0056] Use the calculated compensation value to compensate the initial mapping relationship to obtain the second target mapping relationship.

[0057] In this way, in the actual application scenario, the pixel points in the second view to be stitched collected by the second camera lens can be mapped using the above second target mapping relationship. Since the above second target mapping relationship can reflect the position conversion relationship between the pixel points in the second view and the pixel points in the first view collected by the first camera lens, the pixel points in the mapped second view to be stitched correspond to the pixel points in the first view to be stitched collected by the first camera lens, thereby avoiding the misalignment of pixel points in different views during stitching. It can be seen that applying the solution provided in the embodiments of the present application can improve the quality of the stitched image.

[0058] The above image stitching method will be introduced in detail below.

[0059] See Figure 1 , Figure 1 which is a schematic flowchart of an image stitching method provided by an embodiment of the present application. The method includes the following steps S101 - S103:

[0060] S101, Obtain the first view to be stitched collected by the first camera lens and obtain the second view to be stitched collected by the second camera lens.

[0061] Among them, the first camera lens and the second camera lens are two adjacent lenses in the same multi-camera, and there is an overlapping area in the first view to be stitched and the second view to be stitched.

[0062] The above multi-camera can be a binocular camera, a trinocular camera, a quad-camera, etc. The types of the above multi-cameras can be spherical eagle eye cameras, bowl-shaped eagle eye cameras, surround view cameras, etc.

[0063] The above overlapping area refers to the image area corresponding to the same actual scene area in the images collected by different camera lenses.

[0064] Specifically, a multi-view camera may include multiple lenses. Two adjacent lenses among the multiple lenses are respectively used as the first-view lens and the second-view lens. For example, the left lens among two adjacent lenses can be used as the first-view lens, and the right lens can be used as the second-view lens. Alternatively, the right lens among two adjacent lenses can be used as the first-view lens, and the left lens can be used as the second-view lens.

[0065] In practical applications, when using the above multi-view camera for image acquisition, the image captured by the first-view lens can be obtained as the first view to be stitched, and the image captured by the second-view lens can be obtained as the second view to be stitched. Since the first-view lens and the second-view lens are adjacent in position, and the orientations of the lenses in the multi-view camera are usually the same, there is usually an overlapping area between the first view to be stitched and the second view to be stitched.

[0066] S102, perform mapping processing on each second pixel point in the second view to be stitched according to the second target mapping relationship to obtain a second target image.

[0067] Specifically, in practical applications, after obtaining the second view to be stitched captured by the second-view lens, the second view can be subjected to mapping processing according to the pre-generated second target mapping relationship, so as to obtain the mapped image as the second target image.

[0068] Among them, the generation method of the second target mapping relationship will be introduced in detail later.

[0069] S103, stitch the overlapping areas in the first view to be stitched and the second target image.

[0070] Specifically, the converted second target image matches the pixel points in the overlapping area of the first view to be stitched captured by the first-view lens. Based on this, the overlapping areas in the first view to be stitched and the second target image can be stitched to obtain a stitched image. This can avoid the misalignment of pixel points in different views during stitching and improve the quality of the stitched image.

[0071] Next, the mapping relationship generation method will be introduced in detail.

[0072] See Figure 2 , Figure 2 which is a schematic flowchart of a mapping relationship generation method provided by an embodiment of the present application. This method can be applied to electronic devices such as computers, servers, and mobile phones. The method includes the following steps S201-S205:

[0073] S201, obtain the first-view image captured by the first-view lens and obtain the second-view image captured by the second-view lens.

[0074] Among them, the first-eye view and the second-eye view are views collected by the multi-eye camera in the mapping relationship generation scenario, and there is an overlapping area between the first-eye view and the second-eye view.

[0075] The above mapping relationship generation scenario is a scenario for generating a mapping relationship, which can be an actual application scenario, or a scenario such as a laboratory or a production line.

[0076] Specifically, when using the above multi-eye camera to collect images for subsequent generation of the mapping relationship, the image collected by the first-eye lens can be obtained as the first-eye view, and the image collected by the second-eye lens can be obtained as the second-eye view. Since the positions of the first-eye lens and the second-eye lens are adjacent, and the orientations of the lenses in the multi-eye camera are usually the same, there is usually an overlapping area between the first-eye view and the second-eye view.

[0077] In an embodiment of the present application, the view collected by the first-eye lens can be obtained, and the obtained view can be preprocessed based on the camera parameters of the first-eye lens to obtain the first-eye view.

[0078] Among them, the preprocessing includes: distortion correction processing and / or image projection processing. The image projection processing is: the processing of projecting the image onto a preset curved surface;

[0079] The above distortion correction processing is used to eliminate the distortion phenomenon in the image collected by the lens;

[0080] The above image projection processing is used to project the image collected by the lens onto a preset curved surface to obtain the projected image, so as to eliminate the image difference caused by different shooting angles of different lenses.

[0081] The above camera parameters include at least one of the following parameters: installation position, shooting angle of the lens, focal length, distance between the lenses, distortion coefficient, etc. The above camera parameters can be obtained by manual measurement or pre-calibrated based on the calibration images collected in the calibration scenario.

[0082] Specifically, after obtaining the view collected by the first-eye lens, the view collected by the first-eye lens can be preprocessed based on the camera parameters of the first-eye lens, and the preprocessed view is used as the first-eye view.

[0083] Correspondingly to the above solution, the view collected by the second-eye lens can also be obtained, and the obtained view can be preprocessed based on the camera parameters of the second-eye lens to obtain the second-eye view.

[0084] S202, determine the matching pixel points in the first-eye view and the second-eye view to obtain a plurality of matching pairs.

[0085] Among them, each matching pair includes a first pixel point located in the overlapping area in the first-eye view and a second pixel point located in the overlapping area in the second-eye view.

[0086] The matching pixel points refer to: pixel points with the same described content.

[0087] It can be understood that: each matching pair contains a first pixel point and a second pixel point, and the two pixel points contained in the matching pair are matched.

[0088] In an embodiment of the present application, features of each first pixel point located in the overlapping area in the first-eye view can be extracted, and features of each second pixel point located in the overlapping area in the second-eye view can be extracted; calculate the similarity between the features of each first pixel point and the features of each second pixel point, select the first pixel points and second pixel points whose similarity is greater than a preset similarity threshold to obtain multiple matching pairs; screen the obtained matching pairs to obtain matching pairs that meet the consistency.

[0089] Specifically, there is an overlapping area between the first-eye view and the second-eye view, and there are pixel points with the same described content in this overlapping area. In view of this, each first pixel point in the overlapping area in the first-eye view can be determined, the features of the above first pixel points can be extracted, and each second pixel point in the overlapping area in the second-eye view can be determined, the features of the above second pixel points can be extracted. Based on the similarity between the features of the first pixel points and the second pixel points, select the first pixel points and second pixel points whose similarity between each other is greater than a preset similarity threshold as matching pairs. After obtaining the matching pairs based on the features, the above matching pairs can be further screened to obtain more accurate matching pairs.

[0090] In an embodiment of the present application, when screening the matching pairs, different random sample consensus algorithms can be used to screen the obtained matching pairs to obtain different screening results; fuse the obtained different screening results to obtain matching pairs that meet the consistency.

[0091] Specifically, multiple different random sample consensus algorithms can be used to screen each matching pair respectively. For each random sample consensus algorithm, a screening result can be obtained. The different screening results can be fused, and the fused screening result can be used as the final matching pair that meets the consistency. In this way, different algorithms can be used to purify the above matching pairs, and the accuracy of the obtained matching pairs after purification is higher.

[0092] In an embodiment of the present application, when fusing multiple different screening results, the intersection of the multiple different screening results can be determined as the final screening result; alternatively, the union of the multiple different screening results can be determined as the final screening result.

[0093] In one embodiment of the present application, different random sample consensus algorithms include at least two of the following algorithms:

[0094] Random sample consensus algorithm based on similarity transformation, random sample consensus algorithm based on affine transformation, random sample consensus algorithm based on projective transformation.

[0095] Specifically, at least two of the above three algorithms can be used to screen the matching pairs obtained in step S102 to obtain at least two different screening results.

[0096] Taking the random sample consensus algorithm based on affine transformation as an example, the coordinates of two pixel points in the matching pair can be input into a 3×3 affine transformation matrix for mapping, the values of the parameters in the affine transformation matrix are calculated, and an affine transformation matrix adapted to multiple matching pairs is obtained based on the mapping result. For the matching pairs that do not adapt to this matrix, they are excluded, and the matching pairs that adapt to the above matrix are used as the screening results.

[0097]

[0098] Among them, the above x' and y' represent the abscissa and ordinate of the second pixel point in the second view, x and y represent the abscissa and ordinate of the first pixel point in the first view, m00, m01, m02, m10, m11, m12 represent the preset transformation parameters in the affine transformation matrix, and m00', m01', m10', m11' represent the transformation parameters determined based on the coordinate correspondence of the pixel points in the matching pair in the affine transformation matrix.

[0099] S203. Using the position correspondence relationship between the first pixel point and the second pixel point in each matching pair, determine the position transformation relationship from each pixel point in the second view to each pixel point in the first view, and obtain the initial mapping relationship for mapping the second view to the first view.

[0100] Specifically, the position correspondence relationship between the first pixel point and the second pixel point in each matching pair can be determined. This position correspondence relationship can reflect the position transformation relationship from each pixel point in the second view to each pixel point in the first view, and then the above position transformation relationship can be used as the initial mapping relationship for mapping the second view to the first view.

[0101] In one embodiment of the present application, a preset matrix transformation method can be used to calculate the position transformation matrix from the second pixel point to the first pixel point in the obtained matching pairs, and obtain the position transformation relationship from each pixel point in the second view to each pixel point in the first view as the initial mapping relationship.

[0102] Among them, the matrix transformation method includes any one of the methods for obtaining the following matrices: similarity transformation matrix, affine transformation matrix, and projection transformation matrix.

[0103] Specifically, based on a preset matrix, a matrix for mapping the second pixel point to the first pixel point in the matching pair can be calculated as the position transformation matrix between the second pixel point and the first pixel point in the matching pair, and then the above position transformation matrix can be used as the initial mapping relationship.

[0104] Taking the matrix transformation method as the method for performing position transformation based on the affine transformation matrix as an example, the coordinates (x, y) of each second pixel point in the matching pair can be mapped into a 3×3 affine transformation matrix one by one to obtain the mapped coordinates (x', y'). Based on the difference between the mapping result and the first pixel point in the matching pair, the mapping matrix is adjusted, and finally the adjusted affine transformation matrix is obtained. Based on the affine transformation matrix, the initial mapping relationship between the positions of each pixel point in the second view image and each pixel point in the first view image is obtained.

[0105] In one embodiment of the present application, after obtaining the above affine transformation matrix, a correspondence list between the coordinates of each pixel point in the second view image and the mapped coordinates can be obtained based on the affine transformation matrix, and the above list is used as the initial mapping relationship, which is convenient for subsequent processing directly based on the correspondence list.

[0106] S204. Calculate a compensation value for the initial mapping relationship according to the focal length, image width, camera spacing, sensor width, first distance, and second distance of the multi-view camera.

[0107] Among them, the image width is: the width of the image collected by the multi-view camera, in pixel points;

[0108] The camera spacing is: the spacing between the first view lens and the second view lens, and the unit can be millimeters, centimeters, etc.;

[0109] The sensor width is: the width of the sensor of each view camera in the multi-view camera, and the unit can be millimeters, centimeters, etc.;

[0110] The first distance is: the distance of the object in the overlapping area relative to the multi-view camera, and the unit can be meters, centimeters, millimeters, etc.;

[0111] The second distance is: the distance of the target in the actual application scenario of the multi-view camera relative to the multi-view camera, and the unit can be meters, centimeters, millimeters, etc.

[0112] Specifically, after obtaining the initial mapping relationship, a compensation value for correcting the initial mapping relationship can be determined based on the focal length, image width, camera spacing, sensor width, first distance, and second distance of the multi-camera, according to the actual application scenario. Subsequently, this compensation value can be used to correct the initial mapping relationship, so that the corrected relationship better adapts to the actual application scenario, and avoid problems such as misalignment and ghosting in the stitched image due to mapping relationship errors during application.

[0113] In one embodiment of the present application, the compensation value deta_x of the initial mapping relationship can be calculated according to the following formula:

[0114] deta_x = fbWk(1 / L1 – 1 / L2)

[0115] Wherein, f represents the focal length of the multi-camera, W represents the image width, b represents the camera spacing, k represents the sensor width, L1 represents the first distance, and L2 represents the second distance.

[0116] S205, use the calculated compensation value to compensate the initial mapping relationship to obtain the second target mapping relationship.

[0117] Specifically, after calculating the compensation value, this compensation value can be used to compensate the above initial mapping relationship, so that the compensated mapping relationship better adapts to the actual application scenario of the multi-camera.

[0118] In one embodiment of the present application, when the above initial mapping relationship is a correspondence list, the coordinates of the mapped pixel points in the above list can be directly compensated based on the calculated compensation value, so as to obtain the second target mapping relationship.

[0119] In one embodiment of the present application, after obtaining the initial mapping relationship in step S203, the initial mapping relationship can be imported into the multi-camera, so as to deploy the multi-camera in the application scenario, and calculate the above compensation value based on the first distance, second distance measured in the application scenario, and parameters such as the focal length, image width, camera spacing, and sensor width of the multi-camera, and correct the initial mapping relationship imported into the multi-camera based on this compensation value, so that the corrected multi-camera better adapts to this application scenario.

[0120] See Figure 3 , Figure 3 is a schematic flowchart of a method for obtaining an initial mapping relationship provided by an embodiment of the present application. For the above step S203 when obtaining the initial mapping relationship, it includes the following steps S301 - S303:

[0121] S301. Determine the position conversion relationship from each pixel point in the second view to each pixel point in the first view by using the position correspondence between the first pixel point and the second pixel point in each matching pair.

[0122] Specifically, the position correspondence between the first pixel point and the second pixel point in each matching pair can be determined, and this position correspondence can reflect the position conversion relationship from each pixel point in the second view to each pixel point in the first view.

[0123] S302. Convert the positions of each pixel point in the second view according to the position conversion relationship to obtain a converted image, and crop the area in the converted image that exceeds the coordinate range of the second view to obtain a first cropping conversion relationship for cropping the converted second view.

[0124] Specifically, the positions of each pixel point in the second view can be converted according to the above position conversion relationship to obtain a converted image. There may be a situation where the positions of pixel points in the converted image exceed the coordinate range of the pixel points in the second view. Therefore, the pixel points that exceed in the converted image can be cropped to obtain a cropped converted image, and a first cropping relationship for cropping the converted second view can be determined.

[0125] In an embodiment of the present application, when determining the first cropping relationship, the converted image can be horizontally translated and vertically translated to remove the area in the converted image that exceeds the coordinate range of the second view, and the translation conversion relationship of the above horizontal translation and vertical translation can be used as the first cropping conversion relationship.

[0126] S303. Integrate the position conversion relationship and the first cropping conversion relationship to obtain an initial mapping relationship from each pixel point in the second view to each pixel point in the cropped converted image.

[0127] Specifically, the position conversion relationship obtained in step S301 and the first cropping conversion relationship obtained in step S302 can be integrated, and the integrated conversion relationship can be used as the initial mapping relationship. By using this initial conversion relationship, the second view can be directly converted into the cropped converted image.

[0128] Based on the above Figure 3 shown embodiment, in the present application, the first view can also be cropped according to the cropping method of the converted image to obtain a mapping relationship from each pixel point in the first view to each pixel point in the cropped first view as the first target mapping relationship.

[0129] Specifically, the first eye view can be cropped in the same way as the converted image is cropped, so that the cropped first eye view is consistent with the cropped converted image, and the position conversion relationship between each pixel point in the first eye view and each pixel point in the cropped first eye view can be obtained as the first target mapping relationship. After performing coordinate conversion on the first eye view using the first target mapping relationship, it can be ensured that the converted first eye view is consistent with the converted second eye view, facilitating subsequent stitching of the converted views.

[0130] In an embodiment of the present application, after obtaining the above mapping relationship, the above mapping relationship can be imported into the multi-eye camera, so that after the multi-eye camera captures multi-eye views, each multi-eye view can be converted based on the above mapping relationship, and then the converted views can be stitched.

[0131] In an embodiment of the present application, when the second target mapping relationship reflects the mapping relationship between pixel points in the second eye view and the cropped image, the first view to be stitched is converted according to the first target mapping relationship to obtain the first target image, and the overlapping regions in the first target image and the second target image are stitched.

[0132] Among them, the cropped image is an image obtained by converting the second eye view to the first eye view and cropping the pixel points outside the coordinate range of the second eye view, and the first target mapping relationship is the mapping relationship between pixel points in the first eye view and the cropped image.

[0133] Specifically, in the above solution, if there are a first target mapping relationship and a second target mapping relationship, the first view to be stitched can be processed according to the first target mapping relationship, and the second view to be stitched can be processed according to the second target mapping relationship to obtain the processed first target image and second target image, and then the overlapping regions in the first target image and the second target image are stitched.

[0134] See Figure 4 , Figure 4 is a schematic diagram of a mapping relationship generation process provided by an embodiment of the present application. Assuming that the first eye lens is the left lens, the first eye view captured by the first eye lens can be called the base image base_img, and the second eye lens is the right lens, and the second eye view captured by the second eye lens can be called the registration image sub_img. When generating the mapping relationship, the base_img and sub_img captured in the mapping relationship generation scenario can be obtained first;

[0135] Then, the ROI (region of interest) regions, that is, the overlapping regions, in the base_img and sub_img can be determined;

[0136] Extract the features of the first pixel point in the ROI region of base_img and the features of the first pixel point in the ROI region of sub_img respectively;

[0137] Determine each matching pair based on the similarity between the features of the first pixel point and the second pixel point;

[0138] Use different random sample consensus algorithms to screen the obtained matching pairs to obtain different screening results; fuse the obtained different screening results to obtain matching pairs that meet the consistency, and achieve the purification of each matching pair;

[0139] Based on the position correspondence relationship between the first pixel point and the second pixel point in each matching pair, determine the projection matrix of each pixel point in sub_img to each pixel point in base_img, and obtain the initial mapping relationship of sub_img mapping to base_img;

[0140] Convert the positions of each pixel point in sub_img according to the initial mapping relationship to obtain a transformed image, and through translation transformation, crop the area that exceeds the coordinate range of the second view image in the transformed image to obtain the first cropping transformation relationship for cropping the transformed sub_img;

[0141] Fuse the position transformation relationship and the first cropping transformation relationship to obtain the initial mapping relationship of each pixel point in the second view image to each pixel point in the cropped transformed image;

[0142] According to the focal length, image width, camera spacing, sensor width, first distance, and second distance of the multi-camera, correct the above initial mapping relationship to obtain the second target mapping relationship.

[0143] In an embodiment of the present application, when the number of lenses in the multi-camera is greater than or equal to 2, the views collected by each lens can also be adjusted for chromatic aberration, which will be introduced in detail below.

[0144] See Figure 5 , Figure 5 It is a schematic flowchart of a chromatic aberration adjustment method provided by an embodiment of the present application. The method includes the following steps S501-S503:

[0145] S501, for each view collected by each lens, calculate the initial color gain of the view collected by this lens relative to the view collected by the adjacent lens adjacent to this lens based on the pixel values of the view collected by this lens and the view collected by the adjacent lens adjacent to this lens.

[0146] Among them, the pixel value of each view includes: the components of the pixel point in the RGB three channels of this view.

[0147] Specifically, for each camera lens, the adjacent lenses in the multi-camera can be used as adjacent lenses. By using the views captured by the above adjacent lenses and the RGB values of the pixel points in the view captured by this lens, the color gain of the view captured by this lens relative to the view captured by the adjacent lens can be calculated as the initial color gain.

[0148] In an embodiment of the present application, when determining the initial color gain corresponding to the view captured by each lens, the initial color gain corresponding to the pixel value of each channel in the RGB channel can be calculated separately.

[0149] Specifically, for the pixel value of each channel, based on the view captured by this camera lens and the pixel value of this channel in the view captured by the adjacent lens adjacent to this camera lens, the initial color gain of the pixel value of this channel in the view captured by this camera lens relative to the view captured by the adjacent lens can be calculated. That is, the initial color gain corresponding to the view captured by each lens includes the color gain corresponding to the pixel value of each channel in the RGB channel.

[0150] S502. Update the initial color gain of the view captured by each camera lens according to the calculated initial color gains to obtain the target color gain of the view captured by each camera lens.

[0151] Specifically, after calculating the color gain of the view captured by each lens relative to the view captured by its adjacent lens, the color gains can be updated to obtain the target color gain of the views captured by all lenses.

[0152] In an embodiment of the present application, the minimum color gain among the calculated initial color gains can be determined, and the difference between the initial color gain of the view captured by each camera lens and the minimum color gain can be calculated as the target color gain of the view captured by each camera lens.

[0153] Specifically, the minimum value among the initial color gains corresponding to the views captured by each lens can be determined as the minimum color gain. Then, for the view captured by each lens, the difference between the color gain corresponding to this view and the above minimum color gain is calculated as the target color gain of the view captured by this lens.

[0154] In addition, in an embodiment of the present application, the difference between the initial color gain of the view corresponding to each lens and a preset gain threshold can also be calculated as the target color gain of the view captured by each lens. The embodiments of the present application do not limit this.

[0155] S503, use each target color gain to perform color gain processing on the views captured by each objective lens respectively.

[0156] Specifically, for the view captured by each lens, the target color gain corresponding to this view can be used to perform color processing on this view to achieve color difference adjustment for this view.

[0157] In an embodiment of the present application, for the view captured by each objective lens, the following formula can be used to perform color gain processing on this view:

[0158] Pj = Py + (Zm * β * C4 + Py * (C5 - β) * Zm) / C6 / C7

[0159] Where, Pj represents the pixel value of the pixel point after color gain processing, Py represents the pixel value of the pixel point before color gain processing, Zm represents the target color gain, β represents the preset gain processing weight, and the value of this weight can be set manually or obtained based on experiments. The embodiments of the present application do not limit this;

[0160] C4 represents the fourth preset parameter, and the value of C4 can be 256, representing the maximum value of the pixel value;

[0161] C5 represents the fifth preset parameter, and the value of C5 can be 16, representing the color bit number;

[0162] C6 represents the sixth preset parameter, and the value of C6 can be 4096;

[0163] C7 represents the seventh preset parameter, and the value of C7 can be 1024, representing the number of bits occupied by the pixel value.

[0164] In an embodiment of the present application, for step S401 above when calculating the initial color gain, for the view captured by each objective lens, based on the pixel values of the view captured by this objective lens and the views captured by the adjacent lenses adjacent to this objective lens, determine the first color gain of the view captured by this objective lens relative to the views captured by the adjacent lenses, and determine the second color gain of the views captured by the adjacent lenses relative to the view captured by this objective lens, and calculate the difference between the first color gain and the second color gain as the initial color gain of the view captured by this objective lens.

[0165] Specifically, for each camera lens, the color gain of the view captured by this lens relative to the view captured by the adjacent lens can be calculated using the views captured by the above adjacent lenses and the RGB values of the pixel points in the view captured by this lens as the first color gain, and the color gain of the view captured by the adjacent lens relative to the view captured by this lens can be calculated as the second color gain. Then, the difference between the first color gain and the second color gain is calculated as the initial color gain of the view captured by this camera lens.

[0166] In an embodiment of the present application, for the view captured by each camera lens, the sum of the pixel values of the pixel points in the overlapping area of the view captured by this camera lens is statistically calculated, and the sum of the pixel values of the pixel points in the overlapping area of the view captured by the adjacent lens adjacent to this camera lens is statistically calculated. The color gain of the pixel points in the overlapping area of the less distinct view with a smaller sum of pixel values relative to the pixel points in the overlapping area of the distinct view with a larger sum of pixel values is determined, and the mathematical statistical value of each color gain is calculated as the first reference gain of the less distinct view. When the view captured by this camera lens is a less distinct view, the difference between the first reference gain and the second reference gain is calculated as the initial color gain. When the view captured by this camera lens is a distinct view, the difference between the second reference gain and the first reference gain is calculated as the initial color gain.

[0167] Among them, the second reference gain represents the color gain corresponding to the distinct view. Since the second reference gain is the color gain corresponding to the view with relatively distinct colors and the distinct view does not need to be gain-processed, the value of the second reference gain can be 0.

[0168] In an embodiment of the present application, the color gain S of the pixel points in the overlapping area of the less distinct view with a smaller sum of pixel values relative to the pixel points in the overlapping area of the distinct view with a larger sum of pixel values can be calculated according to the following formula:

[0169] S = (Px – Pf) / (α * C1 + (C2 - α) * Pf) * C3

[0170] Wherein, Px represents the distinct view, Pf represents the less distinct view, α represents a preset gain adjustment weight, C1 represents a first preset parameter, C2 represents a second preset parameter, and C3 represents a third preset parameter.

[0171] For example, assume that the first objective lens is the left lens. The first objective view captured by the first objective lens can be called the base image base_img, and the second objective lens is the right lens. The second objective view captured by the second objective lens can be called the registration image sub_img. Taking the pixel values of the R channel in the RGB channel as an example, the sum of the pixel values of the R channel of the pixel points in the overlapping area of base_img can be counted to obtain base_r_sum, and the sum of the pixel values of the R channel of the pixel points in the overlapping area of sub_img can be counted to obtain sub_r_sum.

[0172] If base_r_sum > sub_r_sum, it means that base_img is the distinct view. It can be determined that the color gain corresponding to sub_img is 0, and the color gain r_gain_tmp of each pixel point in the R channel of sub_img is calculated according to the following formula:

[0173] r_gain_tmp = (base_r – sub_r) / (α * 256 + (16 - α) * sub_r) * 4096

[0174] Where base_r represents the pixel value of the R channel of the pixel points in the overlapping area of base_img, and sub_r represents the pixel value of the R channel of the pixel points in the overlapping area of sub_img.

[0175] If sub_r_sum > base_r_sum, it means that sub_img is the distinct view. It can be determined that the color gain corresponding to base_img is 0, and the color gain r_gain_tmp of each pixel point in the R channel of base_img is calculated according to the following formula:

[0176] r_gain_tmp = (sub_r – base_r) / (α * 256 + (16 - α) * base_r) * 4096

[0177] In an embodiment of the present application, the color gain of the first target pixel point in the overlapping area of the non-distinct view with a smaller sum of pixel values relative to the second target pixel point in the overlapping area of the distinct view with a larger sum of pixel values can be determined.

[0178] Where the pixel position of the first target pixel point in the non-distinct view is the same as the pixel position of the second target pixel point in the distinct view, and the difference between the pixel value of the first target pixel point and the pixel value of the second target pixel point is less than the preset pixel difference threshold.

[0179] Specifically, when determining the color gain, the first target pixel point and the second target pixel point with relatively small pixel value differences in the corresponding pixel points can be determined first, and then the color gain can be determined based on the above first target pixel point and the second target pixel point.

[0180] See Figure 6 , Figure 6 FIG. is a schematic diagram of a process for obtaining the target color gain provided by an embodiment of the present application. Assume that the above multi-camera is a four-camera, and the four cameras from left to right are the 0th camera, the 1st camera, the 2nd camera, and the 3rd camera respectively. The views collected by the above four cameras are the 0th view, the 1st view, the 2nd view, and the 3rd view respectively. buffer[0], buffer[1], buffer[2], and buffer[3] respectively represent the color gains corresponding to the 0th view, the 1st view, the 2nd view, and the 3rd view. The initial color gain of the 0th view can be set to buffer[0]=0 in advance;

[0181] Then, based on the 0th view and the 1st view, the color gain of the 1st view relative to the 0th view is calculated as the initial color gain buffer[1];

[0182] Based on the 1st view and the 2nd view, the color gain of the 2nd view relative to the 1st view is calculated as the initial color gain buffer[2];

[0183] Based on the 2nd view and the 3rd view, the color gain of the 3rd view relative to the 2nd view is calculated as the initial color gain buffer[3];

[0184] Determine the minimum value min_diff in buffer[0], buffer[1], buffer[2], and buffer[3], and then subtract the above minimum value min_diff from the initial color gains buffer[0], buffer[1], buffer[2], and buffer[3] corresponding to each view to obtain the target color gains buffer[0], buffer[1], buffer[2], and buffer[3] corresponding to each view. The color gain processing is performed on each view by using the respective target color gains buffer[0], buffer[1], buffer[2], and buffer[3].

[0185] In an embodiment of the present application, for each camera, the historical color gain of the view collected by the camera can be used to perform weighted processing on the target color gain of the view collected by the camera, and the color gain processing is performed on the view collected by each camera by using the respective target color gains after weighted processing.

[0186] Specifically, the weighted target color gain Z can be calculated according to the following formula:

[0187] Z = S * (1 - q) + S' * q

[0188] Wherein, S represents the currently calculated target color gain, S' represents the historical color gain, and q represents a preset weight, and the value range of this weight can be (0, 1).

[0189] In an embodiment of the present application, when performing image stitching, the pixel points in the overlapping area can also be fused, and the following is a detailed introduction.

[0190] See Figure 7 , Figure 7 which is a schematic flowchart of a pixel fusion method provided by an embodiment of the present application. The method includes the following steps S701 - S704:

[0191] S701, calculate the first mean value of the Y - channel pixel values of the pixel points in the overlapping area of the first view to be stitched, and calculate the second mean value of the Y - channel pixel values of the pixel points in the overlapping area of the second target image.

[0192] Specifically, for the pixel values of the Y - channel of the pixel points, the mean value of the Y - channel pixel values of the pixel points in the overlapping area of the first view can be statistically calculated as the first mean value, and the mean value of the Y - channel pixel values of the pixel points in the overlapping area of the second target image can be statistically calculated as the second mean value.

[0193] In an embodiment of the present application, the first mean value and the second mean value can be calculated using a sliding window. See Figure 8 , Figure 8 which is a schematic diagram of calculating the mean value using a sliding window provided by an embodiment of the present application. The width of the sliding window can be the width of the overlapping area, the length of the sliding window can be a preset length, and the sliding window moves from top to bottom in the overlapping area. During the movement of the sliding window, the mean value of the pixel values of the Y - channel of the pixel points in the sliding window is statistically calculated. After the sliding window stops sliding, the above - mentioned mean values are further averaged to obtain the mean value of the Y - channel pixel values of the pixel points in the overlapping area. Among them, the length of the above - mentioned sliding window can be 32. In addition, it can also be 16, 64, etc. The embodiments of the present application do not limit this.

[0194] S702, calculate the pixel difference of the second mean value relative to the first mean value, and determine the difference step from the pixel value of the pixel point in the overlapping area of the first view to be stitched to the pixel value of the pixel point in the overlapping area of the second target image according to the pixel difference and the width of the overlapping area.

[0195] Specifically, the pixel difference between the second mean value and the first mean value can be calculated, the width of the overlapping region can be determined, and then the quotient of the above pixel difference and the above width can be calculated as the differential step.

[0196] S703. Using the differential step, correct the Y-channel pixel values of the pixel points in different columns in the overlapping region of the first view to be stitched and the second target image, and obtain the corrected first view to be stitched and the second target image.

[0197] Specifically, for the pixel values of the pixel points in different columns in the overlapping region, the pixel values of the pixel points in this column can be corrected based on the width value where the pixel points in this column are located and the above differential step, so as to obtain the corrected first view to be stitched and the second target image.

[0198] In an embodiment of the present application, when the first objective lens is on the left and the second objective lens is on the right,

[0199] The pixel values of the pixel points in the left region in the overlapping region of the stitched image are: the pixel values of the pixel points in the left region in the first view to be stitched;

[0200] The pixel values of the pixel points in the right region in the overlapping region of the stitched image are: the pixel values of the pixel points in the right region in the second target image;

[0201] The pixel values of the pixel points in the middle region in the overlapping region of the stitched image are: the pixel values obtained by weighting the pixel values of the pixel points in the middle region of the first view to be stitched and the second target image using the distance weight, and the distance weight is: the weight value determined based on the width distance of the pixel points relative to the first view to be stitched and the second view.

[0202] Specifically, when performing correction, the pixel values of the pixel points in the left region in the first view to be stitched can be selected as the pixel values of the pixel points in the left side of the overlapping region after stitching, and the pixel values of the pixel points in the right region in the second target image can be selected as the pixel values of the pixel points in the right side of the overlapping region after stitching.

[0203] For the pixel values of the pixel points in the middle region after stitching, the weight value corresponding to the pixel points in this column can be determined according to the width distance of the pixel points in different columns relative to the first view to be stitched and the second view. Based on the above weight value, the pixel values of the pixel points in this column in the first view to be stitched and the second target image are weighted and summed to obtain the pixel values of the pixel points in this column in the middle region after stitching.

[0204] For example, refer to Figure 9 , Figure 9It is a schematic diagram of a pixel fusion process provided by an embodiment of the present application. Assuming that the width of the middle area in the overlapping area is 16, the pixel values of the pixel points in the left area of the first view to be stitched can be used as the pixel values of the pixel points in the left area of the overlapping area of the stitched image;

[0205] The pixel values of the pixel points in the right area of the second target image are used as the pixel values of the pixel points in the right area of the overlapping area of the stitched image;

[0206] Using distance weights, the pixel values of the pixel points in the middle area of the first view to be stitched and the second target image are weighted to obtain the pixel values of the pixel points in the middle area of the overlapping area of the stitched image. Among them, the distance weight of the pixel points in the first view to be stitched can be i, and the distance weight of the pixel points in the second target image is 16 - i. i represents the order of the column where the pixel point is located relative to the middle area. For example, if the column where the pixel point is located is the first column on the left of the middle area, the value of i is 1; if the column where the pixel point is located is the fifth column on the left of the middle area, the value of i is 5.

[0207] S704. Fuse the overlapping areas in the corrected first view to be stitched and the second target image.

[0208] Specifically, based on the corrected first view to be stitched and the second target image, the pixel points in the overlapping area can be fused.

[0209] In an embodiment of the present application, for the pixel values of the pixel points in the UV channels in the overlapping area after stitching, the pixel values of the pixel points in the UV channels in the overlapping area of the first view to be stitched and the second target image can be directly weighted and fused, and the fused result is used as the pixel value of the pixel points in the UV channels in the overlapping area after stitching.

[0210] In the solution provided by the above embodiments, a first view to be stitched captured by a first objective lens can be obtained, and a second view to be stitched captured by a second objective lens can be obtained. Here, the first objective lens and the second objective lens are two adjacent lenses in the same multi-objective camera, and there is an overlapping area in the first view to be stitched and the second view to be stitched; perform mapping processing on each second pixel point in the second view to be stitched according to the second target mapping relationship to obtain a second target image; stitch the overlapping areas in the first view to be stitched and the second target image; among them, the second target mapping relationship is generated in the following manner: obtain a first objective view captured by the first objective lens, and obtain a second objective view captured by the second objective lens. Here, the first objective view and the second objective view are views captured by the multi-objective camera in the mapping relationship generation scenario, and there is an overlapping area in the first objective view and the second objective view; determine the matching pixel points in the first objective view and the second objective view to obtain a plurality of matching pairs. Each matching pair includes a first pixel point located in the overlapping area in the first objective view and a second pixel point located in the overlapping area in the second objective view; use the position correspondence relationship between the first pixel point and the second pixel point in each matching pair to determine the position conversion relationship between each pixel point in the second objective view and each pixel point in the first objective view, and obtain an initial mapping relationship for mapping the second objective view to the first objective view; calculate a compensation value for the initial mapping relationship according to the focal length, image width, camera spacing, sensor width, first distance, and second distance of the multi-objective camera. Here, the image width is the width of the image captured by the multi-objective camera, the camera spacing is the distance between the first objective lens and the second objective lens, the sensor width is the width of the sensor of each objective camera in the multi-objective camera, the first distance is the distance of the object in the overlapping area relative to the multi-objective camera, and the second distance is the distance of the target in the actual application scenario of the multi-objective camera relative to the multi-objective camera; use the calculated compensation value to compensate the initial mapping relationship to obtain the second target mapping relationship. In this way, in the actual application scenario, the above second target mapping relationship can be used to perform mapping processing on the pixel points in the second view to be stitched captured by the second objective lens. Since the above second target mapping relationship can reflect the position conversion relationship between each pixel point in the second objective view and each pixel point in the first objective view captured by the first objective lens, the pixel points in the mapped second view to be stitched correspond to the pixel points in the first view to be stitched captured by the first objective lens, and thus it is possible to avoid the pixel points in different views being misaligned during stitching. It can be seen that applying the solution provided by the above embodiments can improve the quality of the stitched image.

[0211] Correspondingly, an image stitching device is further provided in an embodiment of the present application, which will be introduced in detail below.

[0212] See Figure 10 , Figure 10Schematic structural diagram of an image stitching device provided by an embodiment of the present application. The device includes:

[0213] A to-be-stitched view acquisition module 1001, configured to acquire a first to-be-stitched view captured by a first objective lens and a second to-be-stitched view captured by a second objective lens. Wherein, the first objective lens and the second objective lens are two adjacent lenses in the same multi-objective camera, and there is an overlapping area in the first to-be-stitched view and the second to-be-stitched view;

[0214] A first mapping module 1002, configured to perform mapping processing on each second pixel point in the second to-be-stitched view according to a second target mapping relationship to obtain a second target image;

[0215] An image stitching module 1003, configured to stitch the overlapping areas in the first to-be-stitched view and the second target image;

[0216] Wherein, the second target mapping relationship is generated by the following modules:

[0217] A view acquisition module, configured to acquire a first objective view captured by the first objective lens and a second objective view captured by the second objective lens. Wherein, the first objective view and the second objective view are views captured by the multi-objective camera in a mapping relationship generation scenario, and there is an overlapping area in the first objective view and the second objective view;

[0218] A matching pair determination module, configured to determine matching pixel points in the first objective view and the second objective view to obtain a plurality of matching pairs. Wherein, each matching pair includes a first pixel point located in the overlapping area in the first objective view and a second pixel point located in the overlapping area in the second objective view;

[0219] An initial mapping relationship acquisition module, configured to use the position correspondence relationship between the first pixel point and the second pixel point in each matching pair to determine the position conversion relationship between each pixel point in the second objective view and each pixel point in the first objective view, and obtain an initial mapping relationship for mapping the second objective view to the first objective view;

[0220] A compensation value calculation module, configured to calculate a compensation value of the initial mapping relationship according to the focal length, image width, camera spacing, sensor width, first distance, and second distance of the multi-objective camera. Wherein, the image width is the width of the image captured by the multi-objective camera, the camera spacing is the spacing between the first objective lens and the second objective lens, the sensor width is the width of the sensor of each objective camera in the multi-objective camera, the first distance is the distance of an object in the overlapping area relative to the multi-objective camera, and the second distance is the distance of a target in the actual application scenario of the multi-objective camera relative to the multi-objective camera;

[0221] A second target acquisition module, configured to compensate the initial mapping relationship by using the calculated compensation value to obtain a second target mapping relationship.

[0222] In one embodiment of the present application, the compensation value calculation module is specifically configured to:

[0223] Calculate the compensation value deta_x of the initial mapping relationship according to the following formula:

[0224] deta_x = fbWk(1 / L1 – 1 / L2)

[0225] Wherein, the f represents the focal length of the multi-view camera, the W represents the image width, the b represents the camera spacing, the k represents the sensor width, the L1 represents the first distance, and the L2 represents the second distance.

[0226] In one embodiment of the present application, the initial mapping relationship acquisition module is specifically configured to:

[0227] Use the position correspondence relationship between the first pixel points and the second pixel points in each matching pair to determine the position conversion relationship between each pixel point in the second view and each pixel point in the first view;

[0228] Convert the positions of each pixel point in the second view according to the position conversion relationship to obtain a converted image, and crop the area outside the coordinate range of the second view in the converted image to obtain a first cropping conversion relationship for cropping the converted second view;

[0229] Fuse the position conversion relationship and the first cropping conversion relationship to obtain the initial mapping relationship between each pixel point in the second view and each pixel point in the cropped converted image;

[0230] The device further includes:

[0231] A first target acquisition module, configured to crop the first view according to the cropping method of the converted image to obtain the mapping relationship between each pixel point in the first view and each pixel point in the cropped first view as the first target mapping relationship.

[0232] In one embodiment of the present application, the matching pair determination module is specifically configured to:

[0233] Extract the features of each first pixel point located in the overlapping area in the first view, and extract the features of each second pixel point located in the overlapping area in the second view;

[0234] Calculate the similarity between the features of each first pixel point and the features of each second pixel point, and select the first pixel points and second pixel points whose similarity is greater than a preset similarity threshold to obtain multiple matching pairs;

[0235] Screen the obtained matching pairs to obtain matching pairs that meet the consistency.

[0236] In an embodiment of the present application, the matching pair determination module is specifically configured to:

[0237] Use different random sample consensus algorithms to screen the obtained matching pairs to obtain different screening results;

[0238] Fuse the obtained different screening results to obtain matching pairs that meet the consistency.

[0239] In an embodiment of the present application, the different random sample consensus algorithms include at least two of the following algorithms:

[0240] Random sample consensus algorithm based on similarity transformation, random sample consensus algorithm based on affine transformation, random sample consensus algorithm based on projective transformation.

[0241] In an embodiment of the present application, the initial mapping relationship obtaining module is specifically configured to:

[0242] Use a preset matrix transformation method to calculate the position transformation matrix from the second pixel point to the first pixel point in the obtained matching pairs, and obtain the position conversion relationship between each pixel point in the second view and each pixel point in the first view as the initial mapping relationship, where the matrix transformation method includes any one of the methods for obtaining the following matrices: similarity transformation matrix, affine transformation matrix, projective transformation matrix.

[0243] In an embodiment of the present application, the image stitching module 1003 is specifically configured to:

[0244] When the second target mapping relationship reflects the mapping relationship of pixel points between the second view and the cropped image, convert the first view to be stitched according to the first target mapping relationship to obtain a first target image, where the cropped image is: the image obtained by converting the second view to the first view and cropping the pixel points outside the coordinate range of the second view, and the first target mapping relationship is: the mapping relationship of pixel points between the first view and the cropped image;

[0245] Stitch the overlapping regions in the first target image and the second target image.

[0246] In one embodiment of the present application, when the number of lenses in the multi-camera is greater than or equal to 2, the device further includes:

[0247] An initial color gain calculation module, configured to calculate, for each view captured by each lens, an initial color gain of the view captured by the lens relative to the view captured by an adjacent lens adjacent to the lens, based on the pixel values of the view captured by the lens and the view captured by the adjacent lens adjacent to the lens, wherein the pixel value of each view includes: the components of the pixel points in the RGB three channels of the view;

[0248] A target color gain obtaining module, configured to update the initial color gain of the view captured by each lens according to the calculated initial color gains, to obtain the target color gain of the view captured by each lens;

[0249] A gain processing module, configured to perform color gain processing on the view captured by each lens respectively by using the respective target color gains.

[0250] In one embodiment of the present application, the initial color gain calculation module is specifically configured to:

[0251] For each view captured by each lens, based on the pixel values of the view captured by the lens and the view captured by the adjacent lens adjacent to the lens, determine a first color gain of the view captured by the lens relative to the view captured by the adjacent lens, and determine a second color gain of the view captured by the adjacent lens relative to the view captured by the lens, and calculate a difference between the first color gain and the second color gain as the initial color gain of the view captured by the lens.

[0252] In one embodiment of the present application, the initial color gain calculation module is specifically configured to:

[0253] For each view captured by each lens, count the sum of the pixel values of the pixel points in the overlapping area in the view captured by the lens, and count the sum of the pixel values of the pixel points in the overlapping area in the view captured by the adjacent lens adjacent to the lens, determine the color gain of the pixel points in the overlapping area in the less distinct view with a smaller sum of pixel values relative to the pixel points in the overlapping area in the more distinct view with a larger sum of pixel values, calculate a mathematical statistical value of each color gain as the first reference gain of the less distinct view, when the view captured by the lens is the less distinct view, calculate a difference between the first reference gain and a second reference gain as the initial color gain, when the view captured by the lens is the more distinct view, calculate a difference between the second reference gain and the first reference gain as the initial color gain, wherein the second reference gain represents: the color gain corresponding to the more distinct view.

[0254] In one embodiment of the present application, the initial color gain calculation module is specifically configured to:

[0255] Calculate the color gain S of the pixel points in the overlapping area of the non - vivid view with a smaller sum of pixel values relative to the pixel points in the overlapping area of the vivid view with a larger sum of pixel values according to the following formula:

[0256] S = (Px – Pf) / (α * C1+(C2 - α)*Pf)*C3

[0257] Wherein, Px represents the vivid view, Pf represents the non - vivid view, α represents a preset gain adjustment weight, C1 represents a first preset parameter, C2 represents a second preset parameter, and C3 represents a third preset parameter.

[0258] In one embodiment of the present application, the initial color gain calculation module is specifically configured to:

[0259] Determine the color gain of the first target pixel point in the overlapping area of the non - vivid view with a smaller sum of pixel values relative to the second target pixel point in the overlapping area of the vivid view with a larger sum of pixel values, wherein the pixel position of the first target pixel point in the non - vivid view is the same as the pixel position of the second target pixel point in the vivid view, and the difference between the pixel value of the first target pixel point and the pixel value of the second target pixel point is less than a preset pixel difference threshold.

[0260] In one embodiment of the present application, the target color gain calculation module is specifically configured to:

[0261] Determine the minimum color gain among the calculated initial color gains, and calculate the difference between the initial color gain of the view collected by each camera lens and the minimum color gain as the target color gain of the view collected by each camera lens.

[0262] In one embodiment of the present application, the gain processing module is specifically configured to:

[0263] For each camera lens, use the historical color gain of the view collected by this camera lens to perform weighted processing on the target color gain of the view collected by this camera lens, and use the weighted target color gains to perform color gain processing on the views collected by each camera lens respectively.

[0264] In one embodiment of the present application, the gain processing module is specifically configured to:

[0265] For the view collected by each camera lens, perform color gain processing on this view according to the following formula:

[0266] Pj = Py + (Zm * β * C4 + Py * (C5 - β) * Zm) / C6 / C7

[0267] Wherein, the Pj represents the pixel value of the pixel point after color gain processing, the Py represents the pixel value of the pixel point before color gain processing, the Zm represents the target color gain, the β represents the preset gain processing weight, the C4 represents the fourth preset parameter, the C5 represents the fifth preset parameter, the C6 represents the sixth preset parameter, and the C7 represents the seventh preset parameter.

[0268] In one embodiment of the present application, the device further includes:

[0269] A mean value calculation module, configured to calculate a first mean value of the Y-channel pixel values of the pixel points in the overlapping region of the first view to be stitched, and calculate a second mean value of the Y-channel pixel values of the pixel points in the overlapping region of the second target image;

[0270] A difference step determination module, configured to calculate the pixel difference of the second mean value relative to the first mean value, and determine the difference step from the pixel value of the pixel point in the overlapping region of the first view to be stitched to the pixel value of the pixel point in the overlapping region of the second target image according to the pixel difference and the width of the overlapping region;

[0271] A pixel correction module, configured to use the difference step to correct the Y-channel pixel values of the pixel points in different columns in the overlapping regions of the first view to be stitched and the second target image, and obtain the corrected first view to be stitched and the second target image;

[0272] A first fusion module, configured to fuse the overlapping regions in the corrected first view to be stitched and the second target image; and / or

[0273] A second fusion module, configured to perform weighted fusion on the pixel values of the UV channels of the pixel points in the overlapping regions of the first view to be stitched and the second target image.

[0274] In one embodiment of the present application, when the first objective lens is on the left and the second objective lens is on the right,

[0275] The pixel value of the pixel point in the left region of the overlapping region of the stitched image is: the pixel value of the pixel point in the left region of the first view to be stitched;

[0276] The pixel value of the pixel point in the right region of the overlapping region of the stitched image is: the pixel value of the pixel point in the right region of the second target image;

[0277] The pixel value of the pixel points in the middle area of the overlapping area of the spliced image is: the pixel value obtained by weighting the pixel values of the pixel points in the middle area of the first view to be spliced and the second target image using distance weights, where the distance weights are: weight values determined based on the width distances of the pixel points relative to the first view to be spliced and the second view.

[0278] In an embodiment of the present application, the view to be spliced obtaining module 1001 is specifically configured to:

[0279] Obtain the view collected by the first eye lens, and preprocess the obtained view based on the camera parameters of the first eye lens to obtain the first eye view, where the preprocessing includes: distortion correction processing and / or image projection processing, and the image projection processing is: processing of projecting the image onto a preset curved surface; and / or

[0280] Obtain the view collected by the second eye lens, and preprocess the obtained view based on the camera parameters of the second eye lens to obtain the second eye view.

[0281] In the solution provided by the above embodiments, a first view to be stitched captured by a first objective lens can be obtained, and a second view to be stitched captured by a second objective lens can be obtained. Herein, the first objective lens and the second objective lens are two adjacent lenses in the same multi-objective camera, and there is an overlapping area in the first view to be stitched and the second view to be stitched; perform mapping processing on each second pixel point in the second view to be stitched according to the second target mapping relationship to obtain a second target image; stitch the overlapping areas in the first view to be stitched and the second target image; wherein, the second target mapping relationship is generated by the following method: obtain a first objective view captured by the first objective lens, and obtain a second objective view captured by the second objective lens. Herein, the first objective view and the second objective view are views captured by the multi-objective camera in the mapping relationship generation scenario, and there is an overlapping area in the first objective view and the second objective view; determine the matching pixel points in the first objective view and the second objective view to obtain a plurality of matching pairs. Each matching pair includes a first pixel point located in the overlapping area in the first objective view and a second pixel point located in the overlapping area in the second objective view; use the position correspondence relationship between the first pixel point and the second pixel point in each matching pair to determine the position conversion relationship between each pixel point in the second objective view and each pixel point in the first objective view, and obtain an initial mapping relationship for mapping the second objective view to the first objective view; calculate a compensation value for the initial mapping relationship according to the focal length, image width, camera spacing, sensor width, first distance, and second distance of the multi-objective camera. Herein, the image width is the width of the image captured by the multi-objective camera, the camera spacing is the spacing between the first objective lens and the second objective lens, the sensor width is the width of the sensor of each objective camera in the multi-objective camera, the first distance is the distance of the object in the overlapping area relative to the multi-objective camera, and the second distance is the distance of the target in the actual application scenario of the multi-objective camera relative to the multi-objective camera; use the calculated compensation value to compensate the initial mapping relationship to obtain the second target mapping relationship. In this way, in the actual application scenario, the above second target mapping relationship can be used to perform mapping processing on the pixel points in the second view to be stitched captured by the second objective lens. Since the above second target mapping relationship can reflect the position conversion relationship between each pixel point in the second objective view and each pixel point in the first objective view captured by the first objective lens, the pixel points in the mapped second view to be stitched correspond to the pixel points in the first view to be stitched captured by the first objective lens, and thus the pixel points in different views can be prevented from being misaligned during stitching. It can be seen that applying the solution provided by the above embodiments can improve the quality of the stitched image.

[0282] An embodiment of the present application further provides an electronic device, as Figure 11 shown, including a processor 1101, a communication interface 1102, a memory 1103, and a communication bus 1104. Among them, the processor 1101, the communication interface 1102, and the memory 1103 communicate with each other through the communication bus 1104.

[0283] A memory 1103 for storing a computer program;

[0284] A processor 1101, when executing the program stored on the memory 1103, implements the above-mentioned image stitching method.

[0285] The communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, only a thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0286] The communication interface is used for communication between the above electronic device and other devices.

[0287] The memory may include a Random Access Memory (RAM), or may also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0288] The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0289] In another embodiment provided by the present application, a computer-readable storage medium is also provided. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the steps of any of the above-mentioned image stitching methods and / or the image stitching method.

[0290] In another embodiment provided by the present application, a computer program product including instructions is further provided. When it runs on a computer, it causes the computer to execute any one of the above-mentioned image stitching methods and / or the image stitching method.

[0291] In the solution provided by the above-mentioned embodiment, a first view to be stitched collected by a first objective lens can be obtained, and a second view to be stitched collected by a second objective lens can be obtained. Among them, the first objective lens and the second objective lens are two adjacent lenses in the same multi-objective camera, and there is an overlapping area in the first view to be stitched and the second view to be stitched; perform mapping processing on each second pixel point in the second view to be stitched according to the second target mapping relationship to obtain a second target image; stitch the overlapping areas in the first view to be stitched and the second target image; among them, the second target mapping relationship is generated in the following manner: obtain a first objective view collected by the first objective lens, and obtain a second objective view collected by the second objective lens. Among them, the first objective view and the second objective view are views collected by the multi-objective camera in the mapping relationship generation scenario, and there is an overlapping area in the first objective view and the second objective view; determine the matching pixel points in the first objective view and the second objective view to obtain a plurality of matching pairs, where each matching pair includes a first pixel point located in the overlapping area in the first objective view and a second pixel point located in the overlapping area in the second objective view; use the position correspondence relationship between the first pixel point and the second pixel point in each matching pair to determine the position conversion relationship between each pixel point in the second objective view and each pixel point in the first objective view, and obtain an initial mapping relationship for mapping the second objective view to the first objective view; calculate a compensation value for the initial mapping relationship according to the focal length, image width, camera spacing, sensor width, first distance, and second distance of the multi-objective camera, where the image width is: the width of the image collected by the multi-objective camera, the camera spacing is: the spacing between the first objective lens and the second objective lens, the sensor width is: the width of the sensor of each objective camera in the multi-objective camera, the first distance is: the distance of the object in the overlapping area relative to the multi-objective camera, and the second distance is: the distance of the target relative to the multi-objective camera in the actual application scenario of the multi-objective camera; use the calculated compensation value to compensate the initial mapping relationship to obtain the second target mapping relationship. In this way, in the actual application scenario, the pixel points in the second view to be stitched collected by the second objective lens can be mapped according to the above-mentioned second target mapping relationship. Since the above-mentioned second target mapping relationship can reflect the position conversion relationship between each pixel point in the second objective view and each pixel point in the first objective view collected by the first objective lens, the pixel points in the mapped second view to be stitched correspond to the pixel points in the first view to be stitched collected by the first objective lens, and thus it is possible to avoid the misalignment of pixel points in different views during stitching. It can be seen that applying the solution provided by the above-mentioned embodiment can improve the quality of the stitched image.

[0292] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more integrated available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0293] It should be noted that in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including", or any other variation thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or device that includes a series of elements includes not only those elements but also other elements that are not explicitly listed, or also includes elements that are inherent to such process, method, article, or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article, or device that includes the element.

[0294] Each embodiment in this specification is described in a related manner. The same or similar parts between the embodiments can be referred to each other, and the differences between each embodiment and other embodiments are emphasized. In particular, for the device embodiments, electronic device embodiments, computer-readable storage medium embodiments, and computer program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments.

[0295] The above are only the preferred embodiments of the present application and are not intended to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application are all included in the protection scope of the present application.

Claims

1. An image stitching method, characterized in that, The method includes: Obtaining a first view to be stitched captured by a first objective lens, and obtaining a second view to be stitched captured by a second objective lens, where the first objective lens and the second objective lens are two adjacent lenses in the same multi-objective camera, and there is an overlapping area in the first view to be stitched and the second view to be stitched; Performing mapping processing on each second pixel point in the second view to be stitched according to a second target mapping relationship to obtain a second target image; Stitching the overlapping areas in the first view to be stitched and the second target image; Among them, the second target mapping relationship is generated in the following manner: Obtaining a first objective view captured by the first objective lens, and obtaining a second objective view captured by the second objective lens, where the first objective view and the second objective view are views captured by the multi-objective camera in a mapping relationship generation scenario, and there is an overlapping area in the first objective view and the second objective view; Determining the matching pixel points in the first objective view and the second objective view to obtain a plurality of matching pairs, where each matching pair includes a first pixel point located in the overlapping area in the first objective view and a second pixel point located in the overlapping area in the second objective view; Using the position correspondence relationship between the first pixel point and the second pixel point in each matching pair, determining the position conversion relationship from each pixel point in the second objective view to each pixel point in the first objective view, and obtaining an initial mapping relationship for mapping the second objective view to the first objective view; Calculating a compensation value for the initial mapping relationship according to the focal length, image width, camera spacing, sensor width, first distance, and second distance of the multi-objective camera, where the image width is the width of the image captured by the multi-objective camera, the camera spacing is the spacing between the first objective lens and the second objective lens, the sensor width is the width of the sensor of each objective camera in the multi-objective camera, the first distance is the distance of the object in the overlapping area relative to the multi-objective camera, and the second distance is the distance of the target in the actual application scenario of the multi-objective camera relative to the multi-objective camera; Compensating the initial mapping relationship with the calculated compensation value to obtain a second target mapping relationship; The calculating the compensation value for the initial mapping relationship according to the focal length, image width, camera spacing, sensor width, first distance, and second distance of the multi-objective camera includes: Calculating the compensation value deta_x of the initial mapping relationship according to the following formula: deta_x = fbWk(1 / L1 – 1 / L2) Where f represents the focal length of the multi-objective camera, W represents the image width, b represents the camera spacing, k represents the sensor width, L1 represents the first distance, and L2 represents the second distance.

2. The method according to claim 1, wherein The determining the position conversion relationship from each pixel point in the second objective view to each pixel point in the first objective view by using the position correspondence relationship between the first pixel point and the second pixel point in each matching pair, and obtaining an initial mapping relationship for mapping the second objective view to the first objective view includes: Using the position correspondence between the first pixel points and the second pixel points in each matching pair, determine the position conversion relationship between each pixel point in the second target view and each pixel point in the first target view; According to the position conversion relationship, convert the positions of each pixel point in the second target view to obtain a converted image, and crop the area in the converted image that exceeds the coordinate range of the second target view to obtain a first cropping conversion relationship for cropping the converted second target view; Fuse the position conversion relationship and the first cropping conversion relationship to obtain an initial mapping relationship between each pixel point in the second target view and each pixel point in the cropped converted image; The method further includes: According to the cropping method of the converted image, crop the first target view to obtain a mapping relationship between each pixel point in the first target view and each pixel point in the cropped first target view as a first target mapping relationship.

3. The method according to claim 1, characterized in that, The determining of the matching pixel points in the first target view and the second target view to obtain a plurality of matching pairs includes: Extract the features of each first pixel point located in the overlapping area in the first target view, and extract the features of each second pixel point located in the overlapping area in the second target view; Calculate the similarity between the features of each first pixel point and the features of each second pixel point, and select the first pixel points and the second pixel points whose similarity is greater than a preset similarity threshold to obtain a plurality of matching pairs; Screen the obtained matching pairs to obtain matching pairs that meet the consistency; 4. The method according to claim 3, characterized in that The screening of the obtained matching pairs to obtain matching pairs that meet the consistency includes: Using different random sample consensus algorithms to screen the obtained matching pairs to obtain different screening results; Fuse the obtained different screening results to obtain matching pairs that meet the consistency; 5. The method according to claim 4, wherein The different random sample consensus algorithms include at least two of the following algorithms: Random sample consensus algorithm based on similarity transformation, random sample consensus algorithm based on affine transformation, random sample consensus algorithm based on projective transformation; 6. The method according to claim 1, characterized in that The using of the position correspondence between the first pixel points and the second pixel points in each matching pair to determine the position conversion relationship between each pixel point in the second target view and each pixel point in the first target view to obtain an initial mapping relationship for mapping the second target view to the first target view includes: Using a preset matrix transformation method, calculate the position transformation matrix from the second pixel points to the first pixel points in the obtained matching pairs to obtain the position conversion relationship between each pixel point in the second target view and each pixel point in the first target view as the initial mapping relationship, where the matrix transformation method includes any one of the methods for obtaining the following matrices: similarity transformation matrix, affine transformation matrix, projective transformation matrix; 7. The method according to claim 2, wherein The splicing of the overlapping area in the first view to be spliced and the second target image includes: When the second target mapping relationship reflects the mapping relationship of pixel points between the second target view and the cropped image, convert the first view to be stitched according to the first target mapping relationship to obtain a first target image, where the cropped image is an image obtained by converting the second target view to the first target view and cropping the pixel points outside the coordinate range of the second target view, and the first target mapping relationship is the mapping relationship of pixel points between the first target view and the cropped image; Stitch the overlapping regions in the first target image and the second target image.

8. The method according to claim 1, wherein The method further includes: For each view captured by each camera lens, calculate the initial color gain of the view captured by this camera lens relative to the view captured by the adjacent camera lens adjacent to this camera lens based on the pixel values of the view captured by this camera lens and the view captured by the adjacent camera lens adjacent to this camera lens, where the pixel value of each view includes the components of the pixel points in the RGB three channels of this view; Update the initial color gain of each view captured by each camera lens according to the calculated initial color gains to obtain the target color gain of each view captured by each camera lens; Use each target color gain to perform color gain processing on each view captured by each camera lens respectively.

9. The method according to claim 8, wherein The step of calculating the initial color gain of the view captured by each camera lens relative to the view captured by the adjacent camera lens adjacent to this camera lens based on the pixel values of the view captured by this camera lens and the view captured by the adjacent camera lens adjacent to this camera lens includes: For each view captured by each camera lens, determine the first color gain of the view captured by this camera lens relative to the view captured by the adjacent camera lens adjacent to this camera lens and determine the second color gain of the view captured by the adjacent camera lens adjacent to this camera lens relative to the view captured by this camera lens based on the pixel values of the view captured by this camera lens and the view captured by the adjacent camera lens adjacent to this camera lens, and calculate the difference between the first color gain and the second color gain as the initial color gain of the view captured by this camera lens.

10. The method according to claim 9, wherein The step of determining the first color gain of the view captured by each camera lens relative to the view captured by the adjacent camera lens adjacent to this camera lens and determining the second color gain of the view captured by the adjacent camera lens adjacent to this camera lens relative to the view captured by this camera lens based on the pixel values of the view captured by this camera lens and the view captured by the adjacent camera lens adjacent to this camera lens, and calculating the difference between the first color gain and the second color gain as the initial color gain of the view captured by this camera lens includes: For each view captured by each camera lens, calculate the sum of the pixel values of the pixel points in the overlapping area in the view captured by the camera lens, and calculate the sum of the pixel values of the pixel points in the overlapping area in the view captured by the adjacent camera lens adjacent to the camera lens. Determine the color gain of the pixel points in the overlapping area in the non-distinct view with the smaller sum of pixel values relative to the pixel points in the overlapping area in the distinct view with the larger sum of pixel values, calculate the mathematical statistical value of each color gain, and use it as the first reference gain of the non-distinct view. When the view captured by the camera lens is the non-distinct view, calculate the difference between the first reference gain and the second reference gain as the initial color gain. When the view captured by the camera lens is the distinct view, calculate the difference between the second reference gain and the first reference gain as the initial color gain, where the second reference gain represents the color gain corresponding to the distinct view.

11. The method according to claim 10, wherein The determination of the color gain of the pixel points in the overlapping area in the non-distinct view with the smaller sum of pixel values relative to the pixel points in the overlapping area in the distinct view with the larger sum of pixel values includes: Calculate the color gain S of the pixel points in the overlapping area in the non-distinct view with the smaller sum of pixel values relative to the pixel points in the overlapping area in the distinct view with the larger sum of pixel values according to the following formula: S = (Px – Pf) / (α * C1+(C2 - α) * Pf) * C3 Where, Px represents the distinct view, Pf represents the non-distinct view, α represents the preset gain adjustment weight, C1 represents the first preset parameter, C2 represents the second preset parameter, and C3 represents the third preset parameter.

12. The method according to claim 10, wherein The determination of the color gain of the pixel points in the overlapping area in the non-distinct view with the smaller sum of pixel values relative to the pixel points in the overlapping area in the distinct view with the larger sum of pixel values includes: Determine the color gain of the first target pixel point in the overlapping area of the non-distinct view with the smaller sum of pixel values relative to the second target pixel point in the overlapping area of the distinct view with the larger sum of pixel values, where the pixel position of the first target pixel point in the non-distinct view is the same as the pixel position of the second target pixel point in the distinct view, and the difference between the pixel value of the first target pixel point and the pixel value of the second target pixel point is less than the preset pixel difference threshold.

13. The method according to claim 8, wherein The update of the initial color gain of each view captured by each camera lens according to the calculated initial color gains to obtain the target color gain of each view captured by each camera lens includes: Determine the minimum color gain among the calculated initial color gains, and calculate the difference between the initial color gain of each view captured by each camera lens and the minimum color gain as the target color gain of each view captured by each camera lens.

14. The method according to claim 8, characterized in that The color gain processing of each view captured by each camera lens using the respective target color gains includes: For each objective lens, using the historical color gain of the view captured by the objective lens, weight the target color gain of the view captured by the objective lens, and use the weighted target color gains to perform color gain processing on the views captured by each objective lens respectively.

15. The method according to claim 8, wherein The performing color gain processing on the views captured by each objective lens respectively using the respective target color gains includes: For the view captured by each objective lens, perform color gain processing on the view using the following formula: Pj = Py + (Zm * β * C4 + Py * (C5 - β) * Zm) / C6 / C7 Wherein, Pj represents the pixel value of the pixel point after color gain processing, Py represents the pixel value of the pixel point before color gain processing, Zm represents the target color gain, β represents a preset gain processing weight, C4 represents a fourth preset parameter, C5 represents a fifth preset parameter, C6 represents a sixth preset parameter, and C7 represents a seventh preset parameter.

16. The method according to claim 1, wherein After the step of stitching the overlapping regions in the first view to be stitched and the second target image, the method further includes: Calculate a first average value of the Y-channel pixel values of the pixel points in the overlapping region of the first view to be stitched, and calculate a second average value of the Y-channel pixel values of the pixel points in the overlapping region of the second target image; Calculate the pixel difference of the second average value relative to the first average value, and determine the difference step from the pixel value of the pixel point in the overlapping region of the first view to be stitched to the pixel value of the pixel point in the overlapping region of the second target image according to the pixel difference and the width of the overlapping region; Use the difference step to correct the Y-channel pixel values of the pixel points in different columns in the overlapping regions of the first view to be stitched and the second target image, to obtain the corrected first view to be stitched and second target image; Fuse the overlapping regions in the corrected first view to be stitched and second target image; and / or Perform weighted fusion on the pixel values of the UV channels of the pixel points in the overlapping regions of the first view to be stitched and the second target image.

17. The method according to claim 16, wherein In the case where the first objective lens is on the left and the second objective lens is on the right, The pixel value of the pixel point in the left region in the overlapping region of the stitched image is: the pixel value of the pixel point in the left region in the first view to be stitched; The pixel value of the pixel point in the right region in the overlapping region of the stitched image is: the pixel value of the pixel point in the right region in the second target image; The pixel value of the pixel point in the middle region in the overlapping region of the stitched image is: the pixel value obtained by weighting the pixel values of the pixel points in the middle region in the first view to be stitched and the second target image using a distance weight, and the distance weight is: a weight value determined based on the width distance of the pixel point relative to the first view to be stitched and the second view.

18. The method according to any one of claims 1 to 17, characterized in that, The obtaining the first view of the first objective lens includes: Obtain the view captured by the first-eye lens, and preprocess the obtained view based on the camera parameters of the first-eye lens to obtain a first-eye view, where the preprocessing includes: distortion correction processing and / or image projection processing, and the image projection processing is: processing of projecting an image onto a preset curved surface; and / or The obtaining of the second-eye view captured by the second-eye lens includes: Obtain the view captured by the second-eye lens, and preprocess the obtained view based on the camera parameters of the second-eye lens to obtain a second-eye view.

19. An image stitching device, characterized in that, The device includes: A view-to-be-stitched obtaining module, configured to obtain a first view-to-be-stitched captured by a first-eye lens and a second view-to-be-stitched captured by a second-eye lens, where the first-eye lens and the second-eye lens are two adjacent lenses in the same multi-eye camera, and there is an overlapping area in the first view-to-be-stitched and the second view-to-be-stitched; A first mapping module, configured to perform mapping processing on each second pixel point in the second view-to-be-stitched according to a second target mapping relationship to obtain a second target image; An image stitching module, configured to stitch the overlapping areas in the first view-to-be-stitched and the second target image; Wherein, the second target mapping relationship is generated by the following modules: A view obtaining module, configured to obtain a first-eye view captured by the first-eye lens and a second-eye view captured by the second-eye lens, where the first-eye view and the second-eye view are views captured by the multi-eye camera in a mapping relationship generation scenario, and there is an overlapping area in the first-eye view and the second-eye view; A matching pair determination module, configured to determine the pixel points that match in the first-eye view and the second-eye view to obtain a plurality of matching pairs, where each matching pair includes a first pixel point located in the overlapping area in the first-eye view and a second pixel point located in the overlapping area in the second-eye view; An initial mapping relationship obtaining module, configured to use the position correspondence relationship between the first pixel point and the second pixel point in each matching pair to determine the position conversion relationship from each pixel point in the second-eye view to each pixel point in the first-eye view, and obtain an initial mapping relationship for mapping the second-eye view to the first-eye view; A compensation value calculation module, configured to calculate a compensation value for the initial mapping relationship according to the focal length, image width, camera spacing, sensor width, first distance, and second distance of the multi-eye camera, where the image width is: the width of the image captured by the multi-eye camera, the camera spacing is: the spacing between the first-eye lens and the second-eye lens, the sensor width is: the width of the sensor of each eye camera in the multi-eye camera, the first distance is: the distance of an object in the overlapping area relative to the multi-eye camera, and the second distance is: the distance of a target in the actual application scenario of the multi-eye camera relative to the multi-eye camera; A second target obtaining module, configured to compensate the initial mapping relationship by using the calculated compensation value to obtain a second target mapping relationship; Calculating a compensation value for the initial mapping relationship according to the focal length, image width, camera spacing, sensor width, first distance, and second distance of the multi-view camera, includes: Calculating the compensation value deta_x for the initial mapping relationship according to the following formula: deta_x = fbWk(1 / L1 – 1 / L2) Wherein, f represents the focal length of the multi-view camera, W represents the image width, b represents the camera spacing, k represents the sensor width, L1 represents the first distance, and L2 represents the second distance.

20. An electronic device, characterized in that, Including a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the communication bus; The memory is used to store computer programs; The processor is configured to implement the method steps described in any one of claims 1-18 when executing the programs stored on the memory.

21. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the method steps described in any one of claims 1-18 are implemented.

Images