Binocular image processing method, device and equipment and storage medium

Abstract

The application belongs to the technical field of image processing, and discloses a binocular image processing method, device, equipment and storage medium. The method comprises the following steps: image registration is performed on a first binocular image and a second binocular image, an overlapping area, a first non-overlapping area and a second non-overlapping area between the first binocular image and the second binocular image are determined, a mean image parameter of the overlapping area is calculated, a reference image is determined, the dependence on the reference image is reduced, the risk of poor quality of a spliced image introduced by the reference image when the overlapping area of the binocular image is too different is avoided, contrast correction is performed on the first non-overlapping area and the second non-overlapping area, binocular splicing is performed based on the reference image, a first corrected image and a second corrected image, the similarity of the structure of the overlapping area is strengthened, and the occurrence of unnatural transition when the contrast of the non-overlapping area is too different is avoided, and the experience of a user is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a binocular image processing method, device, equipment and storage medium. BACKGROUND

[0002] In the field of VR technology, two RGB camera modules respectively capture two images at different angles. In the process of data transmission, the two images need to be spliced into one image. In the image splicing method, brightness correction and color correction are two very key steps. Most of the current methods for overlapping areas are to select one image as a reference, and adjust the brightness and color of the other image to the level of the reference image through a mapping matrix, and there is a lack of processing of the non-overlapping areas on both sides. This results in that the spliced image is very dependent on the reference image in the overlapping area, and there is a significant difference in contrast between the non-overlapping areas on both sides of the image, which seriously affects the user experience. SUMMARY

[0003] The main purpose of the present application is to provide a binocular image processing method, device, equipment and storage medium, aiming at solving the technical problem of how to reduce the dependence on the reference image while improving the coordination of the spliced image when splicing binocular images.

[0004] To achieve the above-mentioned purpose, the present application provides a binocular image processing method, which comprises:

[0005] image registration is performed on the first binocular image and the second binocular image collected by the camera, the overlapping area between the first binocular image and the second binocular image, the first non-overlapping area in the first binocular image and the second non-overlapping area in the second binocular image are determined, and the first binocular image and the second binocular image are a group of binocular images;

[0006] The mean image parameters of the overlapping area are calculated according to the image parameters of the first binocular image and the image parameters of the second binocular image, and a reference image is determined;

[0007] The first non-overlapping area and the second non-overlapping area are respectively subjected to contrast correction, and a first corrected image corresponding to the first non-overlapping area and a second corrected image corresponding to the second non-overlapping area are obtained;

[0008] Binocular splicing is performed according to the reference image, the first corrected image and the second corrected image, and a binocular splicing image of the first binocular image and the second binocular image is obtained.

[0009] Optionally, before the image registration is performed on the first binocular image and the second binocular image collected by the camera, the present application further comprises:

[0010] perform image denoising on the first initial image and the second initial image collected by the camera respectively to obtain a first denoised image corresponding to the first initial image and a second denoised image corresponding to the second initial image;

[0011] perform coordinate conversion on the first denoised image and the second denoised image according to a spherical coordinate system to obtain a first converted image corresponding to the first denoised image and a second converted image corresponding to the second denoised image;

[0012] perform geometric correction on the first converted image and the second converted image respectively to obtain a first binocular image corresponding to the first converted image and a second binocular image corresponding to the second converted image.

[0013] Optionally, the first binocular image and the second binocular image collected by the camera are image-registered to determine an overlapping area between the first binocular image and the second binocular image, a first non-overlapping area in the first binocular image, and a second non-overlapping area in the second binocular image, the first binocular image and the second binocular image being a group of binocular images, comprising:

[0014] perform feature point extraction on the first binocular image and the second binocular image collected by the camera respectively to determine a plurality of first feature points in the first binocular image and a plurality of second feature points in the second binocular image;

[0015] perform feature matching on each first feature point and each second feature point to obtain a plurality of similar feature points in the plurality of first feature points and the plurality of second feature points;

[0016] determine the overlapping area between the first binocular image and the second binocular image according to each feature similar point;

[0017] determine a first non-overlapping area in the first binocular image and a second non-overlapping area in the second binocular image according to the overlapping area between the first binocular image and the second binocular image.

[0018] Optionally, the mean image parameter of the overlapping area is calculated according to the image parameter of the first binocular image and the image parameter of the second binocular image to determine a reference image, comprising:

[0019] perform color space conversion on the first binocular image and the second binocular image respectively to obtain a first processed image corresponding to the first binocular image and a second processed image corresponding to the second binocular image;

[0020] obtain a first brightness value and a first color value of the overlapping area in the first processed image;

[0021] acquiring a second luminance value and a second color value of the overlapping area in the second processed image;

[0022] calculating a mean image parameter of the overlapping area according to the first luminance value, the first color value, the second luminance value and the second color value, and determining a reference image.

[0023] Optionally, the mean image parameter comprises a color mean value and a luminance mean value.

[0024] The calculating a mean image parameter of the overlapping area according to the first luminance value, the first color value, the second luminance value and the second color value, and determining a reference image comprises:

[0025] performing mean calculation on the first luminance value and the second luminance value to determine a luminance mean value of the overlapping area;

[0026] performing mean calculation on the first color value and the second color value to determine a color mean value of the overlapping area;

[0027] constructing a reference image corresponding to the overlapping area according to the color mean value and the luminance mean value.

[0028] Optionally, the binocular splicing according to the reference image, the first corrected image and the second corrected image to obtain a binocular splicing image of the first binocular image and the second binocular image comprises:

[0029] mapping the overlapping area of the first binocular image and the overlapping area of the second binocular image in color channel and luminance channel respectively according to the reference image to obtain a first coincident area corresponding to the overlapping area of the first binocular image and a second coincident area corresponding to the overlapping area of the second binocular image;

[0030] determining an optimal seam according to the first coincident area and the second coincident area;

[0031] performing binocular splicing on the first coincident area, the second coincident area, the first corrected image and the second corrected image according to the optimal seam to obtain a binocular splicing image of the first binocular image and the second binocular image.

[0032] Optionally, the determining an optimal seam according to the first coincident area and the second coincident area comprises:

[0033] determining a plurality of seams according to the first coincident area and the second coincident area;

[0034] acquiring an image evaluation parameter of each seam;

[0035] Determine the stitching evaluation value of each suture line according to the image evaluation parameter of each suture line and the preset evaluation energy function;

[0036] Determine the optimal suture line in each suture line according to the stitching evaluation value of each suture line.

[0037] In addition, to achieve the above object, the application further provides a binocular image processing device, which comprises:

[0038] A registration module is configured to perform image registration on a first binocular image and a second binocular image collected by a camera, determine an overlapping area between the first binocular image and the second binocular image, a first non-overlapping area in the first binocular image, and a second non-overlapping area in the second binocular image, wherein the first binocular image and the second binocular image are a group of binocular images.

[0039] A calculation module is configured to calculate a mean image parameter of the overlapping area according to an image parameter of the first binocular image and an image parameter of the second binocular image, and determine a reference image.

[0040] A correction module is configured to perform contrast correction on the first non-overlapping area and the second non-overlapping area respectively, and obtain a first corrected image corresponding to the first non-overlapping area and a second corrected image corresponding to the second non-overlapping area.

[0041] A splicing module is configured to perform binocular splicing according to the reference image, the first corrected image and the second corrected image, and obtain a binocular splicing image of the first binocular image and the second binocular image.

[0042] In addition, to achieve the above object, the application further provides a binocular image processing device, which comprises a memory, a processor and a binocular image processing program stored in the memory and executable on the processor, wherein the binocular image processing program is configured to implement the binocular image processing method as described above.

[0043] In addition, to achieve the above object, the application further provides a storage medium, wherein the storage medium stores a binocular image processing program, and the binocular image processing program is executed by a processor to implement the binocular image processing method as described above.

[0044] This invention performs image registration on a first and a second binocular image captured by a camera to determine the overlapping region between the first and second binocular images, a first non-overlapping region in the first binocular image, and a second non-overlapping region in the second binocular image. The first and second binocular images form a set of binocular images. The mean image parameters of the overlapping region are calculated based on the image parameters of the first and second binocular images to determine a reference image. Contrast correction is performed on the first and second non-overlapping regions respectively to obtain a first corrected image corresponding to the first non-overlapping region and a second corrected image corresponding to the second non-overlapping region. Finally, binocular stitching is performed based on the reference image, the first corrected image, and the second corrected image to obtain a stitched binocular image of the first and second binocular images. By employing the above method, image registration is performed on the first and second binocular images to determine the overlapping, first, and second non-overlapping regions between them. The mean image parameters of the overlapping regions are calculated to determine the reference image, reducing dependence on the reference image and avoiding the risk of poor stitched image quality introduced by the reference image when the overlapping regions of the binocular images differ significantly. Contrast correction is performed on the first and second non-overlapping regions. Binocular stitching is then performed based on the reference image, the first corrected image, and the second corrected image, strengthening the similarity of the overlapping region structure and avoiding unnatural transitions when the contrast difference in the non-overlapping regions is too large. This improves the coordination of the stitched binocular images and enhances the user experience. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the structure of a binocular image processing device in the hardware operating environment involved in the embodiments of the present invention;

[0046] Figure 2 This is a flowchart illustrating the first embodiment of the binocular image processing method of the present invention;

[0047] Figure 3 This is a schematic diagram of a region according to an embodiment of the binocular image processing method of the present invention;

[0048] Figure 4 This is a flowchart illustrating the second embodiment of the binocular image processing method of the present invention;

[0049] Figure 5 This is a stitching diagram of an embodiment of the binocular image processing method of the present invention;

[0050] Figure 6 This is a structural block diagram of the first embodiment of the binocular image processing device of the present invention.

[0051] The objectives, functional characteristics and advantages of the present application will be further illustrated in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0052] It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

[0053] Reference Figure 1 , Figure 1 The structural schematic diagram of a binocular image processing device related to the hardware running environment of the embodiment of the present application.

[0054] As Figure 1 shown, the binocular image processing device can include a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 can include a display, an input unit such as a keyboard, and can also include a standard wired interface, a wireless interface. The network interface 1004 can optionally include a standard wired interface, a wireless interface (such as a wireless fidelity (Wi-Fi) interface). The memory 1005 can be a high-speed random access memory (RAM) memory, or a stable non-volatile memory (NVM), such as a disk memory. The memory 1005 can also be a storage device independent of the aforementioned processor 1001.

[0055] Those skilled in the art can understand Figure 1 that the structure shown in the figure does not constitute a limitation on the binocular image processing device, and can include more or fewer components than the figure, or combine certain components, or different component arrangements.

[0056] As Figure 1 shown, the memory 1005 as a storage medium can include an operating system, a network communication module, a user interface module, and a binocular image processing program.

[0057] In Figure 1The network interface 1004 is mainly used for data communication with a network server, and the user interface 1003 is mainly used for data interaction with a user. The processor 1001 and the memory 1005 in the binocular image processing device can be arranged in the binocular image processing device, the binocular image processing device calls the binocular image processing program stored in the memory 1005 through the processor 1001, and executes the binocular image processing method provided in the embodiment of the application.

[0058] The embodiment of the application provides a binocular image processing method, which refers to Figure 2 , Figure 2 FIG. 1 is a flowchart of a first embodiment of the binocular image processing method.

[0059] The binocular image processing method comprises the following steps:

[0060] Step S10: performing image registration on the first binocular image and the second binocular image collected by the camera, determining an overlapping area between the first binocular image and the second binocular image, a first non-overlapping area in the first binocular image, and a second non-overlapping area in the second binocular image, the first binocular image and the second binocular image being a group of binocular images.

[0061] It should be noted that the execution subject of the embodiment is a binocular image processing device, wherein the binocular image processing device has functions of data processing, data communication and program running, and can be an integrated controller, a control computer or other devices with similar functions, and the embodiment is not limited thereto.

[0062] It can be understood that the first binocular image and the second binocular image refer to a group of binocular images that need to be spliced, and are two images captured by two RGB cameras at different angles.

[0063] In the specific implementation, the first binocular image and the second binocular image are subjected to image registration, specifically, the overlapping area between the first binocular image and the second binocular image is identified, and then the non-overlapping area in the first binocular image except the overlapping area and the non-overlapping area in the second binocular image except the overlapping area are distinguished based on the overlapping area, the non-overlapping area in the first binocular image except the overlapping area is the first non-overlapping area, and the non-overlapping area in the second binocular image except the overlapping area is the second non-overlapping area.

[0064] It should be noted that in order to ensure the accuracy of subsequent image stitching of the first binocular image and the second binocular image, image preprocessing needs to be performed in advance. Further, before the image registration of the first binocular image and the second binocular image collected by the camera, it further includes: performing image denoising on the first initial image and the second initial image collected by the camera respectively to obtain a first denoised image corresponding to the first initial image and a second denoised image corresponding to the second initial image; performing coordinate conversion on the first denoised image and the second denoised image according to a spherical coordinate system to obtain a first converted image corresponding to the first denoised image and a second converted image corresponding to the second denoised image; and performing geometric correction on the first converted image and the second converted image respectively to obtain a first binocular image corresponding to the first converted image and a second binocular image corresponding to the second converted image.

[0065] It can be understood that the first initial image is the first binocular image before image preprocessing, that is, the image obtained after the first initial image is subjected to image preprocessing is the first binocular image, and similarly, the second initial image is the second binocular image before image preprocessing.

[0066] In a specific implementation, the image preprocessing includes but is not limited to the processes of image denoising, coordinate conversion and geometric correction. The first initial image and the second initial image are subjected to image denoising, and the specific process of denoising is as follows: wavelet transform is performed on an image I(x, y) to obtain G(w), high-frequency noise components are removed through a low-pass filter to obtain G1(w), and then inverse wavelet transform is performed to obtain a noise-removed image. The first initial image after noise removal is the first denoised image, and the second initial image after noise removal is the second denoised image.

[0067] It should be noted that in order to ensure the visual consistency after image stitching, the first denoised image and the second denoised image need to be converted from two-dimensional coordinates to other unified coordinate systems. In this embodiment, the spherical coordinate system is used to convert the coordinates of the first denoised image and the second denoised image, and the first denoised image and the second denoised image are projected into the spherical coordinate system, (x, y, z) is the coordinate in the two-dimensional plane, is the spherical coordinate, and r is the radius of the spherical coordinate. The first denoised image after coordinate conversion is the first converted image, and the second denoised image after coordinate conversion is the second converted image.

[0068] It can be understood that after the coordinate conversion is completed, the first converted image and the second converted image are geometrically corrected, and the specific process is that the control position of the image is transformed and the pixel gray value is recalculated to correct the stretching and distortion of the image. The first converted image after geometric correction is the first binocular image, and the second converted image after geometric correction is the second binocular image.

[0069] In a specific implementation, in order to ensure the accuracy of image registration, further, the image registration of the first binocular image and the second binocular image collected by the camera is performed to determine the overlapping area between the first binocular image and the second binocular image, the first non-overlapping area in the first binocular image, and the second non-overlapping area in the second binocular image, the first binocular image and the second binocular image being a group of binocular images, including: performing feature point extraction on the first binocular image and the second binocular image collected by the camera respectively to determine a plurality of first feature points in the first binocular image and a plurality of second feature points in the second binocular image; performing feature matching on each first feature point and each second feature point to obtain a plurality of similar feature points in the plurality of first feature points and the plurality of second feature points; determining the overlapping area between the first binocular image and the second binocular image according to each feature similar point; determining the first non-overlapping area in the first binocular image and the second non-overlapping area in the second binocular image according to the overlapping area between the first binocular image and the second binocular image.

[0070] It should be noted that a plurality of feature points in the first binocular image and the second binocular image are extracted respectively, the plurality of feature points in the first binocular image are the first feature points, the plurality of feature points in the second binocular image are the second feature points, each first feature point is sequentially matched with each second feature point to determine similar feature points in each first feature point and second feature point, the similar feature points in the first feature points and the second feature points are the similar feature points, so as to determine the overlapping area between the first binocular image and the second binocular image and align them in space. The non-overlapping area of the first binocular image except the overlapping area is the first non-overlapping area, and the non-overlapping area of the second binocular image except the overlapping area is the second non-overlapping area. As shown in Figure 3

[0071] It can be understood that when the image registration is performed, the feature points of the first binocular image can also be extracted first, and then the corresponding matching points are searched in the second binocular image, so as to determine the overlapping area of the two images and align them in space.

[0072] ​Step S20: calculating mean image parameters of the overlapping area according to the image parameters of the first binocular image and the image parameters of the second binocular image, and determining a reference image.

[0073] It should be noted that the reference image refers to an image used for mapping of the overlapping area, and the mean image parameters include color mean and brightness mean. In order to ensure the accuracy of the mean image parameter calculation, further, the step of calculating the mean image parameters of the overlapping area according to the image parameters of the first binocular image and the image parameters of the second binocular image, and determining a reference image, comprises: respectively performing color space conversion on the first binocular image and the second binocular image to obtain a first processed image corresponding to the first binocular image and a second processed image corresponding to the second binocular image; obtaining a first brightness value and a first color value of the overlapping area in the first processed image; obtaining a second brightness value and a second color value of the overlapping area in the second processed image; calculating the mean image parameters of the overlapping area according to the first brightness value, the first color value, the second brightness value and the second color value, and determining a reference image.

[0074] It can be understood that the first binocular image and the second binocular image are respectively converted from RGB space to LAB space, and the first binocular image after color space conversion is the first processed image, and the second binocular image after color space conversion is the second processed image. The brightness value and the color value of the overlapping area in the L channel of the first processed image are obtained, and the brightness value of the overlapping area in the first processed image is the first brightness value, and the color value of the overlapping area in the processed image is the first color value. Similarly, the second brightness value and the second color value of the second image are obtained.

[0075] In a specific implementation, in order to obtain an accurate reference image according to the first brightness value, the first color value, the second brightness value and the second color value, further, the mean image parameters include color mean and brightness mean; the step of calculating the mean image parameters of the overlapping area according to the first brightness value, the first color value, the second brightness value and the second color value, and determining a reference image, comprises: performing mean calculation on the first brightness value and the second brightness value to determine the brightness mean of the overlapping area; performing mean calculation on the first color value and the second color value to determine the color mean of the overlapping area; and constructing the reference image corresponding to the overlapping area according to the color mean and the brightness mean.

[0076] It should be noted that the brightness mean of the overlapping area in the L channel is calculated as , wherein, is the first brightness value, is the second brightness value. The color mean of the overlapping area is calculated as , wherein, is a first color value, is a second color value, and the average brightness value is taken as the brightness value of the reference image and the average color value is taken as the brightness value of the reference image, thereby forming a reference image corresponding to the overlapping area. This ensures consistency of the double RGB images in terms of color and brightness, while also reducing the risk of poor quality of the spliced image due to poor quality of the reference image.

[0077] Step S30: performing contrast correction on the first non-overlapping area and the second non-overlapping area respectively to obtain a first corrected image corresponding to the first non-overlapping area and a second corrected image corresponding to the second non-overlapping area.

[0078] It should be noted that the histogram equalization is performed on the first non-overlapping area and the second non-overlapping area respectively, thereby realizing the contrast correction of the first non-overlapping area and the second non-overlapping area. The corrected first non-overlapping area is the first corrected image, and the corrected second non-overlapping area is the second corrected image.

[0079] Step S40: performing binocular splicing according to the reference image, the first corrected image and the second corrected image to obtain a binocular splicing image of the first binocular image and the second binocular image.

[0080] It should be noted that the color and brightness of the overlapping area between the first binocular image and the second binocular image are adjusted under the reference image. The adjusted overlapping area, the first corrected image and the second corrected image are spliced to obtain the completed image, which is the binocular splicing image.

[0081] The embodiment determines the overlapping area between the first binocular image and the second binocular image, the first non-overlapping area in the first binocular image and the second non-overlapping area in the second binocular image by image registration on the first binocular image and the second binocular image collected by the camera, the first binocular image and the second binocular image being a group of binocular images; calculates the mean image parameter of the overlapping area according to the image parameter of the first binocular image and the image parameter of the second binocular image, and determines the reference image; performs contrast correction on the first non-overlapping area and the second non-overlapping area respectively to obtain the first corrected image corresponding to the first non-overlapping area and the second corrected image corresponding to the second non-overlapping area; and performs binocular splicing according to the reference image, the first corrected image and the second corrected image to obtain the binocular splicing image of the first binocular image and the second binocular image. In the above manner, the first binocular image and the second binocular image are registered, the overlapping area between the first binocular image and the second binocular image, the first non-overlapping area and the second non-overlapping area are determined, the mean image parameter of the overlapping area is calculated, the reference image is determined, the dependence on the reference image is reduced, the risk of poor splicing image quality introduced by the reference image when the overlapping area of the binocular image is too different is avoided, the first non-overlapping area and the second non-overlapping area are contrast corrected, the binocular splicing is performed based on the reference image, the first corrected image and the second corrected image, the similarity of the structure of the overlapping area is strengthened, the unnatural transition of the non-overlapping area when the contrast is too different is avoided, the coordination of the binocular splicing image is improved, and the user experience is improved.

[0082] Reference Figure 4 , Figure 4 The flowchart of the second embodiment of the binocular image processing method is shown.

[0083] Based on the above first embodiment, in the binocular image processing method, the step S40 comprises:

[0084] Step S41: mapping the overlapping area of the first binocular image and the overlapping area of the second binocular image in the color channel and the brightness channel respectively according to the reference image to obtain the first coincident area corresponding to the overlapping area of the first binocular image and the second coincident area corresponding to the overlapping area of the second binocular image.

[0085] It should be noted that the brightness and color of the overlapping area between the first binocular image and the first binocular image are mapped to the brightness value and color value of the reference image, the overlapping area in the mapped first binocular image is the first coincident area, and the overlapping area in the mapped second binocular image is the second coincident area.

[0086] It can be understood that the specific mapping process is: , wherein M1 and M2 are mapping matrices respectively, is an image of the overlapping region after color channel mapping with the reference image as a standard, represents the overlapping region without color mapping, is an image of the overlapping region after brightness channel mapping with the reference image as a standard, represents the overlapping region without brightness mapping. The specific solving process of the mapping matrices M1 and M2 is as follows: N pairs of pixel points are selected from the overlapping region without mapping and the reference image, and according to the brightness values and color values of the N pairs of pixel points, the mapping matrix M1 of the color channel and the mapping matrix M2 of the brightness channel can be obtained.

[0087] Step S42: determining an optimal stitching line according to the first overlapping region and the second overlapping region.

[0088] It should be noted that the optimal stitching line refers to the optimal cutting line for image cutting of the first overlapping region and the second overlapping region when the first binocular image and the second binocular image are spliced. The optimal stitching line can be a straight line or an irregular curve, which is not limited in the embodiment. The optimal stitching line can ensure that the difference between the left and right ends of the spliced image is the smallest when the image is spliced.

[0089] It can be understood that, in order to accurately select the optimal stitching line, further, the determining of the optimal stitching line according to the first overlapping region and the second overlapping region comprises: determining a plurality of stitching lines according to the first overlapping region and the second overlapping region; obtaining image evaluation parameters of each stitching line; determining a stitching evaluation value of each stitching line according to the image evaluation parameters of each stitching line and a preset evaluation energy function; and determining the optimal stitching line in each stitching line according to the stitching evaluation value of each stitching line.

[0090] In a specific implementation, a plurality of stitching lines are selected in the first overlapping region and the second overlapping region, and image evaluation parameters of each stitching line are obtained. In the embodiment, the color value and the brightness value of the left and right ends of each stitching line are used as the evaluation parameters, and other ways can also be used, which are not limited in the embodiment.

[0091] It should be noted that the preset evaluation energy function refers to an energy function preset for calculating the stitching evaluation value of each stitching line according to the image evaluation parameters. The lower the stitching evaluation value is, the smaller the difference between the left and right ends when the image is spliced. The stitching line with the smallest stitching evaluation value is selected as the optimal stitching line according to the stitching evaluation value of each stitching line.

[0092] Step S43: binocular stitching the first coincident region, the second coincident region, the first corrected image and the second corrected image according to the optimal seam line, to obtain a binocular stitched image of the first binocular image and the second binocular image.

[0093] It should be noted that after the optimal seam line is determined, the first coincident region and the second coincident region are cut based on the optimal seam line, and binocular stitching is performed based on the cut first coincident region, the cut second coincident region, the first corrected image and the second corrected image, to obtain a binocular stitched image that is completed. Figure 5 As shown in FIG. 6, the overlapping region A is the first coincident region, the non-overlapping region B is the first corrected image, the overlapping region C is the second coincident region, the non-overlapping region D is the second corrected image, the optimal seam line is l, the right side of the overlapping region A is cut based on the optimal seam line l, the left side of the overlapping region A is retained, the left side of the overlapping region C is cut, and the right side of the overlapping region C is retained, to finally obtain the binocular stitched image E.

[0094] In this embodiment, the overlapping region of the first binocular image and the overlapping region of the second binocular image are respectively mapped in the color channel and the brightness channel according to the reference image, to obtain the first coincident region corresponding to the overlapping region of the first binocular image and the second coincident region corresponding to the overlapping region of the second binocular image; the optimal seam line is determined according to the first coincident region and the second coincident region; and the first coincident region, the second coincident region, the first corrected image and the second corrected image are binocular stitched according to the optimal seam line, to obtain a binocular stitched image of the first binocular image and the second binocular image. In the above manner, the color and brightness are mapped based on the reference image, to obtain the first coincident region and the second coincident region after mapping, which ensures the consistency of the binocular RGB images in color and brightness, reduces the risk of poor quality of the stitched image caused by poor quality of the reference image, and finally obtains the binocular stitched image by stitching the images based on the optimal seam line, thereby reducing the difference between the left and right of the binocular stitched image and improving the user experience.

[0095] In addition, with reference to Figure 6 , the embodiment of the present application further provides a binocular image processing device, which comprises:

[0096] The registration module 10 is configured to perform image registration on the first binocular image and the second binocular image collected by the camera, to determine an overlapping region between the first binocular image and the second binocular image, a first non-overlapping region in the first binocular image and a second non-overlapping region in the second binocular image, and the first binocular image and the second binocular image are a group of binocular images.

[0097] a calculating module 20, configured to calculate mean image parameters of the overlapping region according to the image parameters of the first binocular image and the image parameters of the second binocular image, and determine a reference image.

[0098] a correcting module 30, configured to respectively perform contrast correction on the first non-overlapping region and the second non-overlapping region, to obtain a first corrected image corresponding to the first non-overlapping region and a second corrected image corresponding to the second non-overlapping region.

[0099] a splicing module 40, configured to perform binocular splicing according to the reference image, the first corrected image and the second corrected image, to obtain a binocular splicing image of the first binocular image and the second binocular image.

[0100] The embodiment determines the overlapping region between the first binocular image and the second binocular image, the first non-overlapping region in the first binocular image and the second non-overlapping region in the second binocular image by performing image registration on the first binocular image and the second binocular image collected by the camera, the first binocular image and the second binocular image being a group of binocular images; calculates mean image parameters of the overlapping region according to the image parameters of the first binocular image and the image parameters of the second binocular image, and determines a reference image; respectively performs contrast correction on the first non-overlapping region and the second non-overlapping region, to obtain a first corrected image corresponding to the first non-overlapping region and a second corrected image corresponding to the second non-overlapping region; and performs binocular splicing according to the reference image, the first corrected image and the second corrected image, to obtain a binocular splicing image of the first binocular image and the second binocular image. In this way, the first binocular image and the second binocular image are registered, the overlapping region between the first binocular image and the second binocular image, the first non-overlapping region and the second non-overlapping region are determined, the mean image parameters of the overlapping region are calculated, the reference image is determined, the dependence on the reference image is reduced, the risk of poor quality of the splicing image introduced by the reference image when the overlapping region of the binocular image is too different is avoided, contrast correction is performed on the first non-overlapping region and the second non-overlapping region, binocular splicing is performed based on the reference image, the first corrected image and the second corrected image, the similarity of the structure of the overlapping region is strengthened, the unnatural transition when the contrast of the non-overlapping region is too different is avoided, the coordination of the binocular splicing image is improved, and the user experience is improved.

[0101] In an embodiment, the registration module 10 is further configured to perform image denoising on the first initial image and the second initial image collected by the camera respectively, to obtain a first denoised image corresponding to the first initial image and a second denoised image corresponding to the second initial image.

[0102] The first denoising image and the second denoising image are respectively converted according to a spherical coordinate system, to obtain a first converted image corresponding to the first denoising image and a second converted image corresponding to the second denoising image;

[0103] The first converted image and the second converted image are respectively geometrically corrected, to obtain a first binocular image corresponding to the first converted image and a second binocular image corresponding to the second converted image.

[0104] In an embodiment, the registration module 10 is further configured to extract feature points from the first binocular image and the second binocular image captured by the camera, to determine a plurality of first feature points in the first binocular image and a plurality of second feature points in the second binocular image;

[0105] The first feature points and the second feature points are matched, to obtain a plurality of similar feature points in the plurality of first feature points and the plurality of second feature points;

[0106] An overlapping region between the first binocular image and the second binocular image is determined according to the similar feature points;

[0107] A first non-overlapping region in the first binocular image and a second non-overlapping region in the second binocular image are determined according to the overlapping region between the first binocular image and the second binocular image.

[0108] In an embodiment, the calculation module 20 is further configured to perform color space conversion on the first binocular image and the second binocular image, to obtain a first processed image corresponding to the first binocular image and a second processed image corresponding to the second binocular image;

[0109] A first luminance value and a first color value of the overlapping region in the first processed image are obtained;

[0110] A second luminance value and a second color value of the overlapping region in the second processed image are obtained;

[0111] A mean image parameter of the overlapping region is calculated according to the first luminance value, the first color value, the second luminance value, and the second color value, to determine a reference image.

[0112] In an embodiment, the calculation module 20 is further configured to, in the process of calculating the mean image parameter of the overlapping region according to the first luminance value, the first color value, the second luminance value, and the second color value, to determine the reference image, comprising:

[0113] The first luminance value and the second luminance value are subjected to mean value calculation, to determine a luminance mean value of the overlapping region;

[0114] performing mean calculation on the first color value and the second color value to determine a color mean value of the overlapping area;

[0115] constructing a reference image corresponding to the overlapping area according to the color mean value and the brightness mean value.

[0116] In an embodiment, the stitching module 40 is further configured to map the overlapping area of the first binocular image and the overlapping area of the second binocular image in a color channel and a brightness channel respectively according to the reference image to obtain a first coinciding area corresponding to the overlapping area of the first binocular image and a second coinciding area corresponding to the overlapping area of the second binocular image;

[0117] determining an optimal seam according to the first coinciding area and the second coinciding area;

[0118] stitching the first coinciding area, the second coinciding area, the first corrected image and the second corrected image according to the optimal seam to obtain a binocular stitched image of the first binocular image and the second binocular image.

[0119] In an embodiment, the stitching module 40 is further configured to determine a plurality of seams according to the first coinciding area and the second coinciding area;

[0120] obtaining an image evaluation parameter of each seam;

[0121] determining a seam evaluation value of each seam according to the image evaluation parameter of each seam and a preset evaluation energy function;

[0122] determining the optimal seam from the plurality of seams according to the seam evaluation value of each seam.

[0123] Since the device adopts all the technical solutions of the above-mentioned embodiments, it at least has all the beneficial effects brought by the technical solutions of the above-mentioned embodiments, which will not be repeated here.

[0124] In addition, the embodiment of the present application further proposes a storage medium, wherein the storage medium stores a binocular image processing program, and the binocular image processing program is executed by a processor to realize the steps of the binocular image processing method as described above.

[0125] Since the storage medium adopts all the technical solutions of the above-mentioned embodiments, it at least has all the beneficial effects brought by the technical solutions of the above-mentioned embodiments, which will not be repeated here.

[0126] It should be noted that the above-described workflow is merely illustrative and does not limit the scope of protection of the present application. In actual applications, a person skilled in the art can select part or all of the above-described workflow to achieve the purpose of the embodiment according to actual needs, which is not limited herein.

[0127] In addition, technical details not described in detail in the embodiment can be found in the binocular image processing method provided by any embodiment of the present application, which will not be described here.

[0128] In addition, it should be noted that in this document, the terms "comprise", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or system comprising a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or system. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of another identical element in the process, method, article or system comprising the element.

[0129] The above-mentioned embodiment numbers of the present application are only for description, not representing the advantages and disadvantages of the embodiments.

[0130] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and necessary general hardware platforms, of course, they can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, an optical disk) and includes a number of instructions to make a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) execute the methods described in various embodiments of the present application.

[0131] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A binocular image processing method, characterized by, The binocular image processing method comprises: image registration is performed on the first binocular image and the second binocular image collected by the camera, an overlapping area between the first binocular image and the second binocular image, a first non-overlapping area in the first binocular image, and a second non-overlapping area in the second binocular image are determined, and the first binocular image and the second binocular image are a group of binocular images; mean image parameters of the overlapping area are calculated according to image parameters of the first binocular image and image parameters of the second binocular image, and a reference image is determined; contrast correction is performed on the first non-overlapping area and the second non-overlapping area respectively, a first corrected image corresponding to the first non-overlapping area and a second corrected image corresponding to the second non-overlapping area are obtained; binocular splicing is performed according to the reference image, the first corrected image, and the second corrected image, and a binocular splicing image of the first binocular image and the second binocular image is obtained; the binocular splicing according to the reference image, the first corrected image, and the second corrected image to obtain the binocular splicing image of the first binocular image and the second binocular image comprises: the overlapping area of the first binocular image and the overlapping area of the second binocular image are respectively mapped in a color channel and a brightness channel according to the reference image, a first coincident area corresponding to the overlapping area of the first binocular image and a second coincident area corresponding to the overlapping area of the second binocular image are obtained; an optimal stitching line is determined according to the first coincident area and the second coincident area; binocular splicing is performed on the first coincident area, the second coincident area, the first corrected image, and the second corrected image according to the optimal stitching line, and the binocular splicing image of the first binocular image and the second binocular image is obtained.

2. The binocular image processing method of claim 1, wherein, Before the image registration is performed on the first binocular image and the second binocular image collected by the camera, the method further comprises: image denoising is performed on a first initial image and a second initial image collected by the camera respectively, a first denoised image corresponding to the first initial image and a second denoised image corresponding to the second initial image are obtained; coordinate conversion is performed on the first denoised image and the second denoised image respectively according to a spherical coordinate system, a first converted image corresponding to the first denoised image and a second converted image corresponding to the second denoised image are obtained; geometric correction is performed on the first converted image and the second converted image respectively, a first binocular image corresponding to the first converted image and a second binocular image corresponding to the second converted image are obtained.

3. The binocular image processing method of claim 1, wherein, The image registration is performed on the first binocular image and the second binocular image collected by the camera, an overlapping area between the first binocular image and the second binocular image, a first non-overlapping area in the first binocular image, and a second non-overlapping area in the second binocular image are determined, and the first binocular image and the second binocular image are a group of binocular images, and the method comprises: Feature points are extracted from the first and second binocular images captured by the camera to determine multiple first feature points in the first binocular image and multiple second feature points in the second binocular image. By performing feature matching on each first feature point and each second feature point, multiple similar feature points are obtained from multiple first feature points and multiple second feature points; The overlapping region between the first binocular image and the second binocular image is determined based on the similarity of each feature; The first non-overlapping region in the first binocular image and the second non-overlapping region in the second binocular image are determined based on the overlapping region between the first binocular image and the second binocular image.

4. The binocular image processing method of claim 1, wherein, The step of calculating the mean image parameters of the overlapping region based on the image parameters of the first binocular image and the image parameters of the second binocular image, and determining the reference image, includes: The first stereo image and the second stereo image are respectively converted to color space to obtain a first processed image corresponding to the first stereo image and a second processed image corresponding to the second stereo image. Obtain the first brightness value and the first color value of the overlapping region in the first processed image; Obtain the second brightness value and the second color value of the overlapping region in the second processed image; The mean image parameters of the overlapping region are calculated based on the first brightness value, the first color value, the second brightness value, and the second color value to determine the reference image.

5. The binocular image processing method of claim 4, wherein, The mean image parameters include the mean color and the mean brightness; The step of calculating the mean image parameters of the overlapping region based on the first brightness value, the first color value, the second brightness value, and the second color value to determine the reference image includes: The average value of the overlapping region is determined by averaging the first brightness value and the second brightness value. The average value of the first color value and the second color value is calculated to determine the average color value of the overlapping area; A reference image corresponding to the overlapping region is constructed based on the mean color value and the mean brightness value.

6. The binocular image processing method of claim 1, wherein, Determining the optimal suture line based on the first overlapping region and the second overlapping region includes: Multiple suture lines are determined based on the first overlapping region and the second overlapping region; Obtain image evaluation parameters for each suture line; The suture evaluation value of each suture is determined based on the image evaluation parameters and the preset evaluation energy function of each suture. The optimal suture is determined among all sutures based on their suture assessment values.

7. A binocular image processing apparatus, characterized by comprising: The binocular image processing device includes: The registration module is used to perform image registration on the first stereo image and the second stereo image acquired by the camera, and to determine the overlapping area between the first stereo image and the second stereo image, the first non-overlapping area in the first stereo image and the second non-overlapping area in the second stereo image. The first stereo image and the second stereo image are a set of stereo images. The calculation module is used to calculate the mean image parameters of the overlapping region based on the image parameters of the first stereo image and the image parameters of the second stereo image, and to determine the reference image; The correction module is configured to perform contrast correction on the first non-overlapping area and the second non-overlapping area respectively to obtain a first corrected image corresponding to the first non-overlapping area and a second corrected image corresponding to the second non-overlapping area; The stitching module is configured to perform binocular stitching according to the reference image, the first corrected image and the second corrected image to obtain a binocular stitching image of the first binocular image and the second binocular image. The stitching module is further configured to: map the overlapping area of the first binocular image and the overlapping area of the second binocular image in a color channel and a brightness channel respectively according to the reference image to obtain a first coincident area corresponding to the overlapping area of the first binocular image and a second coincident area corresponding to the overlapping area of the second binocular image; determine an optimal seam according to the first coincident area and the second coincident area; perform binocular stitching on the first coincident area, the second coincident area, the first corrected image and the second corrected image according to the optimal seam to obtain the binocular stitching image of the first binocular image and the second binocular image.

8. A binocular image processing device, characterized by, The device comprises a memory, a processor and a binocular image processing program stored on the memory and executable on the processor, and the binocular image processing program is configured to implement the binocular image processing method according to any one of claims 1 to 6.

9. A storage medium, characterized by The storage medium stores a binocular image processing program, and the binocular image processing program is executed by the processor to implement the binocular image processing method according to any one of claims 1 to 6.

Images