Image processing method, device, electronic device and storage medium

Abstract

The present disclosure provides an image processing method, which relates to the field of autonomous driving, and in particular to the field of deep learning. The specific implementation scheme is: determine the i-th image and the i+1-th image with matching areas in N images, where N is an integer greater than or equal to 2, i=1, ... N-1; determine the matching area between the i-th image and the i+1-th image, and obtain the first matching area in the i-th image and the second matching area in the i+1-th image; determine multiple first matching sub-areas and multiple second matching sub-areas of the first matching area and the second matching area respectively; determine the target matching image pair according to the difference between the images of the multiple first matching sub-areas and the images of the multiple second matching sub-areas; and process at least one of the i-th image and the i+1-th image according to the target matching image pair. The present disclosure also provides an image processing device, an electronic device, and a storage medium.

Description

Technical Field

[0001] The present disclosure relates to the field of autonomous driving technology, and in particular to deep learning technology. More specifically, the present disclosure provides an image processing method, device, electronic device, and storage medium. Background Art

[0002] Multiple acquisition devices may be installed on the vehicle to acquire multiple images around the vehicle. A spliced ​​image (eg, a bird's-eye view) obtained by splicing multiple images may be presented to the driver to assist driving. Summary of the invention

[0003] The present disclosure provides an image processing method, apparatus, device and storage medium.

[0004] According to one aspect of the present disclosure, there is provided an image processing method, the method comprising: determining an i-th image and an i+1-th image having matching areas among N images, wherein N is an integer greater than or equal to 2, i=1, ...N-1; determining a matching area between the i-th image and the i+1-th image to obtain a first matching area in the i-th image and a second matching area in the i+1-th image; determining a plurality of first matching sub-areas and a plurality of second matching sub-areas of each of the first matching area and the second matching area; determining a target matching image pair based on the difference between the images of the plurality of first matching sub-areas and the images of the plurality of second matching sub-areas; and processing at least one of the i-th image and the i+1-th image based on the target matching image pair.

[0005] According to another aspect of the present disclosure, an image processing device is provided, which includes: a first determination module, used to determine the i-th image and the i+1-th image having matching areas among N images, wherein N is an integer greater than or equal to 2, i=1, ...N-1; a second determination module, used to determine the matching area between the i-th image and the i+1-th image, and obtain the first matching area in the i-th image and the second matching area in the i+1-th image; a third determination module, used to determine a plurality of first matching sub-areas and a plurality of second matching sub-areas of each of the first matching area and the second matching area; a fourth determination module, used to determine a target matching image pair based on the difference between the images of the plurality of first matching sub-areas and the images of the plurality of second matching sub-areas; and a processing module, used to process at least one of the i-th image and the i+1-th image based on the target matching image pair.

[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method provided according to the present disclosure.

[0007] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided. The computer instructions are used to cause a computer to execute the method provided according to the present disclosure.

[0008] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the method provided according to the present disclosure is implemented.

[0009] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0011] Figure 1 is a schematic diagram of an exemplary application scenario of an image processing method and apparatus according to an embodiment of the present disclosure;

[0012] Figure 2 is a flowchart of an image processing method according to an embodiment of the present disclosure;

[0013] Figure 3A is a schematic diagram of N images according to another embodiment of the present disclosure;

[0014] Figure 3B is a schematic diagram of a first matching region of an i-th image according to an embodiment of the present disclosure;

[0015] Figure 3C is a schematic diagram of a second matching region of an (i+1)th image according to an embodiment of the present disclosure;

[0016] Figure 4A is a schematic diagram of a first matching area of ​​an i-th image according to another embodiment of the present disclosure;

[0017] Figure 4B is a schematic diagram of a first matching sub-region according to another embodiment of the present disclosure;

[0018] Figure 4C is a schematic diagram of a second matching region of an (i+1)th image according to another embodiment of the present disclosure;

[0019] Figure 4D is a schematic diagram of a second matching sub-region according to another embodiment of the present disclosure;

[0020] Figure 5 is a flowchart of an image processing method according to another embodiment of the present disclosure;

[0021] Figure 6 is a flowchart of an image processing method according to another embodiment of the present disclosure;

[0022] Figure 7 is a block diagram of an image processing apparatus according to an embodiment of the present disclosure; and

[0023] Figure 8 is a block diagram of an electronic device to which an image processing method can be applied according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0025] The acquisition device on the vehicle, such as the on-board camera, can collect images from multiple positions and angles. These images can be stitched into a stitched image to assist driving. Under the influence of the environment at different positions, the contrast and brightness of the images collected by each camera may vary greatly, which results in an uneven picture of the stitched image finally displayed.

[0026] Figure 1 It is a schematic diagram of an application scenario of an image processing method and device according to an embodiment of the present disclosure.

[0027] like Figure 1 As shown, four acquisition devices are mounted at corresponding positions on the vehicle 100. Each acquisition device can be used to acquire images. The four acquisition devices are: an acquisition device 101 mounted on the head of the vehicle, an acquisition device 102 mounted on the left side of the vehicle, an acquisition device 103 mounted on the right side of the vehicle, and an acquisition device 104 mounted on the rear of the vehicle. The four acquisition devices can acquire images.

[0028] Figure 2 is a flowchart of an image processing method according to an embodiment of the present disclosure.

[0029] like Figure 2As shown, the method 200 may include operations S210 to S250.

[0030] In operation S210, an i-th image and an (i+1)-th image having a matching area among N images are determined.

[0031] For example, N is an integer greater than or equal to 2, i=1, ...N-1.

[0032] For example, N images are collected simultaneously by N collection devices installed at corresponding positions on the vehicle. Taking N==4 as an example, the corresponding positions on the vehicle include: the front of the vehicle, the left side of the vehicle, the rear of the vehicle and the right side of the vehicle.

[0033] For example, taking i=1 as an example, the first image may be acquired by the acquisition device 101 mounted on the head of the vehicle as described above, and the second image may be acquired by the acquisition device 102 mounted on the left side of the vehicle as described above.

[0034] In operation S220 , a matching region between the i-th image and the (i+1)-th image is determined to obtain a first matching region in the i-th image and a second matching region in the (i+1)-th image.

[0035] For example, the first matching area of ​​the first image may be the area located at the lower left corner of the first image described above. The second matching area of ​​the second image may be the area located at the upper right corner of the second image described above.

[0036] In operation S230 , a plurality of first matching sub-regions and a plurality of second matching sub-regions of each of the first matching region and the second matching region are determined.

[0037] For example, multiple matching sub-regions may be determined in various ways. In one example, the matching region may be divided into four matching sub-regions, each of which has an equal area.

[0038] In operation S240, a target matching image pair is determined according to the difference between the images of the plurality of first matching sub-regions and the images of the plurality of second matching sub-regions.

[0039] For example, the mean values ​​of pixels in the first matching sub-regions and the mean values ​​of pixels in the second matching sub-regions may be determined respectively to determine the target matching image pair. In one example, the target matching image pair may be determined based on the two matching sub-regions with the smallest difference in the mean values ​​of the pixels.

[0040] In operation S250, at least one of the i-th image and the (i+1)-th image is processed according to the target matching image pair.

[0041] For example, the Wallis filter processing may be performed on the i-th image and the i+1-th image according to the target matching image pair. For another example, the Wallis filter processing may be performed on one of the i-th image and the i+1-th image according to the target matching image pair.

[0042] It can be understood that Wallis filtering is a filtering method used for image enhancement, which can suppress noise while increasing the contrast of the original image, and can map the grayscale mean and variance of the local image to the given grayscale mean and variance values. Wallis filtering is a local image transformation that can enhance the tiny grayscale information in the image. This feature makes the grayscale mean and variance of each area of ​​the image approximately equal, thereby achieving uniform processing between images (such as uniform color processing).

[0043] Through the embodiments of the present disclosure, the color or brightness of the processed image is more uniform, smooth and natural, and can assist the driver in driving more efficiently.

[0044] In some embodiments, determining the matching area between the i-th image and the i+1-th image to obtain the first matching area in the i-th image and the second matching area in the i+1-th image includes: determining the first world coordinate values ​​of multiple pixels in the i-th image; determining the second world coordinate values ​​of multiple pixels in the i+1-th image; determining the pixel area in the i-th image whose first world coordinate value is consistent with the second world coordinate value as the first matching area; and determining the pixel area in the i+1-th image whose second world coordinate value is consistent with the first world coordinate value as the second matching area. FIG. 3A to FIG. 3B Provide detailed explanation.

[0045] Figure 3A is a schematic diagram of N images according to another embodiment of the present disclosure.

[0046] like Figure 3A As shown, the acquisition device 301 mounted on the vehicle 300 can be used to acquire the first image 310. The acquisition device 302 mounted on the vehicle 300 can be used to acquire the second image 320. The acquisition device 303 mounted on the vehicle 300 can be used to acquire the third image 330. The acquisition device 304 mounted on the vehicle 300 can be used to acquire the fourth image 340.

[0047] In one example, the vehicle 300 may be, for example, the vehicle 100 described above. The collection device 301 may be, for example, the collection device 101. The collection device 302 may be, for example, the collection device 102. The collection device 303 may be, for example, the collection device 103. The collection device 304 may be, for example, the collection device 104.

[0048] For example, taking i=1 as an example, the first world coordinate values ​​of the plurality of pixels in the first image may be determined, and the second world coordinate values ​​of the plurality of pixels in the second image may also be determined. In one example, the acquisition device 301 may be a wide-angle camera. The first world coordinate values ​​of the plurality of pixels in the first image may be determined based on the world coordinates of the vehicle 300, the world coordinates of the acquisition device 301, and the maximum wide angle of the acquisition device 301. In a similar manner, the second world coordinate values ​​of the plurality of pixels in the second image may be determined.

[0049] Figure 3B is a schematic diagram of a first matching area of ​​an i-th image according to an embodiment of the present disclosure.

[0050] For example, taking i=1 as an example, for the first image 310, the first matching area 311 can be determined according to the pixel area whose first world coordinate value in the first image 310 is consistent with the second world coordinate value in the second image 320. Figure 3B As shown, the first matching area 311 is located at the lower left corner of the first image 310 .

[0051] Figure 3C is a schematic diagram of a second matching area of ​​the (i+1)th image according to an embodiment of the present disclosure.

[0052] For example, taking i=1 as an example, for the second image 320, the second matching area 321 can be determined based on the pixel area whose second world coordinate value in the second image 320 is consistent with the first world coordinate value in the first image 310. Figure 3C As shown, the second matching area 321 is located at the upper right corner of the second image 320 .

[0053] In some embodiments, as described above, taking i=2 as an example, the first matching area of ​​the second image 320 is located at the lower right corner of the second image 320 , and the second matching area of ​​the third image 330 is located at the upper left corner of the third image 330 .

[0054] In some embodiments, as described above, taking i=3 as an example, the first matching area of ​​the third image 330 is located at the upper right corner of the third image 320 , and the second matching area of ​​the fourth image 340 is located at the lower left corner of the fourth image 340 .

[0055] In some embodiments, the first matching area of ​​the fourth image 340 is located at the upper left corner of the fourth image 340 , and the second matching area of ​​the first image 310 is located at the lower right corner of the first image 310 .

[0056] It can be understood that, based on the four images, four first matching areas and four second matching areas can be determined.

[0057] In some embodiments, determining the first matching area and the second matching area respectively includes: determining a first central sub-area and a plurality of first corner sub-areas of the first matching area, and a second central sub-area and a plurality of second corner sub-areas of the second matching area; and determining a first matching sub-area according to the first central sub-area and one of the plurality of first corner sub-areas, and determining a second matching sub-area according to the second central sub-area and one of the plurality of second corner sub-areas. Figure 4A to Figure 4D Provide detailed explanation.

[0058] Figure 4A is a schematic diagram of a first matching area of ​​an i-th image according to another embodiment of the present disclosure.

[0059] like Figure 4A As shown, the first matching region 411 includes a first central sub-region 4111, a first corner sub-region 4112a, a first corner sub-region 4112b, a first corner sub-region 4112c and a first corner sub-region 4112d. In an example, the first matching region 411 may be, for example, the first matching region 311 described above.

[0060] Figure 4B is a schematic diagram of a first matching sub-region according to another embodiment of the present disclosure.

[0061] For example, a first matching sub-region 4113a may be determined according to the first central sub-region 4111 and the first corner sub-region 4112a.

[0062] Figure 4C is a schematic diagram of a second matching area of ​​the (i+1)th image according to another embodiment of the present disclosure.

[0063] like Figure 4C As shown, the second matching region 421 includes a second central sub-region 4211, a second corner sub-region 4212a, a second corner sub-region 4212b, a second corner sub-region 4212c and a second corner sub-region 4212d. In an example, the second matching region 421 may be, for example, the second matching region 321 described above.

[0064] Figure 4D is a schematic diagram of a second matching sub-region according to another embodiment of the present disclosure.

[0065] For example, a second matching sub-region 4213b may be determined according to the second central sub-region 4211 and the second corner sub-region 4212b.

[0066] The region center can better characterize the features of the matching region. In the embodiment of the present disclosure, each matching sub-region includes the region center, so that the processed image is more realistic.

[0067] In some embodiments, according to the difference between the images of the plurality of first matching sub-regions and the images of the plurality of second matching sub-regions, determining the target matching image pair includes: respectively determining the first mean and the first variance of the pixels in the plurality of first matching sub-regions and the second mean and the second variance of the pixels in the plurality of second matching sub-regions; and determining the mean difference between the first mean and the second mean and the ratio of the first variance to the second variance. According to the mean difference and the ratio, determining the target first matching sub-region and the target second matching sub-region with the smallest difference from each other; and determining the target matching image pair from the image of the target first matching sub-region and the image of the target second matching sub-region.

[0068] For example, taking i=1 as an example, the difference Diff between the first matching sub-region and the second matching sub-region can be determined according to the mean difference and the ratio by the following formula:

[0069]

[0070] M A1j is the first mean, M B2k is the second mean, S A1j is the first variance, S B2k In one example, the first mean of the first matching sub-region 4113a described above is M A11 , the first variance is S A11 In one example, the second mean value of the second matching sub-region 4213b described above is M B22 , the second variance is S B22 .

[0071] In one example, multiple differences can be determined by the above formula 1. If the difference between the first matching sub-region 4113a described above and the second matching sub-region 4213b described above is the smallest among these differences, the first matching sub-region 4113a can be used as the target first matching sub-region, and the second matching sub-region 4213b can be used as the target second matching sub-region. Next, the image of the target first matching sub-region and the image of the target second matching sub-region are determined as the target matching image pair A11_B22.

[0072] As described above, four first matching regions and four second matching regions can be determined based on four images. Furthermore, four target matching image pairs can be determined in a manner similar to that of determining the target matching image pair A11_B22.

[0073] Figure 5is a flowchart of an image processing method according to an embodiment of the present disclosure.

[0074] like Figure 5 As shown, the method 550 can process at least one of the i-th image and the i+1-th image according to the target matching image pair. The following will be described in detail in conjunction with operations S551 to S553. In this embodiment, the target matching image pair A11_B22 described above is taken as an example.

[0075] In operation S551, a reference mean value is determined according to a pixel mean value of the first matching image and a pixel mean value of the second matching image.

[0076] For example, the first matching image may be an image corresponding to the first matching sub-region 4113a described above, that is, an image of the target first matching sub-region.

[0077] For example, the second matching image may be an image corresponding to the second matching sub-region 4213b described above, that is, an image of the target second matching sub-region.

[0078] For example, the average of the pixel mean of the first matching image and the pixel mean of the second matching image can be used as the reference mean M f In one example, the first mean M described above can be calculated A11 and the second mean M mentioned above B22 to determine the reference mean M f .

[0079] In operation S552, a reference variance value is determined based on the pixel variance value of the first matching image and the pixel variance value of the second matching image.

[0080] For example, the average of the pixel variance value of the first matching image and the pixel variance value of the second matching image can be used as the reference variance S f In one example, the first variance S described above may be calculated as A11 and the second variance mentioned above is S B22 to determine the reference variance S f .

[0081] In operation S553, at least one of the i-th image and the (i+1)-th image is processed according to the reference mean value and the reference variance value.

[0082] In the embodiment of the present disclosure, at least one of the i-th image and the (i+1)-th image may be processed using the following formula:

[0083] f(x,y)=[g(x,y)-M g ]*K+B (Formula 2)

[0084] B=[b*M f +(1-b)*M g ] (Formula 3)

[0085] g(x, y) is the pixel value at (x, y) in the i-th image or the i+1-th image, f(x, y) is the pixel value at (x, y) in the i-th image or the i+1-th image after processing, M f is the reference mean, M g is the pixel mean of the i-th image or the i+1-th image, K is obtained according to the reference variance, b is a preset parameter, and both K and b are greater than or equal to 0 and less than or equal to 1.

[0086] For example, K is obtained from the reference variance by the following formula:

[0087]

[0088] c is another preset parameter, which is greater than or equal to 0 and less than or equal to 1. g is the pixel variance value of the i-th image or the i+1-th image. Through this formula, the previous image features can be maintained and the image can be made clearer.

[0089] For another example, c may be a value greater than or equal to 0.25 and less than or equal to 0.9, and b may be a value greater than or equal to 0.25 and less than or equal to 1.0.

[0090] It can be understood that, referring to the way of processing at least one of the i-th image and the i+1-th image according to the target matching image pair A11_B22, at least one of the two images corresponding to the image pair can be processed according to any of the four target matching image pairs mentioned above.

[0091] In some embodiments, after the first image and the second image are processed, the second image and the third image may be processed in a similar manner. Next, the third image and the fourth image may be processed in a similar manner. Finally, the fourth image and the first image may be processed in a similar manner to complete a round of forward image processing. For example, the forward direction may refer to a clockwise direction.

[0092] In some embodiments, after the first image and the second image are processed, the fourth image and the first image may be processed in a similar manner. Next, the third image and the fourth image may be processed in a similar manner. Finally, the second image and the third image may be processed in a similar manner to complete a round of reverse image processing. For example, reverse may refer to a counterclockwise direction.

[0093] It can be understood that multiple rounds of image processing can be performed so that the difference between the N images meets the predetermined condition.

[0094] It is understood that multiple rounds of forward image processing can be performed, multiple rounds of reverse image processing can be performed, or multiple rounds of forward image processing and multiple rounds of reverse image processing can be performed alternately. Through multiple rounds of image processing, the color or brightness of the processed image can be made smoother or more natural.

[0095] Figure 6 is a flowchart of an image processing method according to another embodiment of the present disclosure.

[0096] like Figure 6 As shown, the method 660 may be performed after operation S553. The following will be described in detail in conjunction with operations S661 to S662.

[0097] In operation S661 , N target matching image pairs among N images are determined.

[0098] For example, taking N=4 as an example, as described above, according to a method similar to determining the target matching image pair A11_B22, a total of 4 target matching image pairs can be determined.

[0099] In operation S662, the N images are processed according to the pixel means and pixel variance values ​​of 2*N target matching images among the N target matching image pairs.

[0100] In the embodiment of the present disclosure, the average pixel means and average pixel variance values ​​of 2*N target matching images may be determined.

[0101] For example, determine the average pixel mean of the nth target matching image pair and the average pixel variance In one example, n=1, ... N. In this embodiment, n=1, 2, 3, 4.

[0102] In the embodiment of the present disclosure, the average pixel mean value and the average pixel variance value may be used as the reference mean value and the reference variance value, respectively, to process N images.

[0103] For example, the average pixel mean and the average pixel variance They are used as the parameter mean and reference variance values ​​respectively to process N images.

[0104] In one example, the average pixel mean and the average pixel variance As reference mean and reference variance values ​​respectively, the four images are processed simultaneously using Formula 2 and Formula 3 described above to obtain the four processed images.

[0105] It is understandable that method 660 may be performed after one round of image processing as described above, or after a predetermined number of rounds of image processing, so that the color of the processed image is further smoothed or uniform.

[0106] It should be noted that, in the above-described embodiments, N=4 is used as an example. However, in the present disclosure, the value of N is not limited thereto. In other embodiments, N may be 2. In other embodiments, N may be 5. As long as there are matching areas between the N images, the above-described image processing method can be applied.

[0107] It should be noted that any of the acquisition devices described above can acquire RGB images. The N images described above can be grayscale images of the same channel in the N RGB images acquired simultaneously by the N acquisition devices. After the above-described image processing is performed on the grayscale images of each channel in the N RGB images, the grayscale images of the three channels can be correspondingly fused to obtain the processed N RGB images.

[0108] Figure 7 is a block diagram of an image processing apparatus according to an embodiment of the present disclosure.

[0109] like Figure 7 As shown, the apparatus 700 may include a first determination module 710 , a second determination module 720 , a third determination module 730 , a third determination module 740 and a processing module 750 .

[0110] The first determination module 710 is used to determine the i-th image and the (i+1)-th image having matching areas among the N images, where N is an integer greater than or equal to 2, i=1, ...N-1.

[0111] The second determination module 720 is used to determine the matching area between the i-th image and the (i+1)-th image, and obtain a first matching area in the i-th image and a second matching area in the (i+1)-th image.

[0112] The third determining module 730 is configured to determine a plurality of first matching sub-regions and a plurality of second matching sub-regions of each of the first matching region and the second matching region.

[0113] The fourth determination module 740 is configured to determine a target matching image pair according to the difference between the images of the plurality of first matching sub-regions and the images of the plurality of second matching sub-regions.

[0114] The processing module 750 is used to process at least one of the i-th image and the (i+1)-th image according to the target matching image pair.

[0115] In some embodiments, the third determination module includes: a first determination submodule, used to determine a first central subregion and multiple first corner subregions of the first matching area, and a second central subregion and multiple second corner subregions of the second matching area; and a second determination submodule, used to determine a first matching subregion based on the first central subregion and one of the multiple first corner subregions, and a third determination submodule, used to determine a second matching subregion based on the second central subregion and one of the multiple second corner subregions.

[0116] In some embodiments, the fourth determination module includes: a fourth determination submodule, used to respectively determine the first mean and the first variance of the pixels in the multiple first matching subregions and the second mean and the second variance of the pixels in the multiple second matching subregions; and a fifth determination submodule, used to determine the mean difference between the first mean and the second mean and the ratio of the first variance to the second variance; a sixth determination submodule, used to determine the target first matching subregion and the target second matching subregion with the smallest difference from each other based on the mean difference and the ratio; and a seventh determination submodule, used to determine the image of the target first matching subregion and the image of the target second matching subregion as a target matching image pair.

[0117] In some embodiments, the processing module includes: an eighth determination submodule, used to determine a reference mean based on a pixel mean of the first matching image and a pixel mean of the second matching image; a ninth determination submodule, used to determine a reference variance value based on a pixel variance value of the first matching image and a pixel variance value of the second matching image; and a first processing submodule, used to process at least one of the i-th image and the i+1-th image based on the reference mean and the reference variance value.

[0118] In some embodiments, the first processing submodule includes: a first processing unit for processing at least one of the i-th image and the i+1-th image using the following formula: f(x, y) = [g(x, y) - M g ]*K+B; B=[b*M f +(1-b)*M g ]; g(x, y) is the pixel value at (x, y) in the i-th image or the i+1-th image, f(x, y) is the pixel value at (x, y) in the i-th image after processing or the i+1-th image after processing, M f is the reference mean, M gis the pixel mean of the i-th image or the i+1-th image, and K is based on the reference variance S f It is obtained that b is a preset parameter, and both K and b are greater than or equal to 0 and less than or equal to 1.

[0119] In some embodiments, c is a preset parameter, c is greater than or equal to 0 and less than or equal to 1, S f is the reference variance value, S g is the pixel variance value of the i-th image or the i+1-th image.

[0120] In some embodiments, it also includes: a tenth determination submodule, used to determine N target matching image pairs among the N images; a second processing submodule, used to process the N images according to the pixel mean and pixel variance value of 2*N target matching images among the N target matching image pairs.

[0121] In some embodiments, the second processing submodule includes: a first determination unit, used to determine the average pixel mean and the average pixel variance value of the 2*N target matching images; a second processing unit, used to process the N images using the average pixel mean and the average pixel variance value as the reference mean and the reference variance value respectively.

[0122] In some embodiments, the second processing submodule includes: a second determining unit for determining an average pixel mean of the nth target matching image pair and the average pixel variance The third processing unit is used to average the pixel mean and the average pixel variance The reference mean and the reference variance are used as the reference mean and the reference variance respectively to process the N images.

[0123] In some embodiments, the second determination module includes: an eleventh determination submodule, used to determine the first world coordinate values ​​of multiple pixels in the i-th image; a twelfth determination submodule, used to determine the second world coordinate values ​​of multiple pixels in the i+1-th image; a thirteenth determination submodule, used to determine the pixel area in the i-th image whose first world coordinate value is consistent with the second world coordinate value as the first matching area; and a fourteenth determination submodule, used to determine the pixel area in the i+1-th image whose second world coordinate value is consistent with the first world coordinate value as the second matching area.

[0124] In some embodiments, the N images are simultaneously acquired by N acquisition devices respectively mounted at corresponding positions on the vehicle.

[0125] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0126] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0127] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0128] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0129] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0130] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as image processing methods. For example, in some embodiments, the image processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the image processing method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the image processing method in any other appropriate manner (e.g., by means of firmware).

[0131] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0132] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0133] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0134] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0135] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0136] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0137] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0138] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An image processing method, comprising: Determine an i-th image and an i+1-th image having a matching area among N images, where N is an integer greater than or equal to 2, i=1, ...N-1; Determine a matching area between the i-th image and the (i+1)-th image, and obtain a first matching area in the i-th image and a second matching area in the (i+1)-th image; Determine a plurality of first matching sub-regions and a plurality of second matching sub-regions of the first matching region and the second matching region respectively; Determine a target matching image pair according to a difference between an average value of pixels in the plurality of first matching sub-areas and an average value of pixels in the plurality of second matching sub-areas, wherein the target matching image pair includes a first matching image and a second matching image; Determining a reference mean value according to a pixel mean value of the first matching image and a pixel mean value of the second matching image; Determining a reference variance value according to the pixel variance value of the first matching image and the pixel variance value of the second matching image; and At least one of the i-th image and the (i+1)-th image is processed according to the reference mean value and the reference variance value.

2. The method according to claim 1, wherein: The determining of a plurality of first matching sub-regions and a plurality of second matching sub-regions of the first matching region and the second matching region respectively comprises: Determining a first central sub-region and a plurality of first corner sub-regions of the first matching region, and a second central sub-region and a plurality of second corner sub-regions of the second matching region; and A first matching sub-region is determined according to the first central sub-region and one of the plurality of first corner sub-regions, and a second matching sub-region is determined according to the second central sub-region and one of the plurality of second corner sub-regions.

3. The method according to claim 1, wherein: Determining the target matching image pair according to the difference between the mean values ​​of the pixels in the plurality of first matching sub-areas and the mean values ​​of the pixels in the plurality of second matching sub-areas comprises: respectively determining first means and first variances of pixels in the plurality of first matching sub-areas and second means and second variances of pixels in the plurality of second matching sub-areas; and determining a mean difference between the first mean and the second mean and a ratio of the first variance to the second variance; Determining, according to the mean difference and the ratio, a target first matching subregion and a target second matching subregion having the smallest difference from each other; and The image of the target first matching sub-region and the image of the target second matching sub-region are determined as a target matching image pair.

4. The method according to claim 1, wherein: The processing of at least one of the i-th image and the i+1-th image according to the reference mean value and the reference variance value comprises processing at least one of the i-th image and the i+1-th image using the following formula: is the i-th image or the i+1-th image The pixel value at In the processed i-th image or the processed i+1-th image The pixel value at is the reference mean, is the pixel mean of the i-th image or the i+1-th image, and K is the reference variance value It is obtained that b is a preset parameter, and both K and b are greater than or equal to 0 and less than or equal to 1.

5. The method according to claim 4, wherein: c is a preset parameter, which is greater than or equal to 0 and less than or equal to 1. is the reference variance value, is the pixel variance value of the i-th image or the i+1-th image.

6. The method according to claim 4, further comprising: Determining N target matching image pairs among the N images; The N images are processed according to pixel means and pixel variance values ​​of 2*N target matching images in the N target matching image pairs.

7. The method according to claim 6, wherein: The processing of the N images according to the pixel means and pixel variance values ​​of 2*N target matching images in the N target matching image pairs comprises: Determine the average pixel mean and average pixel variance value of the 2*N target matching images; The average pixel mean value and the average pixel variance value are used as the reference mean value and the reference variance value respectively, and the N images are processed.

8. The method according to claim 6, wherein: The processing of the N images according to the pixel means and pixel variance values ​​of 2*N target matching images in the N target matching image pairs comprises: Determine the average pixel mean of the nth target matching image pair and the average pixel variance , n=1,…N; The average pixel value and the average pixel variance The reference mean and the reference variance are used as the reference mean and the reference variance respectively to process the N images.

9. The method according to claim 1, wherein: The determining of the matching area between the i-th image and the i+1-th image to obtain a first matching area in the i-th image and a second matching area in the i+1-th image comprises: Determining first world coordinate values ​​of a plurality of pixels in the i-th image; Determine second world coordinate values ​​of a plurality of pixels in the (i+1)th image; Determine a pixel region in the i-th image whose first world coordinate value is consistent with the second world coordinate value as a first matching region; and A pixel region in the (i+1)th image whose second world coordinate value is consistent with the first world coordinate value is determined as a second matching region.

10. The method according to any one of claims 1 to 9, wherein: The N images are acquired simultaneously by N acquisition devices respectively mounted at corresponding positions on the vehicle.

11. An image processing device, comprising: A first determination module is used to determine an i-th image and an (i+1)-th image having a matching area among N images, wherein N is an integer greater than or equal to 2, i=1, ...N-1; A second determination module is used to determine a matching area between the i-th image and the (i+1)-th image, and obtain a first matching area in the i-th image and a second matching area in the (i+1)-th image; A third determining module, configured to determine a plurality of first matching sub-regions and a plurality of second matching sub-regions of each of the first matching region and the second matching region; a fourth determination module, configured to determine a target matching image pair according to a difference between an average value of pixels in the plurality of first matching sub-regions and an average value of pixels in the plurality of second matching sub-regions, wherein the target matching image pair includes a first matching image and a second matching image; an eighth determination submodule, configured to determine a reference mean value according to a pixel mean value of the first matching image and a pixel mean value of the second matching image; a ninth determination submodule, configured to determine a reference variance value according to the pixel variance value of the first matching image and the pixel variance value of the second matching image; and The first processing submodule is used to process at least one of the i-th image and the (i+1)-th image according to the reference mean value and the reference variance value.

12. The device according to claim 11, wherein The third determination module comprises: a first determining submodule, configured to determine a first central subregion and a plurality of first corner subregions of the first matching region, and a second central subregion and a plurality of second corner subregions of the second matching region; and a second determining submodule, configured to determine a first matching subregion according to the first central subregion and one of the plurality of first corner subregions, and The third determination submodule is used to determine one of the second matching subregions according to the second central subregion and one of the plurality of second corner subregions.

13. The device according to claim 11, wherein: The fourth determination module comprises: A fourth determination submodule, configured to respectively determine a first mean value and a first variance value of pixels in the plurality of first matching subregions and a second mean value and a second variance value of pixels in the plurality of second matching subregions; and a fifth determination submodule, configured to determine a mean difference between the first mean and the second mean and a ratio of the first variance to the second variance; A sixth determining submodule, configured to determine, according to the mean difference and the ratio, a target first matching subregion and a target second matching subregion having the smallest difference from each other; and The seventh determination submodule is used to determine the image of the target first matching subregion and the image of the target second matching subregion as a target matching image pair.

14. The device according to claim 11, wherein: The first processing submodule includes: a first processing unit, configured to process at least one of the i-th image and the i+1-th image using the following formula: is the i-th image or the i+1-th image The pixel value at In the processed i-th image or the processed i+1-th image The pixel value at is the reference mean, is the pixel mean of the i-th image or the i+1-th image, and K is the reference variance value It is obtained that b is a preset parameter, and both K and b are greater than or equal to 0 and less than or equal to 1.

15. The device according to claim 14, wherein: c is a preset parameter, which is greater than or equal to 0 and less than or equal to 1. is the reference variance value, is the pixel variance value of the i-th image or the i+1-th image.

16. The apparatus according to claim 14, further comprising: a tenth determination submodule, configured to determine N target matching image pairs among the N images; The second processing submodule is used to process the N images according to the pixel means and pixel variance values ​​of 2*N target matching images in the N target matching image pairs.

17. The device according to claim 16, wherein: The second processing submodule includes: A first determining unit, used to determine an average pixel mean and an average pixel variance value of the 2*N target matching images; The second processing unit is used to process the N images by taking the average pixel mean value and the average pixel variance value as the reference mean value and the reference variance value respectively.

18. The device according to claim 16, wherein: The second processing submodule includes: The second determination unit is used to determine the average pixel mean of the nth target matching image pair. and the average pixel variance , n=1,…N; The third processing unit is used to average the pixel mean and the average pixel variance The reference mean and the reference variance are used as the reference mean and the reference variance respectively to process the N images.

19. The device according to claim 11, wherein: The second determining module comprises: an eleventh determining submodule, configured to determine first world coordinate values ​​of a plurality of pixels in the i-th image; A twelfth determining submodule, used to determine second world coordinate values ​​of a plurality of pixels in the (i+1)th image; A thirteenth determining submodule is used to determine a pixel region in the i-th image whose first world coordinate value is consistent with the second world coordinate value as a first matching region; and A fourteenth determining submodule is configured to determine a pixel region in the (i+1)th image whose second world coordinate value is consistent with the first world coordinate value as a second matching region.

20. The device according to any one of claims 11 to 19, wherein: The N images are acquired simultaneously by N acquisition devices respectively mounted at corresponding positions on the vehicle.

21. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.

22. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 10.

23. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 10.

Images