Panoramic image processing method and apparatus, and device

Abstract

Embodiments of the present application provide a panoramic image processing method and apparatus, and a device. The method comprises: acquiring single-side bird's-eye views of different regions collected by a vehicle-mounted camera; determining a composite bird's-eye view on the basis of the single-side bird's-eye views, wherein the composite bird's-eye view comprises an overlapping region of any two single-side bird's-eye views; performing pixel correction on the composite bird's-eye view on the basis of an occlusion handling algorithm; and performing pixel correction on the single-side bird's-eye views on the basis of the corrected composite bird's-eye view, and outputting a panoramic image. Adjustment coefficients of the single-side bird's-eye views can be obtained by correcting the composite bird's-eye view, thereby correcting the single-side bird's-eye views, so as to effectively reduce the impact of obstructions on the quality of the panoramic image, and improve the user experience.

Description

Panoramic image processing methods, devices and equipment

[0001] This application claims priority to Chinese Patent Application No. 202510072485.3, filed on January 16, 2025, entitled "Panoramic Image Processing Method, Apparatus and Device", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of vehicle control, and more specifically to a panoramic image processing method, apparatus, and device. Background Technology

[0003] Panoramic imaging is a common application of automotive cameras, widely used for 360-degree monitoring of the vehicle's surroundings, helping drivers gain a more comprehensive view and aiding in decision-making. White balance processing is a crucial step in panoramic image output, used to correct color casts in images to obtain images with accurate colors. However, due to complex lighting conditions in daytime scenes, automotive cameras are prone to obstruction (such as obstacles or dirty lenses). Existing white balance adjustment methods cannot solve the obstruction problems encountered by vehicle cameras, leading to a decrease in panoramic image quality and impacting user experience. Summary of the Invention

[0004] In view of this, this application provides a panoramic image processing method, apparatus, and device to help solve the problem of panoramic image quality degradation caused by obstacles blocking the vehicle camera in the prior art.

[0005] In a first aspect, embodiments of this application provide a panoramic image processing method, including:

[0006] Acquire one-sided bird's-eye view images of different areas captured by the vehicle-mounted camera;

[0007] A composite bird's-eye view is determined based on the single-sided bird's-eye view, and the composite bird's-eye view includes the overlapping area of ​​any two single-sided bird's-eye views;

[0008] The composite bird's-eye view is pixel-corrected based on an occlusion processing algorithm;

[0009] The single-sided bird's-eye view is pixel-corrected based on the corrected composite bird's-eye view, and a panoramic image is output.

[0010] In one optional embodiment, the pixel correction of the composite bird's-eye view based on the occlusion processing algorithm includes:

[0011] In response to the presence of colored obstacles obstructing the vehicle-mounted camera, the pixel values ​​of the composite bird's-eye view are subjected to color shift correction processing based on the first occlusion processing algorithm;

[0012] In response to the presence of a black obstacle obstructing the vehicle-mounted camera, the pixel values ​​of the composite bird's-eye view are corrected for occlusion based on a second occlusion processing algorithm.

[0013] In one optional embodiment, the color shift correction processing of the pixel values ​​of the composite bird's-eye view based on the first occlusion processing algorithm includes:

[0014] The presence of color cast in the composite bird's-eye view is determined based on the pixel mean of each color channel.

[0015] When it is determined that the composite bird's-eye view has the color shift phenomenon, the pixel values ​​of each color channel of the composite bird's-eye view are converted to grayscale. When it is determined that the composite bird's-eye view does not have the color shift phenomenon, the pixel values ​​of each color channel of the composite bird's-eye view are not processed.

[0016] In one optional embodiment, the occlusion correction processing of the pixel values ​​of the composite bird's-eye view based on the second occlusion processing algorithm includes:

[0017] Based on the grayscale values ​​of each composite bird's-eye view, each composite bird's-eye view is divided into dark areas or non-dark areas;

[0018] For any composite bird's-eye view divided into dark areas, the grayscale value of the current composite bird's-eye view is corrected based on the grayscale values ​​of adjacent composite bird's-eye views.

[0019] In one optional embodiment, the step of correcting the grayscale value of the current composite bird's-eye view based on the grayscale values ​​of adjacent composite bird's-eye views includes:

[0020] If the two composite bird's-eye views adjacent to the current composite bird's-eye view are both non-dark areas, then the average gray value of the current composite bird's-eye view and the two adjacent composite bird's-eye views is determined as the gray value of the current composite bird's-eye view.

[0021] If any composite bird's-eye view adjacent to the current composite bird's-eye view is a non-dark area, then the average gray value of the current composite bird's-eye view and the adjacent composite bird's-eye view that is classified as a non-dark area is determined as the gray value of the current composite bird's-eye view.

[0022] In one optional embodiment, the step of performing pixel correction on the single-sided bird's-eye view based on the corrected composite bird's-eye view and outputting a panoramic image includes:

[0023] Based on the corrected composite bird's-eye view, error equations for each color channel are constructed;

[0024] Solve the error equations for each color channel to obtain the adjustment coefficients for each color channel in each single-sided bird's-eye view;

[0025] The pixel values ​​of the single-sided bird's-eye view are corrected based on the adjustment coefficient, and a panoramic image is output.

[0026] In one optional embodiment, the step of correcting the pixel values ​​of the unilateral bird's-eye view based on the adjustment coefficient and outputting the panoramic image includes:

[0027] The pixel values ​​of the single-sided bird's-eye view are multiplied by the corresponding adjustment coefficients to obtain the corrected single-sided bird's-eye view.

[0028] The overlapping areas of the corrected single-sided bird's-eye view are fused using a fusion algorithm to output the panoramic image.

[0029] Secondly, embodiments of this application provide a panoramic image processing apparatus, including:

[0030] The acquisition module is used to acquire one-sided bird's-eye view images of different areas captured by the vehicle-mounted camera;

[0031] A determining module is used to determine a composite bird's-eye view based on the single-sided bird's-eye view, wherein the composite bird's-eye view includes the overlapping area of ​​any two single-sided bird's-eye views;

[0032] The first correction module is used to perform pixel correction on the composite bird's-eye view based on the occlusion processing algorithm.

[0033] The second correction module is used to perform pixel correction on the single-sided bird's-eye view based on the corrected composite bird's-eye view and output a panoramic image.

[0034] Thirdly, embodiments of this application provide an electronic device, including a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute the method described in any of the first aspects above.

[0035] Fourthly, embodiments of this application provide a computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform the method described in any of the first aspects.

[0036] Fifthly, embodiments of this application provide a computer program product comprising executable instructions that, when executed on a computer, cause the computer to perform the method described in any of the first aspects.

[0037] The solution provided in this application involves acquiring single-sided bird's-eye view images of different areas captured by a vehicle-mounted camera; determining a composite bird's-eye view based on the single-sided bird's-eye view images, the composite bird's-eye view image including the overlapping areas of any two single-sided bird's-eye view images; performing pixel correction on the composite bird's-eye view image based on an occlusion processing algorithm; performing pixel correction on the single-sided bird's-eye view image based on the corrected composite bird's-eye view image; and outputting a panoramic image. By correcting the composite bird's-eye view image, an adjustment coefficient for the single-sided bird's-eye view image can be obtained, thereby correcting the single-sided bird's-eye view image and effectively reducing the impact of occlusions on the panoramic image quality, thus improving the user experience. Attached Figure Description

[0038] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 is a flowchart illustrating a panoramic image processing method provided in an embodiment of this application;

[0040] Figure 2 is a schematic diagram of an example of a panoramic image processing method provided in an embodiment of this application;

[0041] Figure 3 is a schematic diagram of another panoramic image processing method provided in an embodiment of this application;

[0042] Figure 4 is a flowchart illustrating another panoramic image processing method provided in an embodiment of this application;

[0043] Figure 5 is a schematic diagram of another panoramic image processing method provided in the embodiments of this application;

[0044] Figure 6 is a schematic diagram of another panoramic image processing method provided in an embodiment of this application;

[0045] Figure 7 is a flowchart illustrating another panoramic image processing method provided in an embodiment of this application;

[0046] Figure 8 is a flowchart illustrating another panoramic image processing method provided in an embodiment of this application;

[0047] Figure 9 is a schematic diagram of the structure of a panoramic image processing device provided in an embodiment of this application;

[0048] Figure 10 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0049] To better understand the technical solution of this application, the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0050] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0051] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0052] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0053] With the rapid development of autonomous driving technology, panoramic imaging is gradually becoming an essential feature of automobiles. Multiple onboard cameras are installed around the vehicle to monitor its surroundings in 360 degrees, outputting panoramic images on the in-car screen to help drivers gain a more comprehensive view and assist in decision-making. White balance adjustment is a crucial step in the panoramic image output process. Existing technologies typically rely on methods based on physical models or statistical learning to adjust the white balance by calculating color information in the image, thereby eliminating color deviations under different lighting conditions. Most existing white balance algorithms can handle ambient light variations to some extent, but in practical applications, at least the following problems exist:

[0054] (1) Obstacles: Vehicle cameras often encounter obstructions (such as obstacles, dirty lenses, etc.) in daytime scenes. These obstructions will cause large areas of color blocks in the image, accompanied by uneven lighting distribution, affecting the accuracy of white balance adjustment, and thus affecting the quality of panoramic images and subsequent automated analysis and decision-making.

[0055] (2) Algorithm complexity and efficiency issues: Existing white balance algorithms typically require high-performance computing platforms and complex mathematical libraries. For example, deep learning-based image processing methods train convolutional neural network (CNN) models, utilize large amounts of data to learn image features, and then predict and adjust the white balance. The disadvantage of these methods is that they require a large amount of computing resources and training data, rely on advanced hardware support, and have high investment costs.

[0056] To address the aforementioned issues, this application provides a panoramic image processing method that determines adjustment coefficients based on composite bird's-eye views of overlapping areas, corrects single-sided bird's-eye views, and then outputs high-quality panoramic images.

[0057] Figure 1 is a schematic flowchart of a panoramic image processing method provided in an embodiment of this application. This method can be executed by a processing device mounted on a vehicle. As shown in Figure 1, the method may include:

[0058] Step 101: Obtain a one-sided bird's-eye view of different areas captured by the vehicle-mounted camera.

[0059] Step 102: Determine a composite bird's-eye view based on the single-sided bird's-eye view. The composite bird's-eye view includes the overlapping area of ​​any two single-sided bird's-eye views.

[0060] Step 103: Perform pixel correction on the composite bird's-eye view based on the occlusion processing algorithm.

[0061] Step 104: Based on the corrected composite bird's-eye view, perform pixel correction on the single-sided bird's-eye view and output the panoramic image.

[0062] The vehicle is equipped with multiple onboard cameras, and the image captured by each camera can be considered a single-sided bird's-eye view. Optionally, the onboard cameras may include a front camera, a rear camera, a left camera, and a right camera. Referring to Figure 2, the image captured by the front camera is the front bird's-eye view, the image captured by the rear camera is the rear bird's-eye view, the image captured by the left camera is the left bird's-eye view, and the image captured by the right camera is the right bird's-eye view. The front bird's-eye view overlaps with both the left and right bird's-eye views, and the rear bird's-eye view also overlaps with both the left and right bird's-eye views. Referring to Figure 3, each single-sided bird's-eye view contains two overlapping areas, for a total of eight overlapping areas. Each overlapping area can be considered a composite bird's-eye view.

[0063] Under normal circumstances, the obstruction of vehicle cameras by obstacles can include the following situations: (1) Color obstruction: such as colored obstacles being close to the vehicle camera, light-transmitting colored obstacles being attached to the camera, or large areas of colored content in the world coordinates, etc. These situations cause the camera to capture large areas of colored blocks. (2) Black obstruction: such as opaque obstacles occupying most of the field of view of the vehicle camera, opaque obstacles covering the vehicle camera, or large areas of black content in the world coordinates, etc. These situations cause the camera to capture large areas of black shadows.

[0064] To address the issue of colored obstacles obstructing the view from the vehicle-mounted camera, the processing device can perform color shift correction on the pixel values ​​of the composite bird's-eye view based on a first occlusion processing algorithm. To address the issue of black obstacles obstructing the view from the vehicle-mounted camera, the processing device can perform occlusion correction on the pixel values ​​of the composite bird's-eye view based on a second occlusion processing algorithm.

[0065] (1) Color deviation processing: After determining the composite bird's-eye view corresponding to each single-sided bird's-eye view, the processing device can determine whether there is a color deviation phenomenon in the composite bird's-eye view based on the pixel mean values of each color channel of the composite bird's-eye view. When it is determined that there is a color deviation phenomenon in the composite bird's-eye view, the pixel values of each color channel of the composite bird's-eye view are subjected to grayscale conversion. When it is determined that there is no color deviation phenomenon in the composite bird's-eye view, the pixel values of each color channel of the composite bird's-eye view are not processed.

[0066] Specifically, define the width of each composite bird's-eye view as W, the height as H, the pixel point coordinates as (x, y), and meanR, meanG, and meanB as the pixel mean values of the R channel, G channel, and B channel respectively. The processing device can calculate the channel mean values of 8 composite bird's-eye views respectively, and the calculation formula is as follows:

[0067] Define T1, T2, T3, and T4 as the thresholds of the gray series within the color temperature range of 2500K - 7500K. If the ratio of the channel mean values of the current overlapping image is not within the range of the gray series thresholds, it is determined as color deviation, that is, there are large areas of colored blocks in the overlapping image. At this time, the channel mean values are subjected to grayscale conversion, and the converted values are constrained within the range of 1 - 255. The specific steps can refer to Figure 4, mainly including:

[0068] Step 401, input the composite bird's-eye view channel mean values meanR, meanG, and meanB.

[0069] Step 402, determine whether each channel mean value meets the conditions T1 < (meanR / meanG) < T2 and T3 < (meanB / meanG) < T4? If so, enter Step 404, otherwise enter Step 403.

[0070] Step 403, perform grayscale conversion on the pixel values of the composite bird's-eye view.

[0071] Step 404, output the composite bird's-eye view.

[0072] In the embodiment of the present application, for the area with color occlusion, the processing device converts the color information in the composite bird's-eye view into brightness information to weaken the influence of color on adjacent areas.

[0073] (2) Occlusion processing: After determining the composite bird's-eye view corresponding to each single-sided bird's-eye view, the processing device can divide each composite bird's-eye view into a dark area or a non-dark area based on the grayscale value of each composite bird's-eye view. For any composite bird's-eye view divided into a dark area, the processing device corrects the grayscale value of the current composite bird's-eye view based on the grayscale value of the adjacent composite bird's-eye view.

[0074] Because there is an overlapping area between the images captured by the two vehicle-mounted cameras, if one of the cameras produces an abnormal image due to obstruction by an obstacle, the overlapping area can be used as a reference to correct the abnormal image pixel by pixel. Referring to Figure 5, the overlapping part of the left bird's-eye view and the front bird's-eye view is called the left front bird's-eye view, and the overlapping part of the front bird's-eye view and the left bird's-eye view is called the front left bird's-eye view, and so on. The left front bird's-eye view and the front left bird's-eye view are images captured by different vehicle-mounted cameras for the same area. If the front camera is obstructed, the front left bird's-eye view will be abnormal, and the processing device can correct the front left bird's-eye view based on the left front bird's-eye view.

[0075] Specifically, the processing equipment calculates the mean grayscale value of each composite bird's-eye view using the following formula: Grey = 0.114 * meanR + 0.578 * meanG + 0.229 * meanB

[0076] The processing device classifies areas with a grayscale mean less than a grayscale threshold as dark areas and areas with a grayscale mean greater than or equal to the grayscale threshold as non-dark areas. Optionally, the grayscale threshold can be set based on experience, such as 50 or 60. Dark areas have low average brightness and are highly likely to be black occlusions, while non-dark areas have high average brightness and can be considered normal images. The above eight composite bird's-eye view images are sequentially concatenated according to their order in the panoramic view to obtain a concatenated relationship diagram, as shown in Figure 6.

[0077] The processing equipment can process composite bird's-eye views based on the following rules:

[0078] (1) If the number of non-dark areas is 0, that is, the average brightness of all composite bird's-eye view is low and the overall brightness of the panoramic view is balanced, no processing is required.

[0079] (2) If the composite bird's-eye view before and after a dark area is not a dark area, the gray values ​​of the dark area and the composite bird's-eye view before and after are added together and the average value is taken. The average value is used to replace the original gray value of the dark area.

[0080] (3) If the adjacent composite bird's-eye view before or after a dark area is a non-dark area, the gray values ​​of the dark area and the adjacent bird's-eye view before or after are added together and the average value is taken. This average value is used to replace the original gray value of the dark area.

[0081] (4) If the adjacent composite bird's-eye view before and after the dark area is also a dark area, then no processing is required for the current dark area.

[0082] The specific steps can be seen in the flowchart shown in Figure 7, and mainly include:

[0083] Step 701: Construct a connection diagram based on the composite bird's-eye view.

[0084] Step 702: Determine the dark and non-dark areas based on the grayscale mean of the composite bird's-eye view.

[0085] Step 703: Determine if the number of non-dark areas is greater than 0. If yes, proceed to step 704; otherwise, proceed to step 710.

[0086] Step 704: If the i-th composite bird's-eye view is a dark area, determine whether the (i+1)-th and (i-1)-th composite bird's-eye view are both non-dark areas. If so, proceed to step 707; otherwise, proceed to step 705.

[0087] Step 705: If the i-th composite bird's-eye view is a dark area, determine whether the (i+1)-th composite bird's-eye view is a non-dark area. If yes, proceed to step 708; otherwise, proceed to step 706.

[0088] Step 706: If the i-th composite bird's-eye view is a dark area, determine whether the (i-1)-th composite bird's-eye view is a non-dark area. If yes, proceed to step 709; otherwise, proceed to step 707.

[0089] Step 707: Determine the grayscale value of the i-th composite bird's-eye view as the average of the grayscale values ​​of the (i-1), i, and i+1 composite bird's-eye view images.

[0090] Step 708: Determine the grayscale value of the i-th composite bird's-eye view as the average of the grayscale values ​​of the i-th and i+1-th composite bird's-eye views.

[0091] Step 709: Determine the grayscale value of the i-th composite bird's-eye view as the average of the grayscale values ​​of the i-th and i-1-th composite bird's-eye views.

[0092] Step 710: Output the grayscale values ​​of the composite bird's-eye view.

[0093] Through the above process, the processing equipment can reduce the brightness difference between dark and non-dark areas and weaken the spread of darkness in adjacent areas.

[0094] After color cast and occlusion processing are completed, the processing device can construct error equations for each color channel based on the corrected composite bird's-eye view. Solving the error equations for each color channel yields the adjustment coefficients for each color channel in each single-sided bird's-eye view. Based on these adjustment coefficients, the pixel values ​​of the single-sided bird's-eye view are corrected, resulting in a high-quality panoramic image.

[0095] Specifically, the panoramic image consists of n single-sided bird's-eye view images. The composite bird's-eye view image is represented by the numerical combination "ij", where i and j both belong to n, and i and j correspond to different single-sided bird's-eye view images. In this embodiment, the front bird's-eye view image is defined as 1, the left bird's-eye view image as 2, the right bird's-eye view image as 3, and the rear bird's-eye view image as 4. The definition of the composite bird's-eye view image is as follows:

[0096] (1) The front left bird's-eye view is 12, and the front right bird's-eye view is 13;

[0097] (2) The left front bird's-eye view is 21, and the left rear bird's-eye view is 24;

[0098] (3) The right front bird's-eye view is 31, and the right rear bird's-eye view is 34;

[0099] (4) The left rear bird's-eye view is 42, and the right rear bird's-eye view is 43.

[0100] The total number of pixels in the composite bird's-eye view "ij" is defined as Num. One error equation can be constructed for one overlapping region based on the composite bird's-eye view "ij" and the composite bird's-eye view "ji". Four overlapping regions can construct four error equations. After organizing and summarizing all the error equations, the following expression is obtained:

[0101] Where e represents the minimum error, This represents the adjustment factor for the channel of the single-sided bird's-eye view i. This represents the adjustment factor for the channel of a single-sided bird's-eye view j. The pixel mean of the channel in the composite image "ij". σ represents the pixel mean of the channel in the composite bird's-eye view "ji", where channel ∈ (R, G, B). N For color error, it is generally set to 10.0, σ g The standard deviation of the gain parameter is an empirical value, typically set to 0.1. Based on the above error equation formula, setting the minimum error partial derivative to 0, we obtain the following set of error equations, in the form of:

[0102] By using the elimination method to gradually simplify the process, we can eventually obtain the adjustment coefficient. The expression is of the form:

[0103] Where, φ index μ index ω index b index , λ index All are constant terms, with index ∈ [1, 16]. Substitute the data after image color cast or occlusion processing into the adjustment coefficients. The expression only requires four arithmetic operations to solve for each adjustment coefficient.

[0104] The processing device multiplies the pixel values ​​of the single-sided bird's-eye view by the corresponding adjustment coefficients to obtain a corrected single-sided bird's-eye view. Then, a fusion algorithm is used to fuse the corrected single-sided bird's-eye view to output a panoramic image.

[0105] definition For the channel of the single-sided bird's-eye view i, Let be the channel of the adjusted one-sided bird's-eye view i, where i∈[1,4]. The adjustment formula for the one-sided bird's-eye view is:

[0106] Using any fusion algorithm (such as alpha fusion), two composite bird's-eye view images for each overlapping region are merged to achieve a natural overlap effect. A common formula for alpha fusion is as follows: C overlap =αC ij +(1-α)C ji

[0107] Among them, C overlap Defined as the fused image, C ij and C ji The overlapping images “ij” and “ji” are defined respectively, and α is defined as the fusion factor, α∈[0,1]. The adjusted single-sided bird's-eye view and the fused composite bird's-eye view are placed on the corresponding coordinates to complete the panoramic view stitching, and finally output a high-quality panoramic image with balanced color and brightness.

[0108] Figure 8 is a flowchart illustrating another panoramic image processing method provided in an embodiment of this application. As shown in Figure 8, the method may include:

[0109] Step 801: Input the front, back, left, and right bird's-eye view images.

[0110] Step 802: Determine the composite bird's-eye view and calculate the channel mean.

[0111] Step 803: Determine whether the color shift is due to the channel mean. If so, proceed to step 805; otherwise, proceed to step 804.

[0112] Step 804: Determine whether the phenomenon is occlusion based on the channel average. If yes, proceed to step 806; otherwise, proceed to step 807.

[0113] Step 805: Perform color shift processing on the composite bird's-eye view.

[0114] Step 806: Obscuring the composite bird's-eye view.

[0115] Step 807: Construct the error equation and solve for the adjustment coefficient.

[0116] Step 808: Correct the single-sided bird's-eye view based on the adjustment coefficient and output the panoramic image.

[0117] The color shift phenomenon is the aforementioned color occlusion phenomenon, and the occlusion phenomenon is the aforementioned black occlusion phenomenon.

[0118] In this embodiment, the processing device performs two aspects of processing on the composite bird's-eye view: color shift processing ensures that the image color is not distorted during white balance adjustment, and occlusion processing reduces the impact of black shadows on global brightness during white balance adjustment, greatly improving the accuracy of white balance adjustment of the panoramic view.

[0119] The above processes can all be executed on lower-performance chips and processors, without relying on specialized mathematical libraries or complex hardware acceleration devices. The system of equations requiring minimal error can be solved using only basic arithmetic operations (addition, subtraction, multiplication, and division), avoiding high-overhead computations such as matrix operations and vector operations. Image color cast and occlusion detection are performed using the channel mean of a composite bird's-eye view, eliminating the need to traverse and filter pixels one by one. This approach satisfies both real-time performance and ensures the accuracy and stability of image processing even in environments with limited memory and computing power.

[0120] Figure 9 is a schematic diagram of a panoramic image processing device provided in an embodiment of this application. As shown in Figure 9, the device may include:

[0121] The acquisition module 910 is used to acquire single-sided bird's-eye view images of different areas captured by the vehicle-mounted camera.

[0122] The determination module 920 is used to determine a composite bird's-eye view based on a single-sided bird's-eye view, wherein the composite bird's-eye view includes the overlapping area of ​​any two single-sided bird's-eye views.

[0123] The first correction module 930 is used to perform pixel correction on the composite bird's-eye view based on the occlusion processing algorithm.

[0124] The second correction module 940 is used to perform pixel correction on a single-sided bird's-eye view based on the corrected composite bird's-eye view and output a panoramic image.

[0125] Corresponding to the above embodiments, this application also provides an electronic device. Figure 10 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 1000 may include: a processor 1001, a memory 1002, and a communication unit 1003. These components communicate through one or more buses. Those skilled in the art will understand that the structure of the electronic device shown in the figure does not constitute a limitation on the embodiments of this application. It can be a bus topology or a star topology, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0126] The communication unit 1003 is used to establish a communication channel, enabling the electronic device to communicate with other devices. It receives user data from other devices or sends user data to other devices.

[0127] The processor 1001 serves as the control center of the electronic device, connecting various parts of the device via interfaces and lines. It executes software programs, instructions, and / or modules stored in the memory 1002, and calls data stored in the memory to perform various functions and / or process data. The processor may be composed of integrated circuits (ICs), such as a single packaged IC or multiple packaged ICs with the same or different functions connected together. For example, the processor 1001 may consist only of a central processing unit (CPU). In this embodiment, the CPU may have a single processing core or include multiple processing cores.

[0128] The memory 1002 is used to store the execution instructions of the processor 1001. The memory 1002 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0129] When the execution instructions in memory 1002 are executed by processor 1001, the electronic device 1000 is able to perform some or all of the steps in the above embodiments.

[0130] In a specific implementation, this application also provides a computer storage medium, wherein the computer storage medium may store a program, which, when executed, may include some or all of the steps of the panoramic image processing method provided in various embodiments of this application. The storage medium may be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0131] In a specific implementation, this application also provides a computer program product, wherein the computer program product includes executable instructions, which, when executed on a computer, cause the computer to perform some or all of the steps in various embodiments of the panoramic image processing method provided in this application.

[0132] This application also provides a non-transitory computer-readable storage medium that stores computer instructions, which cause the computer to execute the panoramic image processing method provided in this application.

[0133] The aforementioned non-transitory computer-readable storage medium may take any combination of one or more computer-readable media. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory, optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in connection with an instruction execution system, apparatus, or device.

[0134] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including—but not limited to—electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0135] The program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0136] Those skilled in the art will clearly understand that the techniques in the embodiments of this application can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of this application, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application or some parts of the embodiments.

[0137] The same or similar parts between the various embodiments in this specification can be referred to mutually. In particular, the device embodiments and terminal embodiments are basically similar to the method embodiments, so the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

Claims

1. A panoramic image processing method, characterized by, include: Acquire one-sided bird's-eye view images of different areas captured by the vehicle-mounted camera; A composite bird's-eye view is determined based on the single-sided bird's-eye view, and the composite bird's-eye view includes the overlapping area of ​​any two single-sided bird's-eye views; The composite bird's-eye view is pixel-corrected based on an occlusion processing algorithm; The single-sided bird's-eye view is pixel-corrected based on the corrected composite bird's-eye view, and a panoramic image is output.

2. The method of claim 1, wherein, The pixel correction of the composite bird's-eye view based on the occlusion processing algorithm includes: In response to the presence of colored obstacles obstructing the vehicle-mounted camera, the pixel values ​​of the composite bird's-eye view are subjected to color shift correction processing based on the first occlusion processing algorithm; In response to the presence of a black obstacle obstructing the vehicle-mounted camera, the pixel values ​​of the composite bird's-eye view are corrected for occlusion based on a second occlusion processing algorithm.

3. The method of claim 2, wherein, The step of performing color shift correction processing on the pixel values ​​of the composite bird's-eye view based on the first occlusion processing algorithm includes: The presence of color cast in the composite bird's-eye view is determined based on the pixel mean of each color channel. When it is determined that the composite bird's-eye view has the color shift phenomenon, the pixel values ​​of each color channel of the composite bird's-eye view are converted to grayscale. When it is determined that the composite bird's-eye view does not have the color shift phenomenon, the pixel values ​​of each color channel of the composite bird's-eye view are not processed.

4. The method of claim 2, wherein, The occlusion correction processing of the pixel values ​​of the composite bird's-eye view based on the second occlusion processing algorithm includes: Based on the grayscale values ​​of each composite bird's-eye view, each composite bird's-eye view is divided into dark areas or non-dark areas; For any composite bird's-eye view divided into dark areas, the grayscale value of the current composite bird's-eye view is corrected based on the grayscale values ​​of adjacent composite bird's-eye views.

5. The method of claim 4, wherein, The process of correcting the grayscale value of the current composite bird's-eye view based on the grayscale values ​​of adjacent composite bird's-eye views includes: If the two composite bird's-eye views adjacent to the current composite bird's-eye view are both non-dark areas, then the average gray value of the current composite bird's-eye view and the two adjacent composite bird's-eye views is determined as the gray value of the current composite bird's-eye view. If any composite bird's-eye view adjacent to the current composite bird's-eye view is a non-dark area, then the average gray value of the current composite bird's-eye view and the adjacent composite bird's-eye view that is classified as a non-dark area is determined as the gray value of the current composite bird's-eye view.

6. The method according to claim 1, characterized in that, The step of performing pixel correction on the single-sided bird's-eye view based on the corrected composite bird's-eye view and outputting a panoramic image includes: Based on the corrected composite bird's-eye view, error equations for each color channel are constructed; Solve the error equations for each color channel to obtain the adjustment coefficients for each color channel in each single-sided bird's-eye view; The pixel values ​​of the single-sided bird's-eye view are corrected based on the adjustment coefficient, and a panoramic image is output.

7. The method according to claim 6, characterized in that, The step of correcting the pixel values ​​of the single-sided bird's-eye view based on the adjustment coefficient and outputting a panoramic image includes: The pixel values ​​of the single-sided bird's-eye view are multiplied by the corresponding adjustment coefficients to obtain the corrected single-sided bird's-eye view. The overlapping areas of the corrected single-sided bird's-eye view are fused using a fusion algorithm to output the panoramic image.

8. A panoramic image processing device, characterized in that, include: The acquisition module is used to acquire one-sided bird's-eye view images of different areas captured by the vehicle-mounted camera; A determining module is used to determine a composite bird's-eye view based on the single-sided bird's-eye view, wherein the composite bird's-eye view includes the overlapping area of ​​any two single-sided bird's-eye views; The first correction module is used to perform pixel correction on the composite bird's-eye view based on the occlusion processing algorithm. The second correction module is used to perform pixel correction on the single-sided bird's-eye view based on the corrected composite bird's-eye view and output a panoramic image.

9. An electronic device, characterized in that, The device includes a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device performs the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the method according to any one of claims 1 to 7.

Images