Method and system for calculating vehicle panoramic surround view stitching image quality based on statistics

Abstract

The present application provides a statistically-based method and system for calculating the quality of a vehicle's panoramic surround view stitching image. The method comprises the following steps: 1) using a deep learning method to perform edge detection on the vehicle's panoramic surround view image; 2) masking the image at the vehicle's location, processing the edge detection results, and performing binary thresholding on the results to convert the grayscale image into a black and white image; 3) performing image segmentation on the edge detection results and extracting all coordinates of each checkerboard area from them; 4) using a clustering algorithm to count the coordinates of the checkerboard area and using a polynomial to fit the curve equation of the checkerboard edge; 5) calculating the number and quadrilateral coordinates of each checkerboard in the stitching image based on the curve equation of the checkerboard edge; 6) combining the curve equation, quadrilateral coordinates, and checkerboard number to calculate image stitching quality indicators, including picture symmetry, stitching misalignment, stitching loss, and stitching ghosting. The present application has the advantages of strong interpretability, low deployment cost, and good robustness.

Description

Technical Field

[0001] The present application relates to the field of stitching image quality calculation, and in particular to a method for calculating vehicle panoramic surround view stitching images. Background Art

[0002] In recent years, with the rapid development of road traffic and the transportation industry, the number of vehicles has increased rapidly, leading to increasingly serious issues such as driving safety and traffic disputes. Drivers' blind spots can easily lead to misjudgment and operational errors, resulting in frequent traffic accidents such as vehicle engulfment and rear-end collisions. A vehicle's blind spot refers to the area that is obscured from view by the driver from the driver's seat. To address these issues, panoramic surround-view technology has rapidly developed and holds broad application prospects in the field of assisted driving. Using wide-angle cameras mounted around the vehicle, panoramic surround-view technology reconstructs a bird's-eye view of the vehicle and surrounding scene through image transformation, eliminating blind spots and ensuring safe driving.

[0003] Currently, the quality evaluation method for panoramic surround stitching images mainly uses manual means to evaluate the quality, which has the following four defects: (1) environmental conditions have a significant impact on the results; (2) it is affected by human factors; (3) the evaluation process is cumbersome and inefficient; (4) it requires a lot of material and human resources, which is not conducive to practical application. Therefore, in view of the above technical problems, it is necessary to use an objective, automated, and robust method to evaluate the quality of panoramic surround stitching images.

[0004] Patent CN113191952A proposes a deep learning-based method for evaluating stitched images from 360-degree panoramic view devices. This method uses the YOLOv3 object detection model as its core method, builds and trains the corresponding model, and then uses the trained YOLOv3 model for detection and evaluation. However, this method requires a large dataset and is expensive to train and deploy.

[0005] Patent CN115239724A proposes a method for analyzing and evaluating 360-degree panoramic stitched images, introducing the concepts of stitching smoothness and stitching uniformity, expanding the evaluation dimensions of panoramic stitched images. However, this method treats the stitching position as known information, and its input algorithm, stitching algorithm, and evaluation algorithm are highly coupled, making it impossible to apply this evaluation method to any panoramic stitched image. Summary of the Invention

[0006] In view of this, the purpose of this application is to propose a statistically based method and system for calculating the quality of vehicle panoramic surround view stitching images, which can specifically solve the existing problems.

[0007] Based on the above objectives, this application proposes a statistical method for calculating the quality of vehicle panoramic surround view stitching images, including:

[0008] 1) Use deep learning methods to perform edge detection on vehicle panoramic surround images;

[0009] 2) masking the image at the vehicle's location with black pixels, globally preprocessing the edge detection results, and performing binary thresholding on the edge detection results to convert the grayscale image into a black and white image;

[0010] 3) Perform image segmentation on the edge detection results and extract all coordinates of each checkerboard area;

[0011] 4) Using a clustering algorithm to count the coordinates of the checkerboard area, and using a polynomial to fit the curve equation of the checkerboard edge;

[0012] 5) Calculate the number and quadrilateral coordinates of each chessboard square in the stitched image using the curve equation of the chessboard square edge;

[0013] 6) Combining the curve equation, quadrilateral coordinates and checkerboard grid numbers, respectively calculate image stitching quality indicators, including picture symmetry, stitching dislocation, stitching loss, and stitching ghosting.

[0014] Furthermore, in step 1), the edge detection algorithm is any deep learning model used to solve edge detection problems.

[0015] Furthermore, in the step 2), the global preprocessing uses a method including Gaussian filter, mean filter, median filter, dilation, and erosion to perform global preprocessing on the input image.

[0016] Furthermore, in the step 3), the edge detection result is first segmented, and then the extracted areas are sorted in order of area from large to small. Finally, all areas representing the checkerboard are screened out according to the area size relationship of adjacent areas, and all coordinates within the checkerboard area are output.

[0017] Furthermore, in step 4), the K-means clustering algorithm is first used to count all coordinates within the checkerboard area; then a polynomial is used to fit the curve of the center point of each category; and finally, an interpolation method is used to calculate the curve equation of the checkerboard edge.

[0018] Furthermore, in step 5), the intersection points between each pair of horizontal and vertical curves are first calculated based on the curve equations of the chessboard edges; then, the calculated intersection points are used to represent chessboard quadrilaterals using four intersection points obtained by combining two adjacent pairs of curves, and the chessboard quadrilaterals are numbered to obtain relative chessboard numbers represented by rows and columns; finally, based on the relative number and absolute number of the given special red chessboard in the spliced image, the relative numbers of all chessboard squares are corrected to absolute numbers.

[0019] Furthermore, in the step 6), the symmetry index of the picture is calculated using the fitted straight line equation, the Euclidean distance of the coordinates of the chessboard quadrilaterals with uniform numbering and different viewing angles is used to calculate the splicing misalignment index, the intersection area of the chessboard quadrilaterals is used to calculate the splicing loss index, and the difference area of the chessboard quadrilaterals is used to calculate the splicing ghosting index.

[0020] Based on the above objectives, this application also proposes a statistics-based vehicle panoramic surround view stitching image quality calculation system, including:

[0021] The edge detection module uses deep learning methods to perform edge detection on the vehicle's panoramic surround view image;

[0022] a black-and-white image conversion module that masks the image at the vehicle's location with black pixels, globally preprocesses edge detection results, and performs binary thresholding on the edge detection results to convert the grayscale image into a black-and-white image;

[0023] Image segmentation module, which performs image segmentation on the edge detection results and extracts all the coordinates of each checkerboard area;

[0024] The curve fitting module uses a clustering algorithm to count the coordinates of the checkerboard area and uses a polynomial to fit the curve equation of the checkerboard edge;

[0025] The numbering and coordinate calculation module calculates the number and quadrilateral coordinates of each chessboard square in the spliced image through the curve equation of the chessboard square edge;

[0026] The quality calculation module calculates the image stitching quality indicators, including picture symmetry, stitching dislocation, stitching loss, and stitching ghosting, by combining the curve equation, quadrilateral coordinates, and chessboard grid numbers.

[0027] Compared with the prior art, this application has the following advantages:

[0028] 1. This application replaces manual inspection of the stitched image quality of panoramic view equipment and provides an objective and scientific method for judging the imaging quality of panoramic view equipment;

[0029] 2. This application uses a statistical method to calculate evaluation indicators. The calculation process and results are highly interpretable and the deployment cost is low.

[0030] 3. This application is based on a deep learning-based edge detection method and performs post-processing on the edge detection result image, thereby reducing the impact of interference factors such as noise in the panoramic view stitching image on the index calculation and improving the robustness of the algorithm for the panoramic view stitching image. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar components or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments disclosed in this application and should not be construed as limiting the scope of this application.

[0032] Figure 1 Flowchart of the present application method.

[0033] Figure 2 This is an example of a vehicle panoramic surround view stitching image used to demonstrate the algorithm process of this application.

[0034] Figure 3 Schematic diagram of the output image of the edge detection step in the algorithm of this application.

[0035] Figure 4 Schematic diagram of the output image of the edge detection post-processing step in the algorithm of this application.

[0036] Figure 5 Schematic diagram of the image segmentation step in the algorithm of this application.

[0037] Figure 6 This is a schematic diagram of the fitting curve in the algorithm of this application.

[0038] Figure 7 This is a schematic diagram of the chessboard numbering in the algorithm of this application.

[0039] Figure 8 A diagram showing the structure of a vehicle panoramic surround view stitching image quality calculation system based on statistics according to an embodiment of the present application is shown.

[0040] Figure 9 A schematic structural diagram of an electronic device provided in one embodiment of the present application is shown.

[0041] Figure 10 A schematic diagram of a storage medium provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0042] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the relevant invention are shown in the accompanying drawings.

[0043] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0044] In order to make the purpose, technical solutions and advantages of this application more clear, the following embodiments are combined to illustrate the following. Figure 1 It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

[0045] Step 1: For a given vehicle panoramic surround stitching image I with a height of H and a width of W, input it into the trained edge detection model M based on deep learning to obtain the output edge detection result image I E =M(I). Figure 2 is a schematic diagram of the vehicle panoramic surround view stitching image I, Figure 3 is the edge detection result image I E Schematic diagram. It is worth noting that the edge detection model M here can be any deep learning model used to solve general edge detection problems, and does not rely on a specific algorithm. Furthermore, the edge detection process here does not use traditional edge detection operators such as Sobel and Canny, thereby improving the algorithm's robustness to the input image.

[0046] Step 2: Given the bounding box b of the vehicle's location car =(x0, y0, x1, y1), where x0 and y0 represent the horizontal and vertical coordinates of the upper left corner of the vehicle's position, and x1 and y1 represent the horizontal and vertical coordinates of the lower right corner of the vehicle's position. The edge detection result I is calculated using black pixels. E Mask the corresponding position in , that is:

[0047] I E [y0:y1, x0:y1]=0

[0048] Then, the input image is globally preprocessed using methods such as Gaussian filter, mean filter, median filter, dilation, and corrosion to reduce the noise in the edge detection results and improve the algorithm's robustness to noise in the input image. Finally, binary thresholding is performed to convert the grayscale image into a black and white image. B ,Right now:

[0049]

[0050] Wherein, T is the threshold value, and gray(x, y) represents the gray value of the pixel with the horizontal coordinate x and the vertical coordinate y. Figure 4 Shows the output image of step 2 B .

[0051] Step 3: The main purpose of this step is to segment the image and extract all the coordinates of each chessboard square. The specific process is as follows:

[0052] (1) Using the region growing method to segment the image, all adjacent black pixels belong to the same region, and the region set R = {R1, R2, ..., R M},in is composed of P in the same area i A set of black pixel coordinate points;

[0053] (2) The extracted region set R is divided into two groups according to the area (i.e. R i The number of elements of |R i |) Sort from largest to smallest, and get

[0054] (3) For satisfaction The smallest k a ∈[1, M], k a is the number of the first checkerboard area;

[0055] (4) For satisfaction The smallest k b ∈(k a , M], k b is the number of the last checkerboard area;

[0056] (5) Regional collection Represents all checkerboard regions of similar size in image I.

[0057] Figure 5 Shows step three for image I B The result of image segmentation and checkerboard screening on the upper left corner of the image, where the white pixels represent the selected checkerboard coordinates.

[0058] Step 4: The main purpose of this step is to use the clustering algorithm to count the coordinates of the checkerboard area given in step 3, and use a polynomial to fit the curve equation of the checkerboard edge. The specific process is as follows:

[0059] (1) Use the K-means clustering algorithm to count all coordinates in the chessboard area, divide the horizontal and vertical coordinates of the center point into N categories, and record the clustering results as in And record the category to which each center point coordinate belongs;

[0060] (2) Use polynomials to fit the curve of the center point of each category, and obtain N polynomial curves that pass through their corresponding center points. Taking the quadratic equation as an example, the polynomial Fitting a transverse K i The corresponding curves of several center points, polynomials Fitting a longitudinal path through K j The curves corresponding to several center points;

[0061] (3) Use interpolation to calculate the curve equation of the edge of the chessboard. By taking the mean of the parameters of the two curves, we can get the curve in the middle of the two rows of chessboard. For example, for adjacent polynomials and The corresponding polynomial can be obtained by interpolation:

[0062]

[0063] Similarly, by subtracting the constant terms of several adjacent curves, we can obtain the curve on a certain side of the chessboard. Figure 6 This is a schematic diagram of the curves fitted in step 4 in the upper left corner of the stitched image. You can see that these curves fit the edges of the checkerboard in the image.

[0064] Step 5: The main purpose of this step is to calculate the coordinates and number of each square in the stitched image using the curve equation given in step 4. The specific process is as follows:

[0065] (1) For the curves given in step 4, calculate the intersection points between each pair of horizontal and vertical curves by solving two equations simultaneously and calculating the polynomial solution. Taking the quadratic curve as an example, the formula is as follows:

[0066]

[0067] After combining these two equations, we get four complex solutions, from which we select the real roots (x, y) that fall within the image coordinate range;

[0068] (2) The intersection point (x, y) obtained by calculation is used to represent the checkerboard quadrilateral [(x1, y1), (x2, y2), (x3, y3), (x4, y4)] by the four intersection points obtained by combining two adjacent pairs of curves, and they are numbered to obtain the relative number of the checkerboard represented by (row-column) (i r ,j r );

[0069] (3) According to the relative number and absolute number of the given positioning chessboard in the spliced image, the relative number (i r ,j r ) is corrected to absolute number (i a ,j a ). In the vehicle's panoramic view system, K cameras are installed in different directions of the vehicle to capture images in front, behind, left, right, and other directions of the vehicle. In order to facilitate the calculation of various splicing indicators, when arranging the test scene, it is necessary to select a special chessboard for positioning from all the chessboards in the K directions, which is called a positioning chessboard. There are a total of K positioning chessboards in the entire scene. The chessboard at the center of the direction can be selected as the positioning chessboard, or the selection rules can be formulated according to the actual situation. Let the relative number of a positioning chessboard be (Ri r , Rj r ), absolute number (Ri a , Rj a ), then the absolute number of the ordinary chessboard square at this position is (i a ,j a )=(i a -Ri r +Ri a ,j a -Rj r +Rj a ).

[0070] Figure 7 This is a schematic diagram of the chessboard number output in step 5 in the upper left corner of the stitched image.

[0071] Step 6: The main purpose of this step is to combine the curve equation, quadrilateral coordinates, and checkerboard labels obtained in steps 4 and 5 to calculate the image stitching quality indicators, including picture symmetry, stitching misalignment, stitching loss, and stitching ghosting.

[0072] The calculation method of the screen symmetry index is as follows: According to the clustering result S given in step 1 of step 4 K , use a linear equation to fit the center point of each category, and then calculate the inclination angle α of each straight line in the four areas of top, bottom, left, and right x or α y , and finally take the maximum tilt angle in each area As the inclination angle of the area. Among them, the longitudinal inclination angle α x With the lateral inclination angle α y Calculated by the following formula, k x is the slope of the vertical line, k y is the slope of the horizontal line:

[0073]

[0074]

[0075] The calculation method of the splicing misalignment index is as follows: Based on the chessboard numbers and quadrilateral coordinates given in step 5, calculate the sum of the distances between each pair of diagonally adjacent chessboard squares (for example, a pair of chessboard squares numbered (3,5) and (4,6)), which is the total length of the splicing error. Among them, for two given chessboard square numbers, calculate the Euclidean distance of each pair of coordinates in their quadrilateral. There are 4*4=16 combinations in total, the minimum of which is the length of the splicing misalignment.

[0076] The calculation method of the splicing loss index is as follows: (1) According to the chessboard number and quadrilateral coordinates given in step 5, for a given target chessboard, let the set of all coordinates in the area enclosed by it be A, and the coordinate sets of the adjacent chessboards above, below, left and right of the chessboard be B respectively. up 、B down 、B left 、B right . Calculate the intersection A∩B between these sets up 、A∩B down 、A∩B left 、A∩B right , the coordinates in the intersection are the locations where there may be splicing loss (2) For the above four intersections, check whether the pixel grayscale value of each coordinate falls between the grayscale values of the black and white grids. If so, the pixel is the location of splicing loss (3) Count the total number of these splicing loss points, which is the area of splicing loss.

[0077] The calculation method of the splicing ghost index is as follows: (1) Based on the chessboard number and quadrilateral coordinates given in step 5, for two chessboard squares at the same position under different viewing angles, let the sets of all coordinates in the area enclosed by them be A and B respectively. Calculate the two difference sets AB and BA between A and B respectively. The coordinates in these two difference sets are the locations where splicing ghosts may exist. (2) For the difference sets AB and BA, check whether the pixel grayscale value of each coordinate in them falls between the grayscale values of the black and white squares. If so, the pixel is the location of the splicing ghost. (3) Count the total number of these splicing ghost points, which is the area of the splicing ghost.

[0078] The embodiment of the application provides a vehicle panoramic surround view stitching image quality calculation system based on statistics, which is used to execute the vehicle panoramic surround view stitching image quality calculation method based on statistics described in the above embodiment, such as Figure 8 As shown, the system includes:

[0079] The edge detection module 801 performs edge detection on the vehicle panoramic surround view image using a deep learning method;

[0080] A black and white image conversion module 802 masks the image at the location of the vehicle using black pixels, globally preprocesses the edge detection results, and performs binary thresholding on the edge detection results to convert the grayscale image into a black and white image;

[0081] The image segmentation module 803 performs image segmentation on the edge detection result and extracts all coordinates of each checkerboard area;

[0082] The curve fitting module 804 uses a clustering algorithm to count the coordinates of the checkerboard area and uses a polynomial to fit the curve equation of the checkerboard edge;

[0083] The numbering and coordinate calculation module 805 calculates the number and quadrilateral coordinates of each chessboard square in the spliced image using the curve equation of the chessboard square edge;

[0084] The quality calculation module 806 calculates the image stitching quality indicators, including picture symmetry, stitching misalignment, stitching loss, and stitching ghosting, based on the curve equation, quadrilateral coordinates, and chessboard grid numbers.

[0085] The statistics-based vehicle panoramic view stitching image quality calculation system provided in the above-mentioned embodiment of the present application and the statistics-based vehicle panoramic view stitching image quality calculation method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0086] The present application also provides an electronic device corresponding to the statistical-based vehicle panoramic surround view stitching image quality calculation method provided in the above embodiment, to execute the statistical-based vehicle panoramic surround view stitching image quality calculation method. This embodiment of the present application is not limited thereto.

[0087] Please refer to Figure 9 , which shows a schematic diagram of an electronic device provided by some embodiments of the present application. Figure 9 As shown, the electronic device 20 includes: a processor 200, a memory 201, a bus 202 and a communication interface 203, and the processor 200, the communication interface 203 and the memory 201 are connected via the bus 202; the memory 201 stores a computer program that can be run on the processor 200, and when the processor 200 runs the computer program, it executes the statistical-based vehicle panoramic surround view stitching image quality calculation method provided in any of the aforementioned embodiments of the present application.

[0088] The memory 201 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage. The system network element communicates with at least one other network element via at least one communication interface 203 (which may be wired or wireless), and may utilize the Internet, a wide area network, a local area network, a metropolitan area network, or the like.

[0089] The bus 202 may be an ISA bus, a PCI bus, or an EISA bus. The bus may be divided into an address bus, a data bus, a control bus, and the like. The memory 201 is used to store programs, and the processor 200 executes the programs upon receiving execution instructions. The statistically-based vehicle panoramic surround view stitching image quality calculation method disclosed in any of the aforementioned embodiments of the present application may be applied to or implemented by the processor 200.

[0090] The processor 200 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in the processor 200 or by software instructions. The above processor 200 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory 201 , and the processor 200 reads the information in the memory 201 and completes the steps of the above method in combination with its hardware.

[0091] The electronic device provided in the embodiment of the present application and the statistical-based vehicle panoramic surround view stitching image quality calculation method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, operated or implemented therein.

[0092] The present application also provides a computer-readable storage medium corresponding to the statistical vehicle panoramic surround view stitching image quality calculation method provided in the above embodiment. Please refer to Figure 10 The computer-readable storage medium shown is a CD 30, on which a computer program (i.e., a program product) is stored. When the computer program is run by the processor, it will execute the statistical-based vehicle panoramic view stitching image quality calculation method provided by any of the aforementioned embodiments.

[0093] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0094] The computer-readable storage medium provided in the above-mentioned embodiment of the present application and the statistical-based vehicle panoramic surround view stitching image quality calculation method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, run or implemented by the application program stored therein.

[0095] It should be noted that:

[0096] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems may also be used in conjunction with the teachings herein. Based on the above description, it is apparent that the structure required for constructing such systems is suitable. In addition, the present application is not directed to any specific programming language. It should be understood that various programming languages may be utilized to implement the present application described herein, and the description of specific languages above is provided for the purpose of disclosing the best mode of implementation of the present application.

[0097] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.

[0098] Similarly, it should be understood that in order to streamline the present application and aid in understanding one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, various features of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting an intention that the claimed application requires more features than are expressly recited in each claim. Rather, as reflected in the claims below, inventive aspects lie in fewer than all the features of the individual embodiments disclosed above. Accordingly, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim itself serving as a separate embodiment of the present application.

[0099] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition may be divided into multiple submodules or subunits or subcomponents. All features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed herein may be combined in any combination, except that at least some of such features and / or processes or units are mutually exclusive. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0100] Furthermore, those skilled in the art will appreciate that although some embodiments described herein include certain features included in other embodiments but not other features, combinations of features from different embodiments are intended to be within the scope of this application and to form different embodiments. For example, in the claims below, any of the claimed embodiments may be used in any combination.

[0101] The various component embodiments of the present application can be implemented in hardware, or implemented in a software module running on one or more processors, or implemented in a combination thereof. Those skilled in the art will appreciate that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all functions of some or all components in the creation system of the virtual machine according to an embodiment of the present application. The application can also be implemented as a part or all of the equipment or system program (for example, computer program and computer program product) for performing the method described herein. Such a program realizing the present application can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0102] It should be noted that the above embodiments illustrate rather than limit the present application, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbols placed between brackets should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present application may be implemented by means of hardware comprising several different elements and by means of appropriately programmed computers. In a unit claim enumerating several systems, several of these systems may be embodied by the same item of hardware. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names.

[0103] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various modifications or substitutions within the technical scope disclosed in this application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A statistical method for calculating the quality of vehicle panoramic surround view stitching images, characterized in that: include: 1) Use deep learning methods to perform edge detection on vehicle panoramic surround images; 2) masking the image at the vehicle's location with black pixels, globally preprocessing the edge detection results, and performing binary thresholding on the edge detection results to convert the grayscale image into a black and white image; 3) Perform image segmentation on the edge detection results and extract all coordinates of each checkerboard area; 4) Using a clustering algorithm to count the coordinates of the checkerboard area, and using a polynomial to fit the curve equation of the checkerboard edge; 5) Calculate the number and quadrilateral coordinates of each chessboard square in the stitched image using the curve equation of the chessboard square edge; 6) Combining the curve equation, quadrilateral coordinates and checkerboard grid numbers, respectively calculate image stitching quality indicators, including picture symmetry, stitching dislocation, stitching loss, and stitching ghosting.

2. The method for calculating the quality of vehicle panoramic surround view stitching images based on statistics according to claim 1, characterized in that: In step 1), the edge detection algorithm is any deep learning model used to solve edge detection problems.

3. The method for calculating the quality of vehicle panoramic surround view stitching images based on statistics according to claim 1, characterized in that: In the step 2), the global preprocessing uses methods including Gaussian filter, mean filter, median filter, dilation, and erosion to perform global preprocessing on the input image.

4. The method for calculating the quality of vehicle panoramic surround view stitching images based on statistics according to claim 1, characterized in that: In the step 3), the edge detection result is first segmented, and then the extracted areas are sorted in order from large to small. Finally, all areas representing the checkerboard are screened out according to the area size relationship of adjacent areas, and all coordinates within the checkerboard area are output.

5. The method for calculating vehicle panoramic surround view stitching image quality based on statistics according to claim 1, characterized in that: In step 4), the K-means clustering algorithm is first used to count all coordinates within the chessboard area; then a polynomial is used to fit the curve of the center point of each category; and finally, an interpolation method is used to calculate the curve equation of the chessboard edge.

6. The method for calculating the quality of vehicle panoramic surround view stitching images based on statistics according to claim 1, characterized in that: In step 5), first, according to the curve equation of the chessboard edge, the intersection points between each pair of horizontal curves and vertical curves are traversed and calculated; Then, the four intersection points obtained by combining two adjacent pairs of curves are used to represent the checkerboard quadrilaterals, and the checkerboard quadrilaterals are numbered to obtain the relative numbering of the checkerboard represented by rows and columns; finally, according to the relative numbering and absolute numbering of the given positioning checkerboard in the spliced image, the relative numbers of all checkerboards are corrected to absolute numbers.

7. The method for calculating vehicle panoramic surround view stitching image quality based on statistics according to claim 1, characterized in that: In the step 6), the symmetry index of the picture is calculated using the fitted straight line equation, the Euclidean distance of the coordinates of the chessboard quadrilaterals with uniform numbering and different viewing angles is used to calculate the splicing misalignment index, the intersection area of the chessboard quadrilaterals is used to calculate the splicing loss index, and the difference area of the chessboard quadrilaterals is used to calculate the splicing ghosting index.

8. A statistics-based vehicle panoramic surround view stitching image quality calculation system, characterized by: include: The edge detection module uses deep learning methods to perform edge detection on the vehicle's panoramic surround view image; a black-and-white image conversion module that masks the image at the vehicle's location with black pixels, globally preprocesses edge detection results, and performs binary thresholding on the edge detection results to convert the grayscale image into a black-and-white image; Image segmentation module, which performs image segmentation on the edge detection results and extracts all the coordinates of each checkerboard area; The curve fitting module uses a clustering algorithm to count the coordinates of the checkerboard area and uses a polynomial to fit the curve equation of the checkerboard edge; The numbering and coordinate calculation module calculates the number and quadrilateral coordinates of each chessboard square in the spliced image through the curve equation of the chessboard square edge; The quality calculation module calculates the image stitching quality indicators, including picture symmetry, stitching dislocation, stitching loss, and stitching ghosting, by combining the curve equation, quadrilateral coordinates, and chessboard grid numbers.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: The processor runs the computer program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the method according to any one of claims 1 to 7.

Images