Road panoramic splicing method for road condition simulation
Patent Information
- Application Number
- CN202611479153.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-22
- Publication Date
- 2026-10-27
AI Technical Summary
[0003]在现有技术中，道路全景拼接主要包括图像预处理、特征点检测与匹配、图像配准以及图像融合四个部分，其中特征点检测与匹配是图像拼接的重要部分，现有技术利用ORB算法（Oriented FAST and Rotated BRIEF）对不同视角的道路图像进行特征点检测与匹配，但是，由于路况模拟过程中道路图像内存在各种类别的关键点，包括车道线节点、路面箭头标识、路测标牌、绿化带等类别的关键点，其中绿化带区域内树叶以及灌木丛的灰度纹理具有重复性以及遮挡性的特点，容易加大不同视角的道路图像之间的误匹配率，而传统的ORB算法依赖于道路图像中的灰度差异选择关键点，无法准确地规避掉绿化带内可靠性较低的关键点，使得特征点检测与匹配的准确性较差，导致道路全景拼接后道路全景图像的质量较差
本申请通过筛选变化像素点，找到处于纹理变化区域的像素点，有助于后续提取绿化带内树叶以及灌木丛中的关键点；进而通过分析各变化像素点的近邻范围内的纹理特征的重复性，并结合各变化像素点的近邻范围内变化像素点的数目，对各道路灰度图像中各变化像素点的近邻范围内的重复性分布特征进行准确度量，有利于后续准确地规避掉路况模拟过程中绿化带内可靠性较低的关键点；
Smart Images
Figure CN122989439A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, specifically to a method for road panorama stitching for road condition simulation. Background Technology
[0002] With the development of computer vision, image stitching technology has been widely applied in vehicle safety. This technology stitches together road images captured by different cameras or from different perspectives to obtain a panoramic road image without blind spots, as required by driver assistance systems. Currently, to test the quality of the stitched panoramic road image, a six-axis motion platform is typically used for simulation. During the simulation, road images from different perspectives are acquired in real time and then stitched together to meet the stitching quality and real-time requirements of driver assistance systems.
[0003] In existing technologies, road panoramic stitching mainly includes four parts: image preprocessing, feature point detection and matching, image registration, and image fusion. Among these, feature point detection and matching is a crucial part of image stitching. Existing technologies utilize the ORB algorithm (Oriented Fast and Rotated BRIEF) to detect and match feature points in road images from different perspectives. However, during road condition simulation, various types of key points exist in road images, including lane line nodes, road surface arrow markings, roadside signs, and green belts. The grayscale textures of leaves and shrubs within the green belt area exhibit repetitive and occlusive characteristics, which can easily increase the mismatch rate between road images from different perspectives. Furthermore, the traditional ORB algorithm relies on grayscale differences in road images to select key points, which cannot accurately avoid key points with low reliability within the green belt, resulting in poor accuracy in feature point detection and matching and consequently, poor quality of the road panoramic image after stitching. Summary of the Invention
[0004] In view of the above, it is necessary to provide a road panoramic stitching method for road condition simulation. Compared with the traditional road panoramic stitching method for road condition simulation, this method improves the accuracy of matching between all road grayscale images by accurately avoiding key points with low reliability in the green belt during the road condition simulation process, thereby improving the quality of the road panoramic image after stitching.
[0005] The road panoramic stitching method for road condition simulation proposed in this application adopts the following technical solution: One embodiment of this application provides a method for road panorama stitching for road condition simulation, the method comprising the following steps: During the road condition simulation, grayscale images of the roads surrounding the simulated vehicle are collected synchronously and in real time. For all road grayscale images at any given time, the distribution difference of each pixel is obtained by comparing the neighborhood grayscale distribution of each pixel with its nearest neighbor pixels in each road grayscale image, and then each changing pixel is selected from each road grayscale image; by combining the degree of repetition and dispersion of the distribution difference of all changing pixels in the nearest neighbor range of each changing pixel in each road grayscale image with the number of changing pixels in the nearest neighbor range, the repetitive distribution degree of each changing pixel is obtained; The process involves obtaining each connected region within the nearest neighbor range, acquiring the significance coefficient of the number of each changing pixel based on the size of the connected regions within the nearest neighbor range, acquiring the discreteness significance of each changing pixel based on the dispersion of the neighborhood grayscale distribution of the upper edge pixels of each connected region within the nearest neighbor range, and acquiring the comprehensive feature value of each changing pixel based on the repetitive distribution degree. Then, it involves extracting each unusable keypoint from each road grayscale image; and extracting keypoints for stitching the road grayscale image from the pixels other than the unusable keypoints in each road grayscale image, thereby obtaining the panoramic road image at any given time.
[0006] In one embodiment, the process of obtaining the distribution difference is as follows: By analyzing the differences in gray values between each pixel and its multiple preset neighboring pixels, a gray-level difference vector for each pixel is obtained. The average value of the difference between the grayscale difference vectors of each pixel and all its preset nearest neighbor pixels is used as the distribution difference of each pixel.
[0007] In one embodiment, the process of filtering the changing pixels is as follows: Obtain the average level of the distribution difference of all pixels in each road grayscale image, and take each pixel in each road grayscale image whose distribution difference is greater than the average level as each changed pixel.
[0008] In one embodiment, the process of obtaining the repeatability distribution degree is as follows: In the distribution difference of all changing pixels within the nearest neighbor range, the frequency of occurrence of each distribution difference value is counted, and the information entropy and dispersion of the frequency of occurrence of all distribution difference values within the nearest neighbor range are calculated respectively. The repeatability distribution degree is inversely proportional to the information entropy and the dispersion, and directly proportional to the quantity.
[0009] In one embodiment, the method for calculating the repeatability distribution degree is as follows: The dispersion and the information entropy are mapped to a first positive number and a second positive number, respectively, and the repeatability distribution degree is the ratio of the quantity to the fusion result of the first positive number and the second positive number.
[0010] In one embodiment, the process of obtaining the numerical significance coefficient is as follows: The total number of pixels in the largest connected region within the nearest neighbor range of each changed pixel in each grayscale road image is counted. The ratio of the counted total number to the total number of pixels in the nearest neighbor range is used as the significance coefficient.
[0011] In one embodiment, the process of obtaining the discrete significance is as follows: Calculate the gradient magnitude of each edge pixel in each connected region and evaluate the dispersion of the gradient magnitude of all edge pixels in each connected region. Calculate the average of the evaluation results for all connected regions within the nearest neighbor range; The discrete significance is obtained by combining the average value with the numerical significance coefficient.
[0012] In one embodiment, the discrete significance is the product of the mean and the numerical significance coefficient.
[0013] In one embodiment, the comprehensive feature value is the sum of the normalized result of the repeatability distribution degree and the normalized result of the discrete significance degree.
[0014] In one embodiment, the extraction process of the unusable key points is as follows: The segmentation threshold is obtained by acquiring the comprehensive feature value of all changing pixels in each grayscale road image; Each pixel in the grayscale road image whose comprehensive feature value is greater than the segmentation threshold is considered an unusable keypoint.
[0015] This application has at least the following beneficial effects: This application identifies pixels in texture variation regions by filtering out changing pixels, which helps in the subsequent extraction of key points in leaves and shrubs within green belts. Furthermore, by analyzing the repetition of texture features within the neighborhood of each changing pixel and combining this with the number of changing pixels within the neighborhood of each changing pixel, the repetition distribution characteristics within the neighborhood of each changing pixel in the grayscale image of each road are accurately measured. This helps to accurately avoid key points with low reliability within green belts during road condition simulation. Furthermore, by analyzing the random occlusion features of leaves and shrubs in the green belt area of the road grayscale image, the discrete significance of each changing pixel in the road grayscale image is obtained, which helps to identify key points of leaves and shrubs in the green belt, thereby avoiding key points with low reliability in the green belt during subsequent processing. Furthermore, by fully considering the periodic distribution characteristics and random occlusion characteristics of leaves and shrubs within the green belt area, and combining the repetitive distribution degree and discrete significance of each changing pixel, the possibility of each changing pixel being a low-reliability key point within the green belt can be assessed more accurately. This allows for more accurate avoidance of low-reliability key points within the green belt during road condition simulation, improving the accuracy of matching between all road grayscale images, and ultimately enhancing the quality of the road panoramic image after panoramic stitching. Attached Figure Description
[0016] Figure 1 A flowchart illustrating the steps of a road panoramic stitching method for road condition simulation provided in this application; Figure 2 This is a schematic diagram illustrating the process of obtaining unavailable key points. Detailed Implementation
[0017] The following description, in conjunction with the accompanying drawings, details a specific scheme for a road panoramic stitching method for road condition simulation provided in this application.
[0018] This application provides a method for road panorama stitching for road condition simulation in one embodiment. Specifically, it provides the following method for road panorama stitching for road condition simulation. Please refer to [link to relevant documentation]. Figure 1 The method includes the following steps: Step 1: During the road condition simulation, simultaneously and in real time acquire various grayscale images of the roads surrounding the simulated vehicle.
[0019] The PreScan simulation software is used to build virtual simulation scenarios and provides a rich set of virtual sensor models. These virtual simulation scenarios can reflect highly realistic and diverse traffic conditions, including simulated traffic flow, road lane line nodes, road surface arrow markings, roadside signs, green belts, and other scene elements. The virtual sensor models include wide-angle cameras, LiDAR, and millimeter-wave radar, supporting researchers and developers in testing and optimizing autonomous driving technologies.
[0020] The simulation test of road conditions for the simulated vehicle is carried out in a virtual simulation scenario. The hardware for the simulation test includes a six-degree-of-freedom motion platform, which is a load-bearing platform supported by six electric cylinders. The load-bearing platform and the piston rod end of the electric cylinder are connected by six sets of Hooke hinges. The cylinder body end of the electric cylinder is also connected to the lower platform by six sets of Hooke hinges. The six electric cylinders are driven by servo motors.
[0021] During the road condition simulation test, wide-angle cameras are used to simultaneously and in real time capture various road images around the simulated vehicle.
[0022] In this embodiment, four road images around the simulated vehicle are acquired in real time. The four road images refer to the front view, rear view, left view, and right view of the simulated vehicle in the virtual simulation scene. The field of view of each road image is 210°.
[0023] Furthermore, taking any given moment as an example, preprocessing is performed on all road grayscale images acquired at that moment. Since the road images acquired by the wide-angle camera will have image distortion, causing distortion of road image features, the image correction method in OpenCV is used to correct the distortion of each road image. Then, the front view and back view after distortion correction are subjected to inverse projection transformation to eliminate the perspective effect in the front view and back view. After that, the left view and right view after distortion correction, as well as the front view and back view after inverse projection transformation, are converted to grayscale for subsequent faster feature point detection and matching, to obtain each road grayscale image after image preprocessing. Distortion correction and inverse projection transformation are well known techniques and will not be described in detail in this application.
[0024] In this embodiment, a weighted average method is used to grayscale the distortion-corrected left view, right view, and the inverse projection transformation front view and rear view. The weighted average method is a well-known technique and will not be described in detail here. As other implementation methods, based on the ability to grayscale the distortion-corrected left view, right view, and the inverse projection transformation front view and rear view, the implementer may use other existing feasible techniques. This application does not impose any special restrictions.
[0025] Step 2: For all road grayscale images at any given time, the distribution difference of each pixel is obtained by comparing the neighborhood grayscale distribution of each pixel with its nearest neighbor pixels in each road grayscale image. Then, each changing pixel is selected from each road grayscale image. By combining the degree of repetition and dispersion of the distribution difference of all changing pixels in the nearest neighbor range of each changing pixel in each road grayscale image with the number of changing pixels in the nearest neighbor range, the repetitive distribution degree of each changing pixel is obtained.
[0026] Traditional ORB (Oriented Fast and Rotated BRIEF) algorithms typically identify key points in each road image based on significant grayscale differences. However, they cannot accurately avoid key points with low reliability within green belts during road condition simulation. These low-reliability key points within green belts tend to increase the mismatch rate between road images from different perspectives, resulting in poor quality of the subsequent panoramic road image after stitching. Therefore, to improve the quality of panoramic road stitching, it is necessary to more accurately avoid key points with low reliability within green belts during road condition simulation.
[0027] Therefore, in order to extract the local texture features of each pixel in each road grayscale image, the grayscale difference vector of each pixel in each road grayscale image is obtained by the difference of grayscale values between each pixel and its preset multiple neighboring pixels, which reflects the local texture features of each pixel in each road grayscale image.
[0028] In this embodiment, the neighboring pixels of each pixel are the pixels within the 8-neighborhood of each pixel. The neighboring pixels of each pixel can be set by the implementer according to the actual situation, and this application does not impose any special restrictions.
[0029] In this embodiment, the process of obtaining the grayscale difference vector is as follows: each grayscale image of the road after image preprocessing is used as the input of the LBP algorithm (Local Binary Patterns). The LBP algorithm outputs an 8-bit binary vector of each pixel in each grayscale image of the road. The 8-bit binary vectors of each pixel are then used to form the grayscale difference vector of each pixel. The LBP algorithm is a well-known technology, and the specific process will not be described in detail. Based on the ability to characterize the difference in grayscale values between each pixel and its neighboring pixels, the implementer may use other existing technologies, and this application does not impose any special restrictions.
[0030] Furthermore, to analyze the repetitive distribution characteristics of the local textures of leaves and shrubs within the green belt, it is necessary to identify pixels within areas of texture variation. The difference in grayscale difference vectors between each pixel in each road grayscale image and its preset nearest neighbors is calculated. The average level of these differences is taken as the distribution difference of each pixel in each road grayscale image. Here, the average level is the mean. The larger the calculated distribution difference, the more likely the pixels in each road grayscale image are located within areas of texture variation, which include areas of leaves and shrubs within the green belt.
[0031] In this embodiment, the nearest neighbor pixels of each pixel are the pixels within the 8-neighborhood of each pixel. The nearest neighbor pixels of each pixel can be set by the implementer according to the actual situation, and this application does not impose any special restrictions.
[0032] It should be noted that: the degree of difference represents the degree of difference between variables, which can be calculated using techniques such as Hamming distance, Edit Distance (ED), and Euclidean distance. This application does not impose any special restrictions on this.
[0033] In this embodiment, the difference between grayscale difference vectors is the Hamming distance.
[0034] To further identify the repetitive distribution characteristics of leaves and shrubs in the green belt during the road condition simulation, the average level of distribution difference of all pixels in each road grayscale image was calculated, and each pixel in each road grayscale image with a distribution difference greater than the average level was taken as a variable pixel.
[0035] Furthermore, generally speaking, the more varied the number of varied pixels in the neighborhood of each varied pixel, the more it matches the grayscale variation characteristics of the green belt area in the road grayscale image. At the same time, in the distribution difference of all varied pixels in the neighborhood of each varied pixel, the frequency of occurrence of each distribution difference value is counted. If the information entropy of the frequency of occurrence of all distribution difference values in the neighborhood of each varied pixel is smaller, and the frequency distribution of all distribution difference values in the neighborhood of each varied pixel is more concentrated, it indicates that the repetitive distribution characteristics of the local texture features in the neighborhood of each varied pixel are more significant. Then, each varied pixel is more likely to belong to the key points of leaves and shrubs in the green belt.
[0036] Based on the above analysis, the dispersion of the frequency of occurrence of all distribution difference values within the nearest neighbor range of each changed pixel is calculated. The smaller the dispersion, the more concentrated the frequency distribution of all distribution difference values within the nearest neighbor range of each changed pixel.
[0037] It should be noted that: dispersion represents the degree of unevenness in the distribution of variables, which can be calculated using techniques such as standard deviation, variance, and coefficient of variation. This application does not impose any special restrictions on this.
[0038] In this embodiment, the dispersion of the frequency of occurrence is the standard deviation.
[0039] Furthermore, by analyzing the degree of repetition and dispersion of the distribution differences among all changing pixels within the nearest neighbor range of each changing pixel in each road grayscale image, and combining this with the number of changing pixels within the nearest neighbor range of each changing pixel, the repetitive distribution degree of each changing pixel in each road grayscale image is obtained. Specifically: The dispersion of the frequency of occurrence of all distribution difference values within the nearest neighbor range of each changed pixel is mapped to a first positive number, and the information entropy is mapped to a second positive number. The fusion result of the first positive number and the second positive number is obtained. The repeatability distribution degree of each changed pixel in each road grayscale image is: the ratio of the number of changed pixels within the nearest neighbor range of each changed pixel to the fusion result.
[0040] It should be noted that fusion refers to combining multiple variables in a way that enhances the overall effect. Specifically, it can be calculated by summing, multiplying, etc. This application does not impose any restrictions on the comparison.
[0041] It should be noted that the purpose of mapping variables to positive numbers is to avoid denominators of 0. Specifically, this can be done by calculating the sum of the variable and a preset value greater than 0, or by treating the variable as the exponent of an exponential function with the natural constant as the base. This application does not impose any special restrictions on this.
[0042] In this embodiment, the nearest neighbor range of each changing pixel is a square window with a side length of 32 centered on each pixel. Here, 32 is only one embodiment of this application. Implementers can set the side length of the square window according to the actual situation. This application does not impose any special restrictions.
[0043] In this embodiment, the expression for the repetition distribution degree of each changing pixel in each road grayscale image is: In the formula, This represents the repetitive distribution degree of the i-th changing pixel in a single grayscale image of a road; The information entropy represents the nearest neighbor range of the i-th changing pixel in a single grayscale road image; The standard deviation represents the frequency of occurrence of all distribution difference values within the nearest neighbor range of the i-th changing pixel in a single grayscale road image; This represents the number of changing pixels within the nearest neighbor range of the i-th changing pixel in a single grayscale road image. This indicates a preset value greater than 0, used to avoid a denominator of 0. The value of affects the calculation results of the repeatability distribution degree. In this embodiment, values are taken within a relatively small data range (0.01, 0.1). The value is 0.05. The fusion result of the first positive number and the second positive number is... The result of mapping the discreteness to a positive number is: The result of mapping information entropy to a positive number is: .
[0044] It should be noted that the repeatability distribution degree is used to reflect the repeatability distribution characteristics of each changed pixel within its nearest neighbor range. The larger the calculated repeatability distribution degree, the more significant the repeatability distribution characteristics of the local texture features within the nearest neighbor range of each changed pixel. In this case, each changed pixel is more likely to belong to the key points of leaves and shrubs in the green belt. In order to avoid increasing the mismatch rate between road images from different perspectives, it is necessary to accurately avoid the key points with low reliability in the green belt during the road condition simulation process.
[0045] Step 3: Obtain each connected region within the nearest neighbor range. Based on the size of the connected regions within the nearest neighbor range, obtain the significance coefficient of the number of each changing pixel. Combined with the dispersion of the neighborhood grayscale distribution of the upper edge pixels of each connected region within the nearest neighbor range, obtain the dispersion significance of each changing pixel. Then, combined with the repetition distribution degree, obtain the comprehensive feature value of each changing pixel. Subsequently, extract each unusable key point from each road grayscale image. Extract key points for stitching the road grayscale image from the pixels other than the unusable key points in each road grayscale image.
[0046] Due to the diversity and complexity of scene elements in road grayscale images, in order to more accurately avoid key points with low reliability in the green belt during road condition simulation, it is necessary to analyze the occlusion characteristics of the grayscale texture of leaves and shrubs in the green belt area.
[0047] Therefore, we obtain all connected regions within the nearest neighbor range of each changing pixel in each road grayscale image, and calculate the gradient magnitude of each edge pixel in each connected region.
[0048] In this embodiment, the nearest neighbor range of each changing pixel in each road grayscale image is used as the input of the seed-filling method, and all connected components within the nearest neighbor range of each changing pixel are output. The seed-filling method is a well-known technology and will not be described in detail in this application. As other implementation methods, based on the ability to obtain all connected components within the nearest neighbor range of each changing pixel, the implementer may adopt other existing feasible technologies, and this application does not impose any special restrictions.
[0049] In this embodiment, the Sobel operator is used to calculate the gradient magnitude of each edge pixel. The Sobel operator is a well-known technology and will not be described in detail here. As other implementation methods, based on the ability to calculate the gradient magnitude of each edge pixel, the implementer may use other existing technologies, such as the Roberts operator, the Prewitt operator, etc. This application does not impose any special restrictions.
[0050] Generally, in a grayscale image of a road, the gradient magnitudes of all edge pixels in the connected domain formed by lane line nodes, road surface arrow markings, and roadside signs are relatively consistent. However, the gradient magnitudes of all edge pixels in the connected domain formed by leaves and shrubs in the green belt area are more chaotic, which can clearly reflect the occlusion characteristics of the grayscale texture of leaves and shrubs in the green belt area.
[0051] Therefore, the dispersion of the gradient magnitude of all edge pixels in each connected region within the nearest neighbor range of each changed pixel is recorded as the gradient dispersion of each connected region within the nearest neighbor range of each changed pixel. The larger the gradient dispersion of all connected regions within the nearest neighbor range, the more it reflects the occlusion characteristics of the grayscale texture of leaves and shrubs in the green belt area, and the more likely each changed pixel is to belong to the key point of leaves and shrubs in the green belt.
[0052] At the same time, because the leaves and shrubs in the green belt area will randomly block the view, and different leaves and shrubs will show different degrees of shading, for example, some leaves are completely blocked, forming a large area of shadow or homogeneous area, while others are only partially blocked, etc., so there is often a connected domain in the vicinity that is significantly larger than other areas.
[0053] Therefore, the total number of pixels in the largest connected region within the nearest neighbor range of each changed pixel in each grayscale image of the road is counted, and the ratio of the counted total to the total number of pixels in the nearest neighbor range is used as the numerical significance coefficient. The larger the numerical significance coefficient, the more it reflects the existence of large areas of continuous shading or homogeneous regions within the green belt, and the more it matches the characteristics of random shading by leaves and shrubs within the green belt. The more likely each changed pixel is to belong to a key point of leaves and shrubs within the green belt.
[0054] Based on the above analysis, the mean of the gradient dispersion of all connected regions within the nearest neighbor range of each changed pixel in each road grayscale image is calculated. The product of the mean and the numerical significance coefficient is used as the dispersion significance of each changed pixel in each road grayscale image. The larger the dispersion significance is, the more it reflects the random occlusion characteristics of leaves and shrubs in the green belt area of the road image. In this case, each changed pixel is more likely to belong to the key point of leaves and shrubs in the green belt.
[0055] Therefore, in order to more accurately avoid key points with low reliability in the green belt during road condition simulation, the green belt features of each changing pixel in the road grayscale image are measured by combining the repeatability distribution degree and discrete significance of each changing pixel.
[0056] Specifically, the repeatability distribution degree and discrete significance of all changing pixels in each road grayscale image are normalized respectively. The sum of the normalized results of the repeatability distribution degree and the discrete significance of each changing pixel is used as the comprehensive feature value of each changing pixel in each road grayscale image. The larger the calculated comprehensive feature value, the more likely each changing pixel belongs to the green belt area in the road grayscale image. Therefore, each changing pixel is more likely to be a key point of leaves and shrubs in the green belt.
[0057] In this embodiment, the Sigmoid function is used to normalize the repeatability distribution degree and the discrete significance degree, respectively. The Sigmoid function is a well-known technique and will not be described in detail in this application.
[0058] Furthermore, a segmentation threshold is obtained by analyzing the comprehensive feature values of all changing pixels in each road grayscale image. Each changing pixel in each road grayscale image whose comprehensive feature value is greater than the segmentation threshold is designated as an unusable keypoint. A schematic diagram of the process for obtaining unusable keypoints is shown below. Figure 2 As shown.
[0059] In this embodiment, the Otsu threshold segmentation algorithm is used to obtain the segmentation threshold of the comprehensive feature value of all changing pixels in each road grayscale image. The Otsu threshold segmentation algorithm is a well-known technology and will not be described in detail in this application. As other implementation methods, based on the ability to obtain the segmentation threshold of the comprehensive feature value of all changing pixels in each road grayscale image, the implementer may use other existing technologies, such as iterative threshold segmentation, global threshold segmentation, etc. This application does not impose any special restrictions.
[0060] Furthermore, to more accurately avoid key points with low reliability within the green belt during road condition simulation, each road grayscale image is used as input to the ORB algorithm (Oriented FAST and Rotated BRIEF). The ORB algorithm employs the Oriented FAST operator to detect key points in each road grayscale image, excluding unusable key points, thus identifying the key points in each image. It then uses the Rotated BRIEF descriptor to perform feature recognition on the identified key points in each image, obtaining feature descriptors for the key points in each image. Finally, a brute-force matching method is used to match overlapping areas between all road grayscale images, with Hamming distance as the matching distance. The ORB algorithm outputs matching pairs between all road grayscale images. The ORB algorithm is a well-known technique and will not be elaborated upon in this application.
[0061] Step 4: Extract key points for stitching the road grayscale images from the pixels in each road grayscale image, excluding the unusable key points, to obtain the road panoramic image at any given time.
[0062] To ensure the alignment accuracy of overlapping areas and the smooth transition of non-overlapping areas among all road grayscale images, matching pairs between all road grayscale images are used as input to the K-nearest neighbor algorithm. The K-nearest neighbor algorithm filters out unreasonable matching pairs. The remaining matching pairs are then iterated through using the RANSAC (Random Sample Consensus) algorithm until the optimal homography matrix required for projection between all road grayscale images is calculated, thus achieving the initial stitching between all road grayscale images. Both the K-nearest neighbor algorithm and the RANSAC algorithm are well-known techniques, and their specific processes will not be elaborated further.
[0063] Furthermore, image fusion is performed on the road panoramic image after preliminary stitching of all road grayscale images. The purpose is to eliminate overlapping areas, achieve smooth transitions in non-overlapping areas, and compensate for some misalignment. The preliminary stitched road panoramic image is used as input to the Laplacian pyramid fusion algorithm. The Laplacian pyramid fusion algorithm first decomposes the preliminary stitched road panoramic image into a series of low-pass filtered images, then fuses the filtered images belonging to a spatial frequency domain band into corresponding stitched sub-images, and finally fuses the stitched sub-images of all frequency bands into a panoramic image. The Laplacian pyramid fusion algorithm outputs the road panoramic image after stitching, eliminating the stitching seams present in the preliminary stitched road panoramic image. The Laplacian pyramid fusion algorithm is a known technique and will not be described in detail in this application.
[0064] In summary, this application identifies pixels in texture variation regions by filtering out changing pixels, which helps in the subsequent extraction of key points in leaves and shrubs within green belts. Furthermore, by analyzing the repetition of texture features within the neighborhood of each changing pixel and combining this with the number of changing pixels within the neighborhood of each changing pixel, the repetition distribution characteristics within the neighborhood of each changing pixel in the grayscale image of each road are accurately measured. This helps to accurately avoid key points with low reliability within green belts during road condition simulation. Furthermore, by analyzing the random occlusion features of leaves and shrubs in the green belt area of the road grayscale image, the discrete significance of each changing pixel in the road grayscale image is obtained, which helps to identify key points of leaves and shrubs in the green belt, thereby avoiding key points with low reliability in the green belt during subsequent processing. Furthermore, by fully considering the periodic distribution characteristics and random occlusion characteristics of leaves and shrubs within the green belt area, and combining the repetitive distribution degree and discrete significance of each changing pixel, the possibility of each changing pixel being a low-reliability key point within the green belt can be assessed more accurately. This allows for more accurate avoidance of low-reliability key points within the green belt during road condition simulation, improving the accuracy of matching between all road grayscale images, and ultimately enhancing the quality of the road panoramic image after panoramic stitching.
Claims
1. A method for road panoramic stitching for road condition simulation, characterized in that, The method includes the following steps: During the road condition simulation, grayscale images of various roads around the simulated vehicle are collected synchronously and in real time. For all road grayscale images at any given time, the distribution difference of each pixel is obtained by comparing the neighborhood grayscale distribution of each pixel with its nearest neighbor pixels in each road grayscale image, and then each changing pixel is selected from each road grayscale image; by combining the degree of repetition and dispersion of the distribution difference of all changing pixels in the nearest neighbor range of each changing pixel in each road grayscale image with the number of changing pixels in the nearest neighbor range, the repetitive distribution degree of each changing pixel is obtained; The process involves obtaining each connected region within the nearest neighbor range, acquiring the significance coefficient of the number of each changing pixel based on the size of the connected regions within the nearest neighbor range, acquiring the discreteness significance of each changing pixel based on the dispersion of the neighborhood grayscale distribution of the upper edge pixels of each connected region within the nearest neighbor range, and acquiring the comprehensive feature value of each changing pixel based on the repetitive distribution degree. Then, it involves extracting each unusable keypoint from each road grayscale image; and extracting keypoints for stitching the road grayscale image from the pixels other than the unusable keypoints in each road grayscale image, thereby obtaining the panoramic road image at any given time.
2. The road panoramic stitching method for road condition simulation as described in claim 1, characterized in that, The process for obtaining the distribution difference is as follows: By analyzing the differences in gray values between each pixel and its multiple preset neighboring pixels, a gray-level difference vector for each pixel is obtained. The average value of the difference between the grayscale difference vectors of each pixel and all its preset nearest neighbor pixels is used as the distribution difference of each pixel.
3. The road panoramic stitching method for road condition simulation as described in claim 1, characterized in that, The process for filtering the changing pixels is as follows: Obtain the average level of the distribution difference of all pixels in each road grayscale image, and take each pixel in each road grayscale image whose distribution difference is greater than the average level as each changed pixel.
4. The road panoramic stitching method for road condition simulation as described in claim 1, characterized in that, The process of obtaining the repeatability distribution degree is as follows: In the distribution difference of all changing pixels within the nearest neighbor range, the frequency of occurrence of each distribution difference value is counted, and the information entropy and dispersion of the frequency of occurrence of all distribution difference values within the nearest neighbor range are calculated respectively. The repeatability distribution degree is inversely proportional to the information entropy and the dispersion, and directly proportional to the quantity.
5. The road panoramic stitching method for road condition simulation as described in claim 4, characterized in that, The method for calculating the repeatability distribution degree is as follows: The dispersion and the information entropy are mapped to a first positive number and a second positive number, respectively, and the repeatability distribution degree is the ratio of the quantity to the fusion result of the first positive number and the second positive number.
6. The road panoramic stitching method for road condition simulation as described in claim 1, characterized in that, The process of obtaining the numerical significance coefficient is as follows: The total number of pixels in the largest connected region within the nearest neighbor range of each variable pixel in each grayscale road image is counted. The ratio of the counted total number to the total number of pixels in the nearest neighbor range is used as the significance coefficient.
7. The road panoramic stitching method for road condition simulation as described in claim 1, characterized in that, The process of obtaining the discrete significance is as follows: Calculate the gradient magnitude of each edge pixel in each connected region and evaluate the dispersion of the gradient magnitude of all edge pixels in each connected region. Calculate the average of the evaluation results for all connected regions within the nearest neighbor range; The discrete significance is obtained by combining the average value with the numerical significance coefficient.
8. The method for road panoramic stitching for road condition simulation as described in claim 7, characterized in that, The significance of dispersion is the product of the average value and the numerical significance coefficient.
9. A road panoramic stitching method for road condition simulation as described in claim 1, characterized in that, The comprehensive feature value is the sum of the normalized result of the repeatability distribution degree and the normalized result of the discrete significance degree.
10. A road panoramic stitching method for road condition simulation as described in claim 1, characterized in that, The process for extracting the unavailable key points is as follows: The segmentation threshold is obtained by acquiring the comprehensive feature value of all changing pixels in each grayscale road image; Each pixel in the grayscale road image whose comprehensive feature value is greater than the segmentation threshold is considered an unusable keypoint.