Cross-view positioning method based on applicable scene judgment

Abstract

The application provides a cross-view positioning method based on a suitable scene judgment. Whether the current scene is suitable for cross-view positioning is judged through an aerial occlusion coefficient and a ground feature coefficient, which is beneficial to improving the use efficiency of the computing resources in actual application, reducing the burden of the computing unit, effectively reducing the cross-view positioning noise, reducing the time consumption of the positioning process, and improving the actual performance of the positioning module.

Description

Technical Field

[0001] This invention belongs to the field of ground-air cross-view positioning, mainly involving image classification, image segmentation and road extraction, specifically involving a cross-view positioning method based on applicable scenario judgment. Background Technology

[0002] Ground-to-air cross-view localization involves selecting the satellite image from a database that most closely resembles the input vehicle-view image, and using this image to determine the vehicle's location. The retrieval process utilizes matching algorithms, which can be categorized into traditional feature point matching methods and the more recently popular network-based methods. By matching, the position of the ground-view image within the satellite-view image can be determined, thus deriving the vehicle's latitude and longitude. Recent research treats this task as an image regression problem, offering more accurate localization results.

[0003] The goal of image classification is to categorize different images into different classes with minimal classification error. This process presents numerous challenges, but opportunities and challenges coexist. In recent years, with the rise of deep learning technology, image classification has developed rapidly, giving rise to a series of research directions, such as: multi-class image classification, fine-grained image classification, multi-label image classification, unsupervised / semi-supervised image classification, and zero-shot image classification.

[0004] Image segmentation is a technique that divides objects in an image into pixels according to their type. It is a key step in the transition from image processing to image analysis. Existing image segmentation methods can be mainly classified into the following categories: threshold-based segmentation methods, region-based segmentation methods, edge-based segmentation methods, and segmentation methods based on specific theories.

[0005] Road extraction aims to extract road networks from satellite imagery. Some methods detect roads using road features such as lines, edges, and textures. However, roads do not always exhibit general characteristics due to lighting, shadows, and occlusion. Human-described features are often insufficient to fully describe roads. With the rapid development of neural network technology, an increasing number of studies utilize neural networks for feature extraction. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide a cross-view positioning method based on applicable scenario judgment, which can achieve cross-positioning of ground view and satellite view while improving the utilization of computing resources.

[0007] A cross-view localization method based on applicable scenario judgment includes:

[0008] Step S1: Acquire multi-camera images of the vehicle surround view;

[0009] Step S2: Stitch the multi-camera images into a panoramic image, segment the obstacles in the top region of the panoramic image, and calculate the aerial occlusion coefficient k based on the proportion of obstacles. ob ;

[0010] Step S3: Extract the drivable area mask from the BEV perspective, and calculate the ground feature coefficient k after type recognition. fe ;

[0011] Step S4: Based on the aerial obstruction coefficient k ob and ground characteristic coefficient k fe Calculate the scenario applicability score k fit This is used to determine whether the current scene is suitable for visual cross-view positioning;

[0012] Step S5: If the scene is suitable for positioning, the positioning result is obtained by matching the panoramic image with the satellite image; if the scene is not suitable for positioning, the positioning process is not initiated.

[0013] Preferably, in step S1, multiple cameras are mounted on the vehicle in a fixed manner, and six images are acquired at eye level and one image is acquired at an upward angle towards the sky.

[0014] Preferably, in step S2, the six images obtained from the eye-level angle are first stitched together into a panoramic image using a stitching algorithm, and a binary image is obtained at the top region of the panoramic image using an obstacle segmentation algorithm, with an aerial occlusion coefficient k. ob The calculation formula is as follows:

[0015]

[0016] Among them, a i For all pixels identified as obstacles, and b i For all pixels identified as non-obstacles;

[0017] Preferably, in step S3, the ground characteristic coefficient k fe The calculation methods include:

[0018] The binary image is divided into 36 square sectors of the same size. Each sector is considered a "pedal sector" if more than 20% of its pixels are drivable. The number of "pedal sectors" n is then used to determine the drivable sector. dri Binary images are classified into different types, based on the following criteria:

[0019]

[0020] Ground characteristic coefficient k fe The calculation formula is as follows:

[0021]

[0022] Preferably, in step S4, based on the aerial obstruction coefficient k ob and ground characteristic coefficient k fe Calculate the scenario applicability score k fit The calculation formula is as follows:

[0023] k fit =1-(0.66×k) ob +0.34×k fe )

[0024] Where k fit ∈(0,1), the higher the score, the more suitable the current scene is for cross-view positioning.

[0025] Preferably, in step S5, the determination of whether to enter the positioning stage is based on the judgment in step S4. The specific implementation of positioning is as follows: the panoramic image and a batch of satellite images in the database are simultaneously input into the twin network, and the center position of the satellite image with the highest matching probability with the panoramic image is selected as the positioning result.

[0026] The present invention has the following beneficial effects:

[0027] This invention provides a cross-view positioning method based on applicable scenario judgment, using an aerial occlusion coefficient k. ob and ground characteristic coefficient k fe Determining whether the current scene is suitable for cross-view positioning is beneficial for improving the efficiency of computing resource utilization in practical applications and reducing the burden on computing units; it is also beneficial for effectively reducing cross-view positioning noise, reducing the time spent in the positioning process, and improving the actual performance of the positioning module. Attached Figure Description

[0028] Figure 1 This is a flowchart of the cross-view positioning method based on applicable scenario judgment of the present invention;

[0029] Figure 2 This is a structural diagram of the aerial obstruction coefficient algorithm of the present invention;

[0030] Figure 3 This is a structural diagram of the ground feature coefficient algorithm of the present invention. Detailed Implementation

[0031] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. Those skilled in the art will understand the advantages and functions of the present invention based on the content described herein. The present invention can also be implemented and applied in other different ways.

[0032] A cross-view localization algorithm based on applicable scenario judgment includes the following steps:

[0033] Step S1: Acquire multi-camera images of the vehicle surround view and input them into the algorithm;

[0034] Step S2: Stitch the multi-camera images into a panoramic image, segment the obstacles in the top region of the panoramic image, and calculate the aerial occlusion coefficient k based on the proportion of obstacles. ob ;

[0035] Step S3: Input multi-camera images into the road extraction algorithm to extract the drivable area mask from the BEV perspective, and calculate the ground feature coefficient k after type recognition. fe ;

[0036] Step S4: Based on the aerial obstruction coefficient k ob and ground characteristic coefficient k fe Calculate the scenario applicability score k fit This is used to determine whether the current scene is suitable for visual cross-view positioning.

[0037] Step S5: If the scene is suitable for positioning, the positioning result is obtained by matching the panoramic image with the satellite image; if the scene is not suitable for positioning, the positioning process is not initiated.

[0038] Furthermore, in step S1, multiple cameras are fixedly mounted on the vehicle, and can acquire 6 images from a level-up angle and 1 image from a direction directly above the sky.

[0039] Furthermore, in step S2, the six images obtained from the eye-level view are first stitched together into a panoramic image using a stitching algorithm. Then, a binary image (distinguished by whether it contains an obstacle) is obtained from the top region of the panoramic image using an obstacle segmentation algorithm. The aerial occlusion coefficient k... ob The calculation formula is as follows:

[0040]

[0041] Among them, a i For all pixels identified as obstacles, and b i For all pixels identified as non-obstacles.

[0042] Aerial obstruction coefficient k ob The theoretical basis is that due to the presence of aerial obstructions, the area of ​​shared view between ground-view images and satellite-view images will be reduced, thus greatly reducing the success rate of cross-visual positioning.

[0043] Furthermore, in step S3, after inputting the multi-camera image into the road extraction algorithm, a binary image is obtained, in which pixels are distinguished by whether they are drivable. After the binary image is processed by the road recognition algorithm, it is identified as a "straight road", "curve", or "intersection" according to its type.

[0044] The road recognition algorithm works as follows: First, the binary image is divided into 36 square sectors of the same size. Each sector is considered a "driving sector" if more than 20% of its pixels are drivable. The algorithm then calculates the number of "driving sectors" n. dri Binary images are classified into different types, based on the following criteria:

[0045]

[0046] Ground characteristic coefficient k fe The calculation formula is as follows:

[0047]

[0048] Ground characteristic coefficient k fe The theoretical basis is that the richness of ground features increases sequentially from straight road to curve, and then from curve to intersection. Richer ground features help improve the success rate of cross-view positioning. Therefore, different types of drivable areas are assigned different weights.

[0049] Furthermore, in step S4, based on the aerial obstruction coefficient k ob and ground characteristic coefficient k fe Calculate the scenario applicability score k fit The calculation formula is as follows:

[0050] k fit =1-(0.66×k) ob +0.34×k fe )

[0051] Where k fit The score is ∈(0,1), and a higher score indicates that the current scene is more suitable for cross-view localization. In practical use, an appropriate threshold can be determined to determine whether the current scene is suitable for cross-view localization.

[0052] Furthermore, in step S5, the determination of whether to proceed to the positioning stage is based on the judgment in S4. The specific implementation of positioning is as follows: the panoramic image and a batch of satellite images from the database are simultaneously input into the twin network, and the center position of the satellite image with the highest matching probability with the panoramic image is selected as the positioning result.

[0053] This implementation example provides a cross-view localization algorithm based on applicable scenario judgment. The specific process of this method is illustrated as follows: Figure 1 The specific process is shown below:

[0054] Step S1: Acquire images from multiple cameras around the vehicle; the multiple cameras are fixedly mounted on the vehicle, and can acquire 6 images from a level view angle and 1 image from a view angle directly above the sky.

[0055] Step S2: Stitch the multi-camera images into a panoramic image, segment the obstacles in the top region of the panoramic image, and calculate the aerial occlusion coefficient k based on the proportion of obstacles. ob ;

[0056] First, a stitching algorithm is used to stitch together six images acquired from a level-view perspective into a panoramic image. Then, an obstacle segmentation algorithm is used in the top region of the panoramic image to obtain a binary image (distinguished by whether it is an obstacle). There are three types of obstacles: "buildings," "trees," and "bridges," thus covering the most common types of aerial obstructions. The aerial obstruction coefficient k... ob The calculation formula is as follows:

[0057]

[0058] Among them, a i For all pixels identified as obstacles, and b i For all pixels identified as non-obstacles.

[0059] Step S3: After inputting multi-camera images into the road extraction algorithm, a binary image is obtained, in which pixels are distinguished by whether they are drivable. After the binary image is processed by the road recognition algorithm, it is identified as "straight road", "curve", or "intersection" according to its type.

[0060] The road recognition algorithm works as follows: First, the binary image is divided into 36 square sectors of the same size. Each sector is considered a "driving sector" if more than 20% of its pixels are drivable. The algorithm then calculates the number of "driving sectors" n. dri Binary images are classified into different types, based on the following criteria:

[0061]

[0062] Ground characteristic coefficient k fe The calculation formula is as follows:

[0063]

[0064] Step S4: Based on the aerial obstruction coefficient k ob and ground characteristic coefficient k fe Calculate the scenario applicability score k fit This is used to determine whether the current scene is suitable for visual cross-view positioning;

[0065] According to the aerial obstruction coefficient k ob and ground characteristic coefficient k fe Calculate the scenario applicability score k fit The calculation formula is as follows:

[0066] k fit =1-(0.66×k)ob +0.34×k fe )

[0067] Where k fit The score is ∈(0,1), and a higher score indicates that the current scene is more suitable for cross-view localization. In practical use, an appropriate threshold can be determined to determine whether the current scene is suitable for cross-view localization.

[0068] Step S5: If the scene is suitable for positioning, the positioning result is obtained by matching the panoramic image with the satellite image; if the scene is not suitable for positioning, the positioning process is not initiated.

[0069] The specific implementation method of positioning is as follows: the panoramic image and a batch of satellite images in the database are simultaneously input into the twin network, and the center position of the satellite image with the highest matching probability with the panoramic image is selected as the positioning result.

[0070] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A cross-view positioning method based on applicable scenario judgment, characterized in that, include: Step S1: Acquire vehicle surround view multi-camera images: The multi-camera is fixedly mounted on the vehicle, and 6 images are acquired from the eye-level angle and 1 image is acquired from the angle directly above the sky. Step S2: Stitch the multi-camera images into a panoramic image, segment the obstacles in the top region of the panoramic image, and calculate the aerial occlusion coefficient k based on the proportion of obstacles. ob : First, a stitching algorithm is used to stitch together six images obtained from the eye-level perspective into a panoramic image. Then, an obstacle segmentation algorithm is used to obtain a binary image at the top region of the panoramic image, with an aerial occlusion coefficient k. ob The calculation formula is as follows: Among them, a i For all pixels identified as obstacles, and b i For all pixels identified as non-obstacles; Step S3: Extract the drivable area mask from the BEV perspective, and calculate the ground feature coefficient k after type recognition. fe The calculation methods include: The binary image is divided into 36 square sectors of the same size. Each sector is considered a "driving sector" if more than 20% of its pixels are drivable. The number of "driving sectors" n is then used to determine the drivable sector. dri Binary images are classified into different types, based on the following criteria: Ground characteristic coefficient k fe The calculation formula is as follows: Step S4: Based on the aerial obstruction coefficient k ob and ground characteristic coefficient k fe Calculate the scenario applicability score k fit This is used to determine whether the current scene is suitable for visual cross-view positioning; The calculation formula is as follows: k fit =1-(0.66×k ob +0.34×k fe ) Where k fit ∈(0,1), the higher the score, the more suitable the current scene is for cross-view localization; Step S5: If the scene is suitable for positioning, the positioning result is obtained by matching the panoramic image with the satellite image; if the scene is not suitable for positioning, the positioning process is not initiated.

2. The cross-view positioning method based on applicable scenario judgment as described in claim 1, characterized in that, In step S5, the determination of whether to enter the positioning stage is based on the judgment in step S4. The specific implementation of positioning is as follows: the panoramic image and a batch of satellite images in the database are simultaneously input into the twin network, and the center position of the satellite image with the highest matching probability with the panoramic image is selected as the positioning result.

Images