Vehicle positioning method, system and equipment

By combining remote sensing satellite image and bird's-eye viewing technology in vehicle positioning, building mask map and multi-view image features are extracted and matching calculations are performed, the problem of degradation of vehicle positioning accuracy in complex environments is solved, and high accuracy and high reliability are achieved , suitable for autonomous driving and other applications.

CN119091105APending Publication Date: 2024-12-06ADVANCED TECH RES INST OF BEIJING UNIV OF TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411271399.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-11
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

In complex environments, traditional vehicle positioning technologies such as GPS and INS are susceptible to interference, resulting in a decrease in positioning accuracy, and lack of a combination of remote sensing satellite image and bird's eye viewing technology to improve positioning accuracy and reliability.

Method used

The building mask map is extracted from the remote sensing image through the building extraction model, and the convolutional neural network is used to extract features from the multi-view image, and a bird's-eye view image is generated by combining pure visual perception algorithms to match features and shapes to calculate the target vehicle's Relative position, and then determine its latitude and longitude.

Benefits of technology

It realizes high-precision vehicle positioning in complex environments, improves the reliability and adaptability of the autonomous driving system, provides multi-level environmental information, improves positioning accuracy, and has a higher level of remote sensing satellite image update frequency and resolution in the future. practicality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091105A_ABST
    Figure CN119091105A_ABST
Patent Text Reader

Abstract

The invention provides a vehicle positioning method, system and equipment, and the method comprises the steps: carrying out the extraction of building vector data in a remote sensing image through a building extraction model, and obtaining a building mask image; performing feature extraction on the acquired multi-view image by using a convolutional neural network to obtain a multi-view feature map; learning the obtained multi-view feature map by using a pure visual perception algorithm to obtain a BEV image in automatic driving; matching the building mask map with the BEV image in automatic driving to determine the relative position of the target vehicle; and calculating the longitude and latitude of the relative position of the target vehicle through satellite information to determine the accurate position of the target vehicle. Based on the vehicle positioning method, the invention also provides a vehicle positioning system and equipment. According to the invention, the environment information acquired by the vehicle sensor is matched with the remote sensing satellite image, so that high-precision vehicle positioning is realized, and the reliability and adaptability of the automatic driving system in a complex environment are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous driving vehicle positioning, and in particular relates to a vehicle positioning method, system and device. Background Art

[0002] With the rapid development of autonomous driving technology, high-precision vehicle positioning technology has become one of the key elements to achieve autonomous driving. Traditional vehicle positioning technology mainly relies on the global positioning system (GPS) and inertial navigation system (INS), but in urban canyons, tunnels and other complex environments, GPS signals are easily interfered or blocked, resulting in reduced positioning accuracy. To solve this problem, researchers began to explore methods to fuse multiple sensor data to improve the reliability and accuracy of vehicle positioning.

[0003] Remote sensing satellite images provide large-scale, high-resolution geospatial information, which is widely used in urban planning, environmental monitoring, disaster management and other fields. In recent years, with the development of remote sensing technology, the resolution and update frequency of satellite images have been continuously improved, making their application prospects in autonomous driving more and more broad. By utilizing the geographic information in remote sensing satellite images, a global perspective can be provided for the vehicle, which helps to improve the accuracy and reliability of vehicle positioning. Bird's Eye View (BEV) technology has also gradually gained attention in the field of autonomous driving. BEV converts environmental information collected by multiple cameras or sensors into a bird's-eye view centered on the vehicle, allowing the vehicle to understand the surrounding environment more intuitively. BEV can not only be used for environmental perception and path planning, but also can be integrated with other sensor data to achieve higher-precision positioning. The vehicle positioning method based on remote sensing satellite images and BEV provides a new direction for the development of autonomous driving technology, but there is no public vehicle positioning method based on the combination of remote sensing satellite images and BEV in the prior art. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a vehicle positioning method, system and device, which achieve high-precision vehicle positioning by fusing multi-source data, and improve the reliability and adaptability of the autonomous driving system in complex environments.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] A vehicle positioning method comprises the following steps:

[0007] Use the building extraction model to extract the building vector data in the remote sensing image to obtain the building mask map;

[0008] Use convolutional neural networks to extract features from the collected multi-view images to obtain multi-view feature maps; use pure visual perception algorithms to learn the obtained multi-view feature maps to obtain BEV images in autonomous driving;

[0009] Matching the building mask image with the BEV image in the autonomous driving to determine the relative position of the target vehicle;

[0010] The latitude and longitude of the target vehicle's relative position are calculated through satellite information to determine the precise position of the target vehicle.

[0011] Furthermore, the process of extracting building vector data from remote sensing images using a building extraction model to obtain a building mask map includes:

[0012] Obtain remote sensing images of outdoor environments, and obtain geospatial data with building labels in the remote sensing images as annotations;

[0013] Convert geospatial data with building labels into binary mask images, where building areas are marked as 1 and other areas are marked as 0;

[0014] The binary mask image is input into the trained building extraction model to extract the building mask map.

[0015] Furthermore, the process of extracting features from the collected multi-view images using a convolutional neural network to generate a multi-view feature map includes:

[0016] The vehicle-mounted camera is used to collect multi-view images of the vehicle, wherein the multi-view images include a front view image, a rear view image, a left view image, a right view image, and a top view image;

[0017] Convolutional neural network is used to extract features from multi-view images and generate multi-view feature maps.

[0018] Furthermore, the process of matching the building mask image with the BEV image in the autonomous driving to determine the relative position of the target vehicle includes:

[0019] Pair the building mask and BEV image to form a paired dataset;

[0020] Constructing a neural network model for feature matching and shape matching in parallel; using a paired data set to train the neural network model to obtain a trained neural network model; inputting the paired data into the trained neural network model to obtain feature matching results and shape matching results respectively; then fusing the feature matching results and shape matching results to obtain a fused output result;

[0021] According to the fusion output result, the building outline in the BEV image is mapped to the coordinate system of the remote sensing image through the transformation matrix, and the relative position of the building in the remote sensing image is calculated; on the premise that the target vehicle is located at the center of the BEV image, the target vehicle position is mapped to the remote sensing image through the transformation matrix, thereby determining the relative position of the target vehicle.

[0022] Furthermore, the process of executing the parallel neural network model includes:

[0023] A first feature point and a second feature point are extracted from the building mask image and the BEV image using a feature detection algorithm; the first feature point and the second feature point are described to obtain a first feature vector and a second feature vector; and the first feature vector and the second feature vector are matched using a feature matching algorithm to obtain a feature matching result;

[0024] Extracting shape features of the building mask image and the BEV image respectively to obtain a first shape feature and a second shape feature, and using a shape matching algorithm to match the first shape feature and the second shape feature to obtain a shape matching result;

[0025] Adaptive weights are used to fuse the feature matching results and shape matching results to obtain the fused output result.

[0026] Furthermore, the feature matching algorithm adopts a SIFT feature detection algorithm; the feature matching algorithm adopts a BFMatcher feature matching algorithm; and the shape matching algorithm adopts a KNN shape matching algorithm.

[0027] Furthermore, the process of fusing the feature matching result and the shape matching result using the adaptive weight to obtain the fused output result includes:

[0028] After normalizing the feature matching results and the shape matching results, initial weights are assigned to the feature matching results and the shape matching results according to the confidence of the matching pairs;

[0029] The weights of the feature matching results and the shape matching results are adaptively calculated, and the matching results are weighted averaged using the adaptive weights to obtain the final matching result.

[0030] Furthermore, the process of calculating the latitude and longitude of the relative position of the target vehicle through satellite information to determine the precise position of the target vehicle includes:

[0031] For a given pixel coordinate (P x ,P y ), calculate the projection coordinates (X, Y) corresponding to the pixel coordinates; where:

[0032] X=X0+P x ×PixelSizeX;

[0033] Y=Y0-P y ×PixelSizeY;

[0034] (X0, Y0) are the coordinates of the reference point; (PixelSizeX, PixelSizeY) represents the pixel resolution;

[0035] Use GIS software to convert the projection coordinates into geographic coordinates, and determine the exact location of the target vehicle based on the latitude and longitude of the target vehicle in the geographic coordinate system.

[0036] The present invention also proposes a vehicle positioning system, comprising: a first extraction module, a second extraction module, a matching module and a positioning module;

[0037] The first extraction module is used to extract building vector data in the remote sensing image using a building extraction model to obtain a building mask map;

[0038] The second extraction module is used to use a convolutional neural network to extract features from the collected multi-view images to obtain a multi-view feature map; the obtained multi-view feature map is learned using a pure visual perception algorithm to obtain a BEV image in autonomous driving;

[0039] The matching module is used to match the building mask image with the BEV image in the autonomous driving to determine the relative position of the target vehicle;

[0040] The positioning module is used to calculate the latitude and longitude of the relative position of the target vehicle through satellite information to determine the precise position of the target vehicle.

[0041] The present invention also proposes a device, comprising:

[0042] Memory for storing computer programs;

[0043] A processor is used to implement the method steps described when executing the computer program.

[0044] The effects provided in the content of the invention are only the effects of the embodiments, not all the effects of the invention. One of the above technical solutions has the following advantages or beneficial effects:

[0045] The present invention proposes a vehicle positioning method, system and device, which includes: using a building extraction model to extract building vector data in remote sensing images to obtain a building mask map; using a convolutional neural network to extract features from the collected multi-view images to obtain a multi-view feature map; using a pure visual perception algorithm to learn the acquired multi-view feature map to obtain a BEV image in autonomous driving; matching the building mask map with the BEV image in autonomous driving to determine the relative position of the target vehicle; calculating the latitude and longitude of the relative position of the target vehicle through satellite information to determine the precise position of the target vehicle. Based on a vehicle positioning method, a vehicle positioning system and device are also proposed. The present invention combines the advantages of remote sensing satellite images and BEV images, aiming to solve positioning problems in complex environments. By matching the environmental information obtained by the vehicle sensor with the remote sensing satellite image, high-precision vehicle positioning is achieved, which improves the reliability and adaptability of the autonomous driving system in complex environments.

[0046] The vehicle positioning method provided by the present invention can provide multi-level and multi-scale environmental information, and realize high-precision positioning in complex environments by integrating remote sensing images of global perspective and sensor data of local details. At the same time, as the update frequency and resolution of remote sensing satellite images continue to increase, the positioning method provided by the present invention will have higher practicality and reliability in the future. It can not only improve the positioning accuracy of autonomous driving vehicles, but also provide strong support for emergency rescue, disaster response and urban planning. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A flow chart of a vehicle positioning method proposed in Embodiment 1 of the present invention;

[0048] Figure 2 A schematic diagram of a vehicle positioning system proposed in Embodiment 2 of the present invention;

[0049] Figure 3 This is a schematic diagram of a vehicle positioning device proposed in Example 3 of the present invention. DETAILED DESCRIPTION

[0050] In order to clearly illustrate the technical features of the present solution, the present invention is described in detail below through specific implementation methods and in conjunction with the accompanying drawings. The disclosure below provides many different embodiments or examples for realizing different structures of the present invention. In order to simplify the disclosure of the present invention, the components and settings of specific examples are described below. In addition, the present invention may repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed. It should be noted that the components illustrated in the accompanying drawings are not necessarily drawn to scale. The present invention omits the description of known components and processing techniques and processes to avoid unnecessary limitations on the present invention.

[0051] Example 1

[0052] Embodiment 1 of the present invention proposes a vehicle positioning method, which combines the advantages of remote sensing satellite images and BEV images, and aims to solve the positioning problem in complex environments. Figure 1 This is a flow chart of a vehicle positioning method proposed in Example 1 of the present invention.

[0053] In step S100, the building extraction model is used to extract the building vector data in the remote sensing image to obtain a building mask map.

[0054] Acquire remote sensing images of outdoor environments, collect high-resolution remote sensing images or aerial images, which should contain a variety of terrains and building diversity; and obtain geospatial data with building labels in remote sensing images as annotations, wherein the geospatial data with building labels are vector data or raster data.

[0055] Convert geospatial data with building labels into binary mask images, where building areas are marked as 1 and other areas are marked as 0;

[0056] The building extraction models in this application include U-Net, SegNet, DeepLab, Mask R-CNN, etc. You can choose a pre-trained model (such as a model pre-trained on COCO, Cityscapes and other datasets) and then perform transfer learning.

[0057] Use GPU for accelerated training, input training data into the building extraction model, perform forward propagation and back propagation, and update model weights. Split the training set and validation set, monitor the performance of the model on the validation set, and prevent overfitting.

[0058] The model performance is evaluated using indicators such as accuracy, IoU, F1-score, recall, and precision. The predicted results are compared with the true labels, and visual images are generated to evaluate the performance of the model in different scenarios.

[0059] In step S200, a convolutional neural network is used to extract features from the collected multi-view images to obtain a multi-view feature map; the obtained multi-view feature map is learned using a pure visual perception algorithm to obtain a BEV image in autonomous driving.

[0060] An on-board camera is used to collect multi-view images of the vehicle, including a front view image, a rear view image, a left view image, a right view image and a top view image; secondly, sensor data corresponding to the image is obtained to provide additional depth information and vehicle posture.

[0061] Use convolutional neural network to extract features from multi-view images and generate multi-view feature map F t .

[0062] In this application, the pure visual perception algorithm is BEVFormer. BEVFormer extracts the image features collected by the surround view camera and converts the extracted surround view features into the BEV space through model learning (the model learns how to convert the features from the image coordinate system to the BEV coordinate system).

[0063] Define a set of BEV query points Q on the bird's-eye view plane B , usually represented using a fixed grid.

[0064] Use the temporal self-attention module to transform the current frame BEV image B t and historical frame BEV image B t-1 Interact with each other to get the updated BEV image. B and multi-view image features F t Perform cross-attention calculations to extract relevant features. This step is repeated multiple times to obtain the final BEV image.

[0065] In step S300, the building mask image and the BEV image in the autonomous driving are matched to determine the relative position of the target vehicle.

[0066] The data sources are: one part is real-time BEV images generated from vehicle sensors such as cameras or lidar, and the other part is building masks of the target area extracted from remote sensing images taken by satellites.

[0067] The two parts of input are preprocessed to ensure data quality and consistency. The building mask and BEV image are paired to form a paired dataset; the two types of images are paired to ensure that there is an accurate correspondence between each pair of images and masks; all paired images form a dataset and are divided into training set, validation set and test set in a ratio of 7:2:1.

[0068] Construct a neural network model for feature matching and shape matching in parallel; use the paired data set to train the neural network model to obtain the trained neural network model; the specific process is: input the training samples in the training set into the initialized parallel network, calculate the loss of each part according to the network propagation process, and adjust each parameter according to the loss to obtain the best network parameters, then test it in the test set and verify it in the verification set to finally obtain the trained neural network model.

[0069] The paired data is input into the trained neural network model to obtain the feature matching results and the shape matching results respectively; then the feature matching results and the shape matching results are fused to obtain the fused output result;

[0070] According to the fusion output result, the building outline in the BEV image is mapped to the coordinate system of the remote sensing image through the transformation matrix, and the relative position of the building in the remote sensing image is calculated; on the premise that the target vehicle is located at the center of the BEV image, the target vehicle position is mapped to the remote sensing image through the transformation matrix, thereby determining the relative position of the target vehicle.

[0071] In this application, the process of executing the parallel neural network model includes:

[0072] A first feature point and a second feature point are extracted from the building mask image and the BEV image using a feature detection algorithm; the first feature point and the second feature point are described to obtain a first feature vector and a second feature vector; and the first feature vector and the second feature vector are matched using a feature matching algorithm to obtain a feature matching result;

[0073] Extracting shape features of the building mask image and the BEV image respectively to obtain a first shape feature and a second shape feature, and using a shape matching algorithm to match the first shape feature and the second shape feature to obtain a shape matching result;

[0074] Adaptive weights are used to fuse the feature matching results and shape matching results to obtain the fused output result.

[0075] The feature matching process includes:

[0076] Feature detection: Apply SIFT feature detection algorithm on the building mask of BEV image and remote sensing image to extract feature points. Feature points are points with significant features in the image, such as corner points or edge points.

[0077] Feature description: For each feature point obtained, SIFT descriptor is applied to generate the corresponding feature vector. A feature vector is a high-dimensional vector used to represent the local image information of the feature point.

[0078] Feature matching: Use the BFMatcher feature matching algorithm to match the feature vectors of the BEV image and the remote sensing image. The matching algorithm finds the most similar feature vector pairs in the two images.

[0079] Filter matching results: Use the RANSAC filtering strategy to filter matching results and remove false matches. Keep verified matching pairs to improve matching accuracy.

[0080] Transformation Matrix: Use the result of feature matching to estimate the transformation matrix between images.

[0081] The shape matching process includes:

[0082] Extract shape features: First, use the edge detection algorithm Canny algorithm to process the BEV image and remote sensing image building mask obtained in the above steps to obtain the building outline.

[0083] Feature description: Use Fourier descriptors to describe the contours of the step buildings and generate feature vectors.

[0084] Feature matching: The feature matching algorithm KNN is used to match the features in the BEV image and remote sensing image, and to filter out incorrect matching pairs.

[0085] Shape matching: The shape context matching algorithm is further used to refine the matching results, remove false matches, and obtain the final matching results.

[0086] The fusion process is:

[0087] Input matching results: Get the results of feature matching and shape matching and their confidence levels.

[0088] Alignment operation: Use the obtained transformation matrix to align the shape matching result to the coordinate system of the feature matching result, ensuring that the matching results of the two paths are aligned before fusion.

[0089] Calculate weights: First, normalize the matching degrees of feature matching and shape matching, assign initial weights to the two matching results, and adaptively calculate the weight of each path based on the confidence of the matching pair. Use adaptive weights to perform weighted averaging on the matching results to obtain the final matching result. Continuously update the weights based on new matching results to achieve dynamic optimization.

[0090] Fusion matching results: Use the calculated weights to weighted fuse the feature matching and shape matching results.

[0091] Filter fusion results: Use the filtering strategy RANSAC to filter the fused matching results.

[0092] Output the final matching results: Output the final matching results after filtering for subsequent positioning operations.

[0093] In step S400, the latitude and longitude of the relative position of the target vehicle are calculated through satellite information to determine the precise position of the target vehicle.

[0094] First, read the metadata of the image, including the image resolution, upper left corner coordinates, projection information, etc.; second, obtain the image affine transformation parameters to convert pixel coordinates into geographic coordinates; for a given pixel coordinate (P x ,P y ), calculate the projection coordinates (X, Y) corresponding to the pixel coordinates; where:

[0095] X=X0+P x ×PixelSizeX;

[0096] Y=Y0-P y ×PixelSizeY;

[0097] (X0, Y0) are the coordinates of the reference point; (PixelSizeX, PixelSizeY) represents the pixel resolution;

[0098] Use GIS software to convert the projection coordinates into geographic coordinates, and determine the exact location of the target vehicle based on the latitude and longitude of the target vehicle in the geographic coordinate system.

[0099] A vehicle positioning method proposed in Example 1 of the present invention combines the advantages of remote sensing satellite images and BEV images to solve the positioning problem in complex environments. By matching the environmental information obtained by the vehicle sensor with the remote sensing satellite image, high-precision vehicle positioning is achieved, thereby improving the reliability and adaptability of the automatic driving system in complex environments.

[0100] A vehicle positioning method provided in Example 1 of the present invention can provide multi-level and multi-scale environmental information, and achieve high-precision positioning in complex environments by fusing remote sensing images of a global perspective and sensor data of local details. At the same time, as the update frequency and resolution of remote sensing satellite images continue to increase, the positioning method provided by the present invention will have higher practicality and reliability in the future. It can not only improve the positioning accuracy of autonomous driving vehicles, but also provide strong support for emergency rescue, disaster response and urban planning.

[0101] Example 2

[0102] Based on a vehicle positioning method proposed in Embodiment 1 of the present invention, Embodiment 2 of the present invention further proposes a vehicle positioning system. Figure 2 A schematic diagram of a vehicle positioning system proposed in Embodiment 2 of the present invention; the system comprises: a first extraction module, a second extraction module, a matching module and a positioning module;

[0103] The first extraction module is used to extract building vector data in the remote sensing image using a building extraction model to obtain a building mask map;

[0104] The second extraction module is used to use a convolutional neural network to extract features from the collected multi-view images to obtain a multi-view feature map; the obtained multi-view feature map is learned using a pure visual perception algorithm to obtain a BEV image in autonomous driving;

[0105] The matching module is used to match the building mask image with the BEV image in the autonomous driving to determine the relative position of the target vehicle;

[0106] The positioning module is used to calculate the latitude and longitude of the relative position of the target vehicle through satellite information to determine the precise position of the target vehicle.

[0107] The process executed by the first extraction module includes: obtaining a remote sensing image of an outdoor environment, and obtaining geospatial data with building labels in the remote sensing image as annotations; converting the geospatial data with building labels into a binary mask image, in which the building area is marked as 1 and other areas are marked as 0; inputting the binary mask image into a trained building extraction model to extract the building mask map.

[0108] The process executed by the second extraction module includes: using a vehicle-mounted camera to collect multi-view images of the vehicle, the multi-view images including a front view image, a rear view image, a left view image, a right view image and a top view image; using a convolutional neural network to extract features from the multi-view images to generate a multi-view feature map.

[0109] In this application, the pure visual perception algorithm is BEVFormer. BEVFormer extracts the image features collected by the surround view camera and converts the extracted surround view features into the BEV space through model learning (the model learns how to convert the features from the image coordinate system to the BEV coordinate system).

[0110] Define a set of BEV query points Q on the bird's-eye view plane B , usually represented using a fixed grid.

[0111] Use the temporal self-attention module to transform the current frame BEV image B t and historical frame BEV image B t-1 Interact with each other to get the updated BEV image. B and multi-view image features F t Perform cross-attention calculations to extract relevant features. This step is repeated multiple times to obtain the final BEV image.

[0112] The process performed by the matching module includes: pairing the building mask and the BEV image to form a paired dataset;

[0113] Constructing a neural network model for feature matching and shape matching in parallel; using a paired data set to train the neural network model to obtain a trained neural network model; inputting the paired data into the trained neural network model to obtain feature matching results and shape matching results respectively; then fusing the feature matching results and shape matching results to obtain a fused output result;

[0114] According to the fusion output results, the building outline in the BEV image is mapped to the coordinate system of the remote sensing image through the transformation matrix, and the relative position of the building in the remote sensing image is calculated; on the premise that the target vehicle is located at the center of the BEV image, the target vehicle position is mapped to the remote sensing image through the transformation matrix to determine the relative position of the target vehicle.

[0115] The parallel neural network model execution process includes:

[0116] A first feature point and a second feature point are extracted from the building mask image and the BEV image using a feature detection algorithm; the first feature point and the second feature point are described to obtain a first feature vector and a second feature vector; and the first feature vector and the second feature vector are matched using a feature matching algorithm to obtain a feature matching result;

[0117] Extracting shape features of the building mask image and the BEV image respectively to obtain a first shape feature and a second shape feature, and using a shape matching algorithm to match the first shape feature and the second shape feature to obtain a shape matching result;

[0118] Adaptive weights are used to fuse the feature matching results and shape matching results to obtain the fused output result.

[0119] The feature matching algorithm adopts the SIFT feature detection algorithm; the feature matching algorithm adopts the BFMatcher feature matching algorithm; and the shape matching algorithm adopts the KNN shape matching algorithm.

[0120] The process of fusing the feature matching results and the shape matching results using adaptive weights to obtain the fused output results includes:

[0121] After normalizing the feature matching results and the shape matching results, initial weights are assigned to the feature matching results and the shape matching results according to the confidence of the matching pairs;

[0122] The weights of the feature matching results and the shape matching results are adaptively calculated, and the matching results are weighted averaged using the adaptive weights to obtain the final matching result.

[0123] The process performed by the positioning module includes: first, reading the metadata of the image, including the image resolution, upper left corner coordinates, projection information, etc.; second, obtaining the affine transformation parameters of the image to convert pixel coordinates into geographic coordinates; for a given pixel coordinate (P x ,P y ), calculate the projection coordinates (X, Y) corresponding to the pixel coordinates; where:

[0124] X=X0+P x ×PixelSizeX;

[0125] Y=Y0-P y ×PixelSizeY;

[0126] (X0, Y0) are the coordinates of the reference point; (PixelSizeX, PixelSizeY) represents the pixel resolution;

[0127] Use GIS software to convert the projection coordinates into geographic coordinates, and determine the exact location of the target vehicle based on the latitude and longitude of the target vehicle in the geographic coordinate system.

[0128] A vehicle positioning system proposed in Example 2 of the present invention combines the advantages of remote sensing satellite images and BEV images to solve positioning problems in complex environments. By matching the environmental information obtained by the vehicle sensor with the remote sensing satellite images, high-precision vehicle positioning is achieved, thereby improving the reliability and adaptability of the autonomous driving system in complex environments.

[0129] A vehicle positioning system provided in Example 2 of the present invention can provide multi-level and multi-scale environmental information, and achieve high-precision positioning in complex environments by integrating remote sensing images of a global perspective and sensor data of local details. At the same time, as the update frequency and resolution of remote sensing satellite images continue to increase, the positioning method provided by the present invention will have higher practicality and reliability in the future. It can not only improve the positioning accuracy of autonomous driving vehicles, but also provide strong support for emergency rescue, disaster response and urban planning.

[0130] Example 3

[0131] The present invention also proposes a device, Figure 3 This is a schematic diagram of a vehicle positioning device proposed in Embodiment 3 of the present invention. It includes: a memory for storing a computer program; a processor for executing the computer program to implement the following method steps:

[0132] In step S100, the building extraction model is used to extract the building vector data in the remote sensing image to obtain a building mask map.

[0133] In step S200, a convolutional neural network is used to extract features from the collected multi-view images to obtain a multi-view feature map; the obtained multi-view feature map is learned using a pure visual perception algorithm to obtain a BEV image in autonomous driving.

[0134] In step S300, the building mask image and the BEV image in the autonomous driving are matched to determine the relative position of the target vehicle.

[0135] In step S400, the latitude and longitude of the relative position of the target vehicle are calculated through satellite information to determine the precise position of the target vehicle.

[0136] The vehicle positioning device proposed in Embodiment 3 of the present invention combines the advantages of remote sensing satellite images and BEV images to solve the positioning problem in complex environments. By matching the environmental information obtained by the vehicle sensor with the remote sensing satellite images, high-precision vehicle positioning is achieved, thereby improving the reliability and adaptability of the automatic driving system in complex environments.

[0137] A vehicle positioning device provided in Example 3 of the present invention can provide multi-level and multi-scale environmental information, and achieve high-precision positioning in complex environments by fusing remote sensing images of a global perspective and sensor data of local details. At the same time, as the update frequency and resolution of remote sensing satellite images continue to increase, the positioning method provided by the present invention will have higher practicality and reliability in the future. It can not only improve the positioning accuracy of autonomous driving vehicles, but also provide strong support for emergency rescue, disaster response and urban planning.

[0138] It should be noted that the technical solution of the present invention also provides an electronic device, including: a communication interface, which can exchange information with other devices such as network devices; a processor, connected to the communication interface to realize information exchange with other devices, and used to execute a vehicle positioning method provided by one or more of the above technical solutions when running a computer program, and the computer program is stored in a memory. Of course, in actual application, the various components in the electronic device are coupled together through a bus system. It can be understood that the bus system is used to realize the connection and communication between these components. In addition to the data bus, the bus system also includes a power bus, a control bus and a status signal bus. The memory in the embodiment of the present application is used to store various types of data to support the operation of the electronic device. Examples of these data include: any computer program for operating on an electronic device. It can be understood that the memory can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a ferromagnetic random access memory, a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache.By way of exemplary but not limiting explanation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), direct memory bus random access memory (DRRAM). The memory described in the embodiment of the present application is intended to include but is not limited to these and any other suitable types of memory. The method disclosed in the above embodiment of the present application can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. The above processor can be a general-purpose processor, a DSP (Digital Signal Processing, which refers to a chip that can implement digital signal processing technology), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor, etc. In conjunction with the steps of the method disclosed in the embodiments of the present application, it can be directly embodied as a hardware decoding processor to perform, or it can be performed by a combination of hardware and software modules in a decoding processor. The software module can be located in a storage medium, which is located in a memory, and the processor reads the program in the memory and completes the steps of the above method in combination with its hardware. When the processor executes the program, the corresponding process in each method of the embodiment of the present application is implemented, and for the sake of brevity, it is not repeated here.

[0139] The description of the relevant parts of a vehicle positioning device provided in Example 3 of the present application can be found in the detailed description of the corresponding parts of a vehicle positioning method provided in Example 1 of the present application, and will not be repeated here.

[0140] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment that includes a series of elements are inherent to the elements. In the absence of more restrictions, the elements limited by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or equipment that includes the elements. In addition, the above-mentioned technical solution provided in the embodiment of the present application is consistent with the corresponding technical solution in the prior art in principle, and the part is not described in detail, so as not to repeat too much.

[0141] Although the above describes the specific implementation of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. For those skilled in the art, other different forms of modifications or deformations can be made on the basis of the above description. It is not necessary and impossible to list all the implementation methods here. On the basis of the technical solution of the present invention, various modifications or deformations that can be made by those skilled in the art without creative work are still within the scope of protection of the present invention.

Claims

1. A vehicle positioning method, It is characterized in that The following steps are involved: Use the building extraction model to extract the building vector data in the remote sensing image to obtain the building mask map; Use convolutional neural networks to extract features from the collected multi-view images to obtain multi-view feature maps; use pure visual perception algorithms to learn the obtained multi-view feature maps to obtain BEV images in autonomous driving; Matching the building mask image with the BEV image in the autonomous driving to determine the relative position of the target vehicle; The latitude and longitude of the target vehicle's relative position are calculated through satellite information to determine the precise position of the target vehicle.

2. A vehicle positioning method according to claim 1, It is characterized in that The process of extracting building vector data from remote sensing images using a building extraction model to obtain a building mask map includes: Obtain remote sensing images of outdoor environments, and obtain geospatial data with building labels in the remote sensing images as annotations; Convert geospatial data with building labels into binary mask images, where building areas are marked as 1 and other areas are marked as 0; The binary mask image is input into the trained building extraction model to extract the building mask map.

3. A vehicle positioning method according to claim 1, It is characterized in that The process of using a convolutional neural network to extract features from the collected multi-view images and generate a multi-view feature map includes: The vehicle-mounted camera is used to collect multi-view images of the vehicle, wherein the multi-view images include a front view image, a rear view image, a left view image, a right view image, and a top view image; Convolutional neural network is used to extract features from multi-view images and generate multi-view feature maps.

4. A vehicle positioning method according to claim 1, It is characterized in that The process of matching the building mask image with the BEV image in the autonomous driving to determine the relative position of the target vehicle includes: Pair the building mask and BEV image to form a paired dataset; Constructing a neural network model for feature matching and shape matching in parallel; using a paired data set to train the neural network model to obtain a trained neural network model; inputting the paired data into the trained neural network model to obtain feature matching results and shape matching results respectively; then fusing the feature matching results and shape matching results to obtain a fused output result; According to the fusion output result, the building outline in the BEV image is mapped to the coordinate system of the remote sensing image through the transformation matrix, and the relative position of the building in the remote sensing image is calculated; on the premise that the target vehicle is located at the center of the BEV image, the target vehicle position is mapped to the remote sensing image through the transformation matrix, thereby determining the relative position of the target vehicle.

5. A vehicle positioning method according to claim 4, It is characterized in that The process of executing the parallel neural network model includes: A first feature point and a second feature point are extracted from the building mask image and the BEV image using a feature detection algorithm; the first feature point and the second feature point are described to obtain a first feature vector and a second feature vector; and the first feature vector and the second feature vector are matched using a feature matching algorithm to obtain a feature matching result; Extracting shape features of the building mask image and the BEV image respectively to obtain a first shape feature and a second shape feature, and using a shape matching algorithm to match the first shape feature and the second shape feature to obtain a shape matching result; Adaptive weights are used to fuse the feature matching results and shape matching results to obtain the fused output result.

6. A vehicle positioning method according to claim 5, It is characterized in that The feature matching algorithm adopts the SIFT feature detection algorithm; the feature matching algorithm adopts the BFMatcher feature matching algorithm; and the shape matching algorithm adopts the KNN shape matching algorithm.

7. A vehicle positioning method according to claim 5, It is characterized in that The process of fusing the feature matching results and the shape matching results using adaptive weights to obtain the fused output results includes: After normalizing the feature matching results and the shape matching results, initial weights are assigned to the feature matching results and the shape matching results according to the confidence of the matching pair; The weights of the feature matching results and the shape matching results are adaptively calculated, and the matching results are weighted averaged using the adaptive weights to obtain the final matching result.

8. A vehicle positioning method according to claim 1, It is characterized in that The process of calculating the latitude and longitude of the relative position of the target vehicle through satellite information to determine the precise position of the target vehicle includes: For a given pixel coordinate (P x ,P y ), calculate the projection coordinates (X, Y) corresponding to the pixel coordinates; where: X=X 0 +P x ×PixelSizeX; Y=Y 0 -P y ×PixelSizeY; (X 0 ,Y 0 ) are the coordinates of the reference point; (PixelSizeX, PixelSizeY) represent the pixel resolution; Use GIS software to convert the projection coordinates into geographic coordinates, and determine the exact location of the target vehicle based on the latitude and longitude of the target vehicle in the geographic coordinate system.

9. A vehicle positioning system, It is characterized in that include: A first extraction module, a second extraction module, a matching module and a positioning module; The first extraction module is used to extract building vector data in the remote sensing image using a building extraction model to obtain a building mask map; The second extraction module is used to use a convolutional neural network to extract features from the collected multi-view images to obtain a multi-view feature map; the obtained multi-view feature map is learned using a pure visual perception algorithm to obtain a BEV image in autonomous driving; The matching module is used to match the building mask image with the BEV image in the autonomous driving to determine the relative position of the target vehicle; The positioning module is used to calculate the latitude and longitude of the relative position of the target vehicle through satellite information to determine the precise position of the target vehicle.

10. A device, It is characterized in that include: Memory for storing computer programs; A processor, configured to implement the method steps according to any one of claims 1 to 8 when executing the computer program.