A high-precision map reconstruction method

Abstract

The present invention discloses a high-precision map reconstruction method, comprising the following steps: extracting the front view features of the surround image through a backbone network; converting the front view features into bird's-eye view features; decoding the front view features and the initialized map vector to obtain the two-dimensional front view object level information and point level information, and using them as formal map prompt words; combining the formal map prompt words, the initialized map vector and the bird's-eye view features, updating the map vector and the bird's-eye view features; obtaining the updated map vector and the bird's-eye view features to generate a high-precision map reconstruction result. The present invention has the beneficial effects of: introducing the object information of the front view image through the UA‑decoder and UI2DPrompt modules, allowing the model to better learn the structure of the scene and increase the generalization ability of the scene.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a high-precision map reconstruction method. Background Art

[0002] With the rapid development of autonomous driving technology, intelligent transportation systems and other fields, the demand for high-precision maps as key infrastructure is growing. Traditional map construction methods are difficult to meet the requirements of high precision, real-time and adaptability. Deep learning technology has made significant progress in image processing and feature extraction, but it still faces many challenges in map reconstruction applications, such as how to effectively integrate multi-view image information, accurately handle uncertainty factors, and improve the accuracy and reliability of reconstructed maps. Summary of the invention

[0003] The purpose of the present invention is to provide a high-precision map reconstruction method to solve the problems raised in the above-mentioned background technology.

[0004] To achieve the above object, the present invention provides the following technical solutions: In a first aspect, the present invention provides a high-precision map reconstruction method, comprising the following steps: Extract the front view features of the surround image through the backbone network; Convert the front view feature to a bird's eye view feature; By decoding the front view features and the initialized map vector, the object level information and point level information of the 2D front view are obtained and used as the formal image prompt words; Combine the formal map prompt words, the initialized map vector and the bird's-eye view features, and update the map vector and the bird's-eye view features; Get updated map vectors and bird's-eye view features to generate high-precision map reconstruction results.

[0005] Preferably, the surround view image refers to an image sequence of the surrounding environment collected by a surround view camera installed on a vehicle or other mobile platform, and the image sequence is preprocessed.

[0006] Preferably, the preprocessing of the image sequence includes image denoising, graying, cropping, and scaling operations to improve the image quality and make it suitable for input into the backbone network.

[0007] Preferably, extracting the front view features of the surround view image through the backbone network means inputting the preprocessed surround view image into the backbone network, and the backbone network extracts features of the image layer by layer according to its pre-trained model parameters and convolutional neural network structure, and finally outputs the front view features.

[0008] Preferably, the step of obtaining the object level information and point level information of the two-dimensional front view by inputting the front view features and the initialized map vector, and using them as formal map prompt words, includes: Initialize the map vector and decode it together with the front view features; Through decoding calculation, the information of the initial reference point and uncertainty of the front view image is obtained, and this information constitutes a part of the front view prompt word; At the same time, the front view features are converted into bird's-eye view features; The front view prompt words, the initialized map vector and the bird's-eye view features are used as two-dimensional uncertainty prompt words to generate updated map vectors and bird's-eye view features to further enrich the prompt word information.

[0009] In a second aspect, the present invention provides a high-precision map reconstruction system for implementing the method of any of the above embodiments, the system comprising: A backbone network, wherein the backbone network is used to extract front view features of the surround view image; A front view to bird's eye view module, which converts front view features into bird's eye view features; A front view uncertainty-aware decoder, wherein the front view uncertainty-aware decoder decodes the front view features and the initialized map vector to obtain the two-dimensional front view object level information and point level information, and uses them as formal image prompt words; A two-dimensional uncertainty prompt word module, wherein the two-dimensional uncertainty prompt word module combines the formal map prompt word, the initialized map vector and the bird's-eye view feature, and updates the map vector and the bird's-eye view feature; A bird's-eye view uncertainty-aware decoder is provided, wherein the bird's-eye view uncertainty-aware decoder obtains updated map vectors and bird's-eye view features and generates a high-precision map reconstruction result.

[0010] In a third aspect, the present invention provides an electronic device comprising at least one processor, wherein the processor is communicatively connected to at least one memory, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in any of the above embodiments.

[0011] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method described in any of the above embodiments when executed.

[0012] Compared with the prior art, the present invention has the following beneficial effects: 1. Improve map accuracy: Through the collaborative work of the backbone network, PV2BEV module and two UA-decoders, it is possible to fully utilize the multi-view information of the surround image, extract and integrate map information from different angles, and thus significantly improve the accuracy of the reconstructed map.

[0013] 2. The UA-attention module in UA-decoder and the cross-attention modules (P2BEV-attention and P2Q-attention) in UI2DPrompt can focus on key features, effectively reduce information loss, and further improve the accuracy of map reconstruction.

[0014] 3. Dealing with uncertainty: Introducing an uncertainty perception mechanism to quantify and process uncertainties during map reconstruction; by calculating the uncertainty of the front view image and incorporating it into the prompt word generation and map reconstruction process, the reliability of the map reconstruction results can be improved, making it better adapted to the complex and changing actual environment.

[0015] 4. Efficiency and adaptability: The method of the present invention is based on a deep learning model and can automatically learn and adapt to the map reconstruction requirements in different scenarios. The design of each module has certain versatility and flexibility, and can be adjusted and optimized according to the actual application scenario, which improves the adaptability and practicality of the method.

[0016] 5. The modular design allows each module to be calculated in parallel or optimized in a targeted manner, which improves the efficiency of the entire map reconstruction process and meets application scenarios with high real-time requirements, such as real-time map updates for autonomous vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a flow chart of a high-precision map reconstruction method according to an embodiment of the present invention; Figure 2 It is a sub-flow chart of step S300; Figure 3 A training and reasoning flow chart of a high-precision map reconstruction system according to an embodiment of the present invention; Figure 4 This is a front view of the uncertainty encoder working principle diagram of the present invention; Figure 5 This is a working principle diagram of the two-dimensional uncertainty prompt word module 400 of the present invention; Figure 6 This is a front view of the uncertainty perception decoder 300 of the present invention; Figure 7 A module diagram of a high-precision map reconstruction system according to an embodiment of the present invention; Figure 8 This is a module diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] See also Figure 1 The present invention provides a technical solution: a high-precision map reconstruction method, comprising the following steps: S100: extracting the front view features of the surround view image through the backbone network; S200: Converting the front view feature into the bird's eye view feature; S300: Obtaining object-level information and point-level information of the two-dimensional front view by decoding the front view features and the initialized map vector, and using them as formal image prompt words; S400: combining the formal map prompt words, the initialized map vector and the bird's-eye view features, and updating the map vector and the bird's-eye view features; S500: Obtain updated map vectors and bird's-eye view features to generate high-precision map reconstruction results.

[0020] In one embodiment of the present invention, the surround view image in step S100 refers to an image sequence of the surrounding environment captured by a surround view camera installed on a vehicle or other mobile platform, and obtained by pre-processing the image sequence.

[0021] In one embodiment of the present invention, the preprocessing of the image sequence includes image denoising, graying, cropping, and scaling operations to improve the image quality and make it suitable for input into the backbone network.

[0022] In one embodiment of the present invention, the backbone network adopts a deep convolutional neural network architecture to extract front view features from the input surround view image. The network is pre-trained and optimized to effectively capture key information in the image and provide rich feature representation for subsequent modules.

[0023] In one embodiment of the present invention, the specific operation of step S100 refers to inputting the preprocessed surround view image into the backbone network, and the backbone network extracts features of the image layer by layer according to its pre-trained model parameters and convolutional neural network structure, and finally outputs the front view features.

[0024] In one embodiment of the present invention, see Figure 2, step S300 specifically includes: S301: Initialize the map vector and decode it together with the front view feature; S302: Obtain information about the initial reference point and uncertainty of the front view image through decoding calculation, and such information constitutes a part of the front view prompt word; S303: Simultaneously converting the front view feature into a bird's eye view feature; S304: Using the front view prompt word, the initialized map vector and the bird's-eye view feature as two-dimensional uncertainty prompt words, generating updated map vectors and bird's-eye view features, and further enriching the prompt word information.

[0025] See also Figure 7 The present application also provides a high-precision map reconstruction system, which includes: A backbone network 100, which is used to extract front view features of the surround view image; A front view to bird's eye view module (PV2BEV) 200, the front view to bird's eye view module converts the front view features into bird's eye view features; Front view uncertainty-aware decoder UA-decoder (PV) 300, the front view uncertainty-aware decoder obtains the two-dimensional front view object-level information and point-level information by decoding the front view features and the initialized map vector, and uses them as formal image prompt words; A two-dimensional uncertainty prompt word module (UI2DPromp) 400, the two-dimensional uncertainty prompt word module combines the formal map prompt word, the initialized map vector and the bird's-eye view feature, and updates the map vector and the bird's-eye view feature; The bird's-eye view uncertainty-aware decoder UA-decoder (BEV) 500 obtains updated map vectors and bird's-eye view features and generates a high-precision map reconstruction result.

[0026] In one embodiment of the present invention, the front view uncertainty-aware decoder 300 includes an uncertainty-aware attention module (UA-attention); the two-dimensional uncertainty prompt word module 400 includes a hybrid injection module (Hybridinjection) 401, a formal bird's-eye view attention module (P2BEV-attention) 402, and a front view conversion vector module (P2Q-attention) 403. Both P2BEV-attention and P2Q-attention are cross-attention modules.

[0027] In one embodiment of the present invention, see Figure 3The system goes through a training and reasoning process as well as a training-only process in the process of implementing a high-precision map reconstruction method in the above-mentioned embodiment.

[0028] In one embodiment of the present invention, see Figure 4 The working principle of the front view uncertainty perception decoder 300 is as follows: the image features extracted by the backbone network of the surround view and the initialized map vector are input into the front view uncertainty perception decoder 300. The initialized map vector is input into the uncertainty perception attention module after the self-attention module and normalization to obtain the updated map vector. The updated map vector and the sampled image features are matrix multiplied to obtain the final map vector. According to this vector, the initial reference point and uncertainty of the front view image can be obtained.

[0029] In one embodiment of the present invention, please refer to 5, the working principle of the two-dimensional uncertainty prompt word module 400: the reference point, uncertainty obtained by the front view uncertainty perception decoder 300 and the BEV features converted by the front view to bird's eye view module 200 are used as the input of the two-dimensional uncertainty prompt word module 400, the reference point is transformed by inverse Perspective Mapping (IPM) to form an initial map vector, the initial map vector and uncertainty are formed by a multilayer perceptron (MLP) and matrix multiplication to form a weight vector of the prompt word, and the weight vector of the prompt word is respectively used in the bird's eye view to bird's eye view attention module 402 and the front view conversion vector module 403 to obtain the updated BEV features and map vector. In the figure, Represents the uncertainty related quantity in the perspective view (PV). It may be an indicator used to quantify the uncertainty of certain elements in the perspective view (such as object position, boundary, etc.), and plays an important role in the process of constructing perspective view prompt words, helping the system to consider and deal with the uncertainty of information. Represents the weight of the front view (PV). This weight is related to the importance of various features and information in the front view. For example, in the front view prompt word construction or subsequent feature fusion process, It is used to adjust the influence of the front view information and determine the proportion of the front view information in the overall processing flow, thereby affecting the final updated map vector and bird's-eye view features. It is the average coordinate in the front view (PV). It can represent the average position coordinate of a certain area (such as road area, object area, etc.) in the front view. When associating and fusing the front view information with map vectors, Position references are provided to facilitate accurate perspective transformation and feature alignment, ensuring that information between the front view and the bird's-eye view can be correctly matched and interacted.

[0030] In one embodiment of the present invention, see Figure 6 The working principle of the bird's-eye view uncertainty perception decoder 500 is as follows: the output of the two-dimensional uncertainty prompt word module 400 is input into the bird's-eye view uncertainty perception decoder 500, and then input into the uncertainty perception attention module UA-attention after passing through the self-attention module and normalization to obtain the updated map vector, and the updated map vector and the sampled image features are matrix multiplied to obtain the final map vector and reconstruct the high-precision map. In the figure, represents the reference vector, represents the mean, Represents variance.

[0031] See also Figure 8 , Figure 8 A schematic diagram of an electronic device 20 that can implement an embodiment of the present invention is shown, and the electronic device is intended to represent various forms of control devices. The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0032] The electronic device 20 includes at least one processor 21, and a memory connected to the at least one processor 21, such as a read-only memory (ROM) 22, a random access memory (RAM) 23, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 21 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 22 or the computer program loaded from the storage unit 28 to the random access memory (RAM) 13. In the RAM 23, various programs and data required for the operation of the electronic device 20 can also be stored. The processor 21, ROM 22 and RAM 23 are connected to each other through a bus 24. An input / output (I / O) interface 25 is also connected to the bus 24.

[0033] A number of components in the electronic device 20 are connected to the I / O interface 25, including: an input unit 26, such as a keyboard, a mouse, etc.; an output unit 27, such as various types of displays, speakers, etc.; a storage unit 28, such as a disk, an optical disk, etc.; and a communication unit 29, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 29 allows the electronic device 20 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0034] The processor 21 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 21 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 21 performs the various methods and processes described above.

[0035] In some embodiments, the method of the above embodiment may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 28. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 20 via the ROM 22 and / or the communication unit 29. When the computer program is loaded into the RAM 23 and executed by the processor 21, one or more steps in the method described above may be performed. Alternatively, in other embodiments, the processor 21 may be configured to perform the method of the above embodiment in any other appropriate manner (e.g., by means of firmware).

[0036] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0037] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0038] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0039] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0040] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0041] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0042] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A high-precision map reconstruction method, characterized in that: The steps include: Extract the front view features of the surround image through the backbone network; Convert the front view feature to a bird's eye view feature; By decoding the front view features and the initialized map vector, the object level information and point level information of the 2D front view are obtained and used as the formal image prompt words; Combine the formal map prompt words, the initialized map vector and the bird's-eye view features, and update the map vector and the bird's-eye view features; Get updated map vectors and bird's-eye view features to generate high-precision map reconstruction results.

2. A high-precision map reconstruction method according to claim 1, characterized in that: The surround view image refers to an image sequence of the surrounding environment collected by a surround view camera installed on a vehicle or other mobile platform, and obtained by preprocessing the image sequence.

3. A high-precision map reconstruction method according to claim 2, characterized in that: The image sequence preprocessing includes image denoising, graying, cropping, and scaling operations to improve image quality and make it suitable for input into the backbone network.

4. A high-precision map reconstruction method according to claim 3, characterized in that: The extracting of the front view features of the surround view image through the backbone network refers to inputting the preprocessed surround view image into the backbone network, and the backbone network extracts features of the image layer by layer according to its pre-trained model parameters and convolutional neural network structure, and finally outputs the front view features.

5. A high-precision map reconstruction method according to claim 4, characterized in that: The object level information and point level information of the two-dimensional front view are obtained by inputting the front view features and the initialized map vector, and they are used as formal map prompt words, including: Initialize the map vector and decode it together with the front view features; Through decoding calculation, the information of the initial reference point and uncertainty of the front view image is obtained, and this information constitutes a part of the front view prompt word; At the same time, the front view features are converted into bird's-eye view features; The front view prompt words, the initialized map vector and the bird's-eye view features are used as two-dimensional uncertainty prompt words to generate updated map vectors and bird's-eye view features to further enrich the prompt word information.

6. A high-precision map reconstruction system, used to implement the method according to any one of claims 1 to 5, characterized in that: The system includes: A backbone network, wherein the backbone network is used to extract front view features of the surround view image; A front view to bird's eye view module, which converts front view features into bird's eye view features; A front view uncertainty-aware decoder, wherein the front view uncertainty-aware decoder decodes the front view features and the initialized map vector to obtain the two-dimensional front view object level information and point level information, and uses them as formal image prompt words; A two-dimensional uncertainty prompt word module, wherein the two-dimensional uncertainty prompt word module combines the formal map prompt word, the initialized map vector and the bird's-eye view feature, and updates the map vector and the bird's-eye view feature; A bird's-eye view uncertainty-aware decoder is provided, wherein the bird's-eye view uncertainty-aware decoder obtains updated map vectors and bird's-eye view features and generates a high-precision map reconstruction result.

7. An electronic device, characterized in that: The method comprises at least one processor, wherein the processor is communicatively connected to at least one memory, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method according to any one of claims 1 to 5 when the processor executes the computer instructions.

Images