A method and apparatus for acquiring ground features in high-precision maps

Abstract

This invention provides a high-precision map ground feature acquisition method and apparatus. The method includes: acquiring ground feature images using a camera on an AR device; obtaining a black-and-white image of the ground features through feature edge detection and color threshold detection; acquiring distance information of the ground features using a ToF camera on the AR device; performing inverse perspective transformation on the ground features in the black-and-white image based on the distance information to generate a bird's-eye view; identifying ground features in the bird's-eye view using a deep learning model and displaying the identification results on the AR device display; generating 2D ground feature data relative to the acquisition operator's position based on direction angle information acquired by a geomagnetic sensor and latitude and longitude information acquired by a positioning device; previewing the 2D ground feature data on the AR device display; and storing the 2D ground feature data after confirming its accuracy. This approach can improve the efficiency of ground feature acquisition, reduce acquisition costs, and facilitate rapid map updates.

Description

Technical Field

[0001] This invention belongs to the field of augmented reality technology, and in particular relates to a method and apparatus for high-precision map ground feature acquisition. Background Technology

[0002] In the field of high-precision map production, the collection of road surface features is crucial, directly impacting mapping efficiency and cost control. Currently, the field data collection of surface features mainly relies on radar and onboard cameras of data collection vehicles to acquire road information. However, updating already collected road data often requires frequent relocation of these vehicles to collect full road data. Since road surface features are frequently updated, using data collection vehicles not only incurs high collection costs but also results in slow map update response times. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a method and apparatus for high-precision map ground feature acquisition, which solves the problems of high cost and slow map update in high-precision map ground feature acquisition.

[0004] In a first aspect of the present invention, a method for acquiring high-precision map ground features is provided, comprising:

[0005] The camera of the AR device captures ground feature images, and the ground feature black and white image is obtained by feature edge detection and color threshold detection;

[0006] The distance information of ground elements is collected by the ToF camera of the AR device, and the ground elements in the black and white image are subjected to inverse perspective transformation based on the distance information to generate a bird's-eye view.

[0007] The ground features in the top view are identified using a deep learning model, and the identification results are displayed on the AR device's screen.

[0008] Based on the direction and angle information collected by the geomagnetic sensor and the latitude and longitude information collected by the positioning device, 2D ground feature data relative to the position of the data collector is generated, and the 2D ground feature data is previewed on the display of the AR device.

[0009] After confirming that the 2D ground feature data is correct, the 2D ground feature is stored.

[0010] In a second aspect of the present invention, a high-precision map ground feature acquisition device is provided, comprising:

[0011] The AR device is used to acquire ground feature images using its camera, and obtain a black and white image of the ground features through feature edge detection and color threshold detection. It also acquires distance information of the ground features using its ToF camera, and performs inverse perspective transformation on the ground features in the black and white image based on this distance information to generate a bird's-eye view. A deep learning model is used to identify the ground features in the bird's-eye view, and the identification results are displayed on the AR device's screen. Furthermore, based on the orientation angle information acquired by the geomagnetic sensor and the latitude and longitude information acquired by the positioning module, 2D ground feature data relative to the operator's position is generated, and this 2D ground feature data is previewed on the AR device's screen. After confirming that the 2D ground feature data is correct, it is stored.

[0012] The positioning module is used to collect the latitude and longitude information of the current location.

[0013] In a third aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor executes the computer program to implement the steps of the method as described in the first aspect of the present invention.

[0014] In a fourth aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method provided in the first aspect of the present invention.

[0015] In this embodiment of the invention, high-precision map ground features are collected on-site using AR devices, enabling flexible collection of ground features. This not only has high collection efficiency and low cost, effectively improving map update efficiency, but also allows for real-time display of collection results, facilitating manual verification and effectively avoiding resampling. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating a high-precision map ground feature acquisition method according to an embodiment of the present invention;

[0018] Figure 2 This is a schematic diagram of the structure of a high-precision map ground feature acquisition device according to an embodiment of the present invention;

[0019] Figure 3 This is a schematic diagram of the structure of an electronic device provided in one embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0021] It should be understood that the terms "comprising" and other similar expressions in the specification, claims, and accompanying drawings of this invention are intended to cover a non-exclusive inclusion, such as a process, method, system, or apparatus that includes a series of steps or units and is not limited to the listed steps or units. Furthermore, "first" and "second" are used to distinguish different objects and are not intended to describe a specific order.

[0022] Please see Figure 1 The present invention provides a flowchart illustrating a method for acquiring ground features in a high-precision map, comprising:

[0023] S101. The camera based on the AR device acquires ground feature images, and obtains a black and white image of the ground features through feature edge detection and color threshold detection;

[0024] The AR device includes at least a regular optical camera, a ToF camera, a geomagnetic sensor, and a display. The ground elements are ground traffic signs, which may include lane lines, stop lines, pedestrian crossings, directional arrows, speed bumps, no-stopping zones, and traffic dividers.

[0025] The data collector points the AR device's camera at the ground features to capture images of the ground features. Through edge detection and color threshold detection, a black and white image of the ground features can be obtained.

[0026] The edge detection is to extract the boundary line between the target and the background through an edge detection algorithm, that is, to identify the area in the image where the pixel gray value changes suddenly through image processing technology; the color threshold detection is based on the identified or set color threshold, and the target (ground element) in the image is separated from the background through color replacement (conversion) to obtain a black and white image.

[0027] S102. Collect distance information of ground elements through the ToF camera of the AR device, and perform inverse perspective transformation on the ground elements in the black and white image based on the distance information to generate a bird's-eye view.

[0028] The ToF (Time of Flight) camera, also known as a depth camera, is used to measure the depth information of an illuminated object by emitting infrared light. Through the ToF camera of an AR device, distance information of ground features can be obtained. Based on this distance information, combined with optical camera parameters and basic ground feature information, an inverse perspective transformation can be performed on the black and white image, converting the camera's viewpoint into a top-down perspective.

[0029] S103. Identify ground features in the top view using a deep learning model and display the identification results on the AR device display.

[0030] By training the deep learning model, different ground features can be identified based on the trained deep learning model. When a top view of a ground feature is input, the deep learning model can detect and output ground feature type information.

[0031] The recognition results include the bounding rectangle of the ground feature and its text description. Based on the recognition results, the bounding rectangle of the ground feature and the ground feature type can be marked on the AR display.

[0032] S104. Generate 2D ground feature data relative to the location of the data collector based on the direction angle information collected by the geomagnetic sensor and the latitude and longitude information collected by the positioning device, and preview the 2D ground feature data on the AR device display.

[0033] The geomagnetic sensor can be used to collect the orientation information of the AR device, and the positioning device is used to locate the AR device or the collector. Based on the AR device's orientation angle information and current position, 2D ground feature data (including ground feature recognition results) relative to the collector's viewpoint can be generated and displayed, and the 2D ground feature data can be displayed on the AR device's display.

[0034] It should be understood that the ground feature recognition results can be displayed in real time on the AR display, and the 2D ground feature data can be previewed on the AR device display for the collector to preview and confirm.

[0035] S105. After confirming that the 2D ground feature data is correct, store the 2D ground feature.

[0036] Once the data collector confirms that the 2D ground feature data is correct, the 2D ground feature can be stored directly.

[0037] Optionally, 2D ground features containing at least orientation and latitude / longitude information are stored, and the data is uploaded via remote wireless communication.

[0038] The map is updated based on the ground feature model, collected direction and angle information, and latitude and longitude information.

[0039] In this embodiment, based on AR equipment and positioning devices, the data collector collects the ground features that need to be updated. The AR equipment identifies and marks the ground features, displays them in real time, and can display them from a top-down perspective. Once the data collector confirms that everything is correct, the data can be stored. Compared to traditional data collection methods, this method is more efficient and less costly in terms of ground feature collection and updating, and can render and display the data in real time, facilitating the confirmation of collection results, ensuring the accuracy of the results, and preventing duplicate data collection.

[0040] It should be noted that the 2D ground feature data collected in this embodiment allows the map update system to directly perform perspective transformations and other fine-tuning of the identified ground features based on the ground feature data, which can then be used for high-precision map updates. This effectively improves the efficiency of high-precision map updates.

[0041] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0042] Figure 2 This is a schematic diagram of a high-precision map ground feature acquisition device provided in an embodiment of the present invention. The device includes:

[0043] AR device 210 is used to acquire ground feature images based on the camera in the AR device, and obtain a black and white image of ground features through feature edge detection and color threshold detection; it acquires distance information of ground features based on the ToF camera in the AR device, and performs inverse perspective transformation on the ground features in the black and white image based on the distance information to generate a bird's-eye view; it identifies ground features in the bird's-eye view through a deep learning model and displays the identification results on the AR device display; it generates 2D ground feature data relative to the position of the data acquisition personnel based on the direction angle information acquired by the geomagnetic sensor in the AR device and the latitude and longitude information acquired by the positioning module, and previews the 2D ground feature data on the AR device display; after confirming that the 2D ground feature data is correct, it stores the 2D ground features.

[0044] The AR device includes at least a camera, a ToF camera, a geomagnetic sensor, and an AR display. The camera is used to collect images of ground features, the ToF camera is used to collect distance information of ground features, the geomagnetic sensor is used to collect direction and angle information relative to the collector, and the AR display is used to display 2D ground features.

[0045] The AR device may further include processing modules such as an image processing module, an inverse perspective transformation module, an element recognition module, a generation module, and a data storage module. The image processing module is used to obtain a black and white image of ground elements through element edge detection and color threshold detection; the inverse perspective transformation module is used to perform inverse perspective transformation on the ground elements in the black and white image to generate a bird's-eye view; the element recognition module is used to identify ground elements in the bird's-eye view based on a deep learning model; the generation module is used to generate 2D ground element data relative to the location of the data collector based on the direction angle information collected by the geomagnetic sensor in the AR device and the latitude and longitude information collected by the positioning module; the data storage module is used to store the 2D ground elements after confirming that the 2D ground element data is correct.

[0046] The recognition results from the feature recognition module can be displayed on the AR display in real time, and the 2D ground features obtained by the generation module can be previewed and displayed on the AR display, making it easy for the data collector to browse and confirm.

[0047] The positioning module 220 is used to collect the latitude and longitude information of the current location.

[0048] The recognition results include the bounding rectangle of the ground features and text descriptions.

[0049] Optionally, storing the 2D ground features further includes:

[0050] It stores 2D ground feature data containing at least direction and latitude / longitude information, and uploads the data based on remote wireless communication;

[0051] The map is updated based on the ground feature model, collected direction and angle information, and latitude and longitude information.

[0052] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the systems and modules described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0053] Figure 3 This is a schematic diagram of an electronic device according to an embodiment of the present invention. The electronic device is used for acquiring high-precision map ground features. Figure 3 As shown, the electronic device 3 of this embodiment includes: a memory 310, a processor 320, and a system bus 330. The memory 310 includes an executable program 3101 stored thereon. As those skilled in the art will understand, Figure 3 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0054] The following is combined Figure 3A detailed introduction to each component of the electronic device:

[0055] The memory 310 can be used to store software programs and modules. The processor 320 executes various functional applications and data processing of the electronic device by running the software programs and modules stored in the memory 310. The memory 310 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device (such as cached data), etc. In addition, the memory 310 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0056] The memory 310 contains an executable program 3101 for a network request method. This executable program 3101 can be divided into one or more modules / units, which are stored in the memory 310 and executed by the processor 320 for purposes such as ground feature acquisition. Each module / unit can be a series of computer program instruction segments capable of performing a specific function, describing the execution process of the computer program 3101 within the electronic device 3. For example, the computer program 3101 can be divided into functional modules such as an image processing module, an inverse perspective transformation module, a feature recognition module, a generation module, and a data storage module.

[0057] The processor 320 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 310, and by calling data stored in the memory 310, it performs various functions and processes data, thereby monitoring the overall status of the electronic device. Optionally, the processor 320 may include one or more processing units; preferably, the processor 320 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, application programs, etc., and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 320.

[0058] The system bus 330 is used to connect various functional components within the computer, transmitting data, address, and control information. Its type can be, for example, a PCI bus, an ISA bus, or a CAN bus. Instructions from the processor 320 are transmitted to the memory 310 via the bus, and the memory 310 sends data back to the processor 320. The system bus 330 is responsible for data and instruction exchange between the processor 320 and the memory 310. Of course, the system bus 330 can also connect to other devices, such as network interfaces and display devices.

[0059] In this embodiment of the invention, the executable program executed by the processing 320 of the electronic device includes:

[0060] The camera of the AR device captures ground feature images, and the ground feature black and white image is obtained by feature edge detection and color threshold detection;

[0061] The distance information of ground elements is collected by the ToF camera of the AR device, and the ground elements in the black and white image are subjected to inverse perspective transformation based on the distance information to generate a bird's-eye view.

[0062] The ground features in the top view are identified using a deep learning model, and the identification results are displayed on the AR device's screen.

[0063] Based on the direction and angle information collected by the geomagnetic sensor and the latitude and longitude information collected by the positioning device, 2D ground feature data relative to the position of the data collector is generated, and the 2D ground feature data is previewed on the display of the AR device.

[0064] After confirming that the 2D ground feature data is correct, the 2D ground feature is stored.

[0065] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0066] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0067] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for acquiring ground features for high-precision maps, characterized in that, include: The camera of the AR device captures ground feature images, and the ground feature black and white image is obtained by feature edge detection and color threshold detection; The distance information of ground elements is collected by the ToF camera of the AR device, and the ground elements in the black and white image are subjected to inverse perspective transformation based on the distance information to generate a bird's-eye view. The ground features in the top view are identified using a deep learning model, and the identification results are displayed on the AR device's screen. Based on the direction and angle information collected by the geomagnetic sensor and the latitude and longitude information collected by the positioning device, 2D ground feature data relative to the position of the data collector is generated, and the 2D ground feature data is previewed on the display of the AR device. After confirming that the 2D ground feature data is correct, the 2D ground feature is stored.

2. The method according to claim 1, characterized in that, The recognition results include the bounding rectangle of the ground features and text descriptions.

3. The method according to claim 1, characterized in that, The storage of the 2D ground features also includes: It stores 2D ground feature data containing at least direction and latitude / longitude information, and uploads the data based on remote wireless communication; The map is updated based on the ground feature model, collected direction and angle information, and latitude and longitude information.

4. A high-precision map ground feature acquisition device, characterized in that, At least include: An AR device is used to acquire ground feature images based on the camera in the AR device, and obtain a black and white image of the ground features through feature edge detection and color threshold detection; a ToF camera in the AR device is used to acquire distance information of the ground features, and inverse perspective transformation is performed on the ground features in the black and white image based on the distance information to generate a bird's-eye view. The deep learning model identifies ground features in the top view and displays the identification results on the AR device's display. Based on the direction angle information collected by the geomagnetic sensor in the AR device and the latitude and longitude information collected by the positioning module, 2D ground feature data relative to the collector's position is generated. The 2D ground feature data is previewed on the AR device's display. After confirming that the 2D ground feature data is correct, the 2D ground features are stored. The positioning module is used to collect the latitude and longitude information of the current location.

5. The apparatus according to claim 4, characterized in that, The recognition results include the bounding rectangle of the ground features and text descriptions.

6. The apparatus according to claim 4, characterized in that, The storage of the 2D ground features also includes: It stores 2D ground feature data containing at least direction and latitude / longitude information, and uploads the data based on remote wireless communication; The map is updated based on the ground feature model, collected direction and angle information, and latitude and longitude information.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of a high-precision map ground feature acquisition method as described in any one of claims 1 to 3.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed, it implements the steps of a high-precision map ground feature acquisition method as described in any one of claims 1 to 3.

Images