Target labeling method and device, server, and computer-readable storage medium

Abstract

The application discloses a target labeling method and device, a server and a computer readable storage medium, and the target labeling method comprises the following steps: acquiring positioning data and image data collected at each moment; for each target image, determining a first coordinate collected at a first collection time matched with a second collection time of the target image; for each target image, calculating third coordinates of each key target on a map coordinate system by using second coordinates of each key target in the target image and a first mapping relationship; and labeling the third coordinates of the key targets in each target image in a three-dimensional map. The above technical scheme can be used to label the key targets in the three-dimensional map in combination with the target image, so that the accuracy of target labeling is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and more particularly, to a target labeling method and device, a server, and a computer readable storage medium. BACKGROUND

[0002] With the continuous maturity of unmanned driving technology, unmanned driving technology is also applied to more and more scenarios. When unmanned driving, an unmanned vehicle needs to rely on a high-precision map to expand the boundary of perception and supplement the ability of perception, so as to complete accurate driving navigation. Therefore, the richness and accuracy of the content in the high-precision map are very important for unmanned driving.

[0003] In the prior art, when labeling a target, data corresponding to the target is labeled in a high-precision map according to data scanned by a laser radar.

[0004] However, in the prior art, when labeling a target, data of a missing target is obtained in a case where the laser radar cannot scan the target, so that the target cannot be labeled in the high-precision map, resulting in incomplete content in the high-precision map. SUMMARY

[0005] The technical problem solved by the present application is how to accurately label each target on a road in a high-precision map, so that the content in the high-precision map is more complete.

[0006] To solve the above technical problem, the present application provides a target labeling method, which comprises: acquiring positioning data and image data collected at each time, the positioning data comprising a first coordinate of a positioning device in a map coordinate system and a first collection time of the first coordinate, and the image data comprising a target image and a second collection time of the target image, the target image comprising key targets, each key target having a second coordinate in the target image; for each target image, determining the first coordinate collected at the first collection time matched with the second collection time of the target image; for each target image, calculating a third coordinate of each key target in the map coordinate system by using the second coordinate of each key target in the target image and a first mapping relationship, the first mapping relationship representing a mapping relationship between each pixel coordinate in the target image and each three-dimensional coordinate in the map coordinate system; and labeling the third coordinate of the key target in each target image in a three-dimensional map, the three-dimensional map being constructed based on the map coordinate system.

[0007] Optionally, the calculating the third coordinates of the key targets in the map coordinate system by using the second coordinates of the key targets in the target image and the first mapping relationship comprises: calculating fourth coordinates of the key targets in a camera coordinate system by using the second coordinates of the key targets in the target image and a second mapping relationship, the second mapping relationship representing a mapping relationship between each pixel coordinate in the target image and each three-dimensional coordinate in the camera coordinate system; calculating fifth coordinates of the key targets in a vehicle coordinate system by using the fourth coordinates of the key targets in the camera coordinate system and a fourth mapping relationship, the fourth mapping relationship representing a mapping relationship between each three-dimensional coordinate in the camera coordinate system and each three-dimensional coordinate in the vehicle coordinate system; and calculating the third coordinates of the key targets in the map coordinate system by using the fifth coordinates of the key targets in the vehicle coordinate system and a fifth mapping relationship, the fifth mapping relationship representing a mapping relationship between each three-dimensional coordinate in the vehicle coordinate system and each three-dimensional coordinate in the map coordinate system.

[0008] Optionally, the second mapping relationship is determined according to intrinsic calibration data of a photographing device used for collecting the target image.

[0009] Optionally, the fourth mapping relationship is determined according to extrinsic calibration data between the photographing device and a positioning device used for collecting the first coordinates; and the fifth mapping relationship is determined according to a rotation relationship and a translation relationship between the vehicle coordinate system and the map coordinate system calculated based on the first coordinates.

[0010] Optionally, the first mapping relationship is determined according to the second mapping relationship, the third mapping relationship and the fourth mapping relationship.

[0011] Optionally, the obtaining of the positioning data and the image data collected at each time comprises: target recognition on the target image to obtain the key targets in the target image.

[0012] Optionally, the calculating the third coordinates of the key targets in the map coordinate system by using the second coordinates of the key targets in the target image and the first mapping relationship comprises: calculating fourth coordinates of the key targets in a camera coordinate system by using the second coordinates of the key targets in the target image and a second mapping relationship, the second mapping relationship representing a mapping relationship between each pixel coordinate in the target image and each three-dimensional coordinate in the camera coordinate system; calculating fifth coordinates of the key targets in the map coordinate system by using the fourth coordinates of the key targets in the camera coordinate system and a third mapping relationship, the third mapping relationship representing a mapping relationship between each three-dimensional coordinate in the camera coordinate system and each three-dimensional coordinate in the map coordinate system.

[0013] The embodiment of the present application also discloses a target labeling device, which comprises: an acquisition module, configured to acquire positioning data and image data collected at each time, wherein the positioning data comprises first coordinates of a positioning device in a map coordinate system and first collection time of the first coordinates, and the image data comprises a target image and second collection time of the target image, the target image comprises key targets, and each key target has second coordinates in the target image; a first coordinate acquisition module, configured to determine, for each target image, the first coordinates collected at the first collection time matched with the second collection time of the target image; a third coordinate acquisition module, configured to calculate, for each target image, third coordinates of each key target in the map coordinate system by using the second coordinates of each key target in the target image and a first mapping relationship, wherein the first mapping relationship represents a mapping relationship between each pixel coordinate in the target image and each three-dimensional coordinate in the map coordinate system; and a labeling module, configured to label the third coordinates of the key targets in each target image in a three-dimensional map, wherein the three-dimensional map is constructed based on the map coordinate system.

[0014] The embodiment of the present application also discloses a server, comprising a memory and a processor, wherein the memory stores a computer program capable of running on the processor, and when the computer program is run by the processor, the steps of the target labeling method described above are executed.

[0015] The embodiment of the present application also discloses a computer readable storage medium, which stores a computer program, wherein the computer readable storage medium is a nonvolatile storage medium or a non-transitory storage medium, and when the computer program is run by a processor, the steps of the target labeling method described above are executed.

[0016] Compared with the prior art, the technical scheme of the embodiment of the present application has the following beneficial effects:

[0017] The present application provides a target labeling method, by acquiring positioning data and image data collected at each time, the positioning data includes the first coordinates of the positioning device in the map coordinate system and the first collection time of the first coordinates, the image data includes the target image and the second collection time of the target image, and the first coordinates collected according to the first collection time matched with the second collection time of the target image can be determined according to the first coordinates and the target image collected at the same time in the map. The precise labeling of the key target is carried out. Then the second coordinates of each key target in each target image and the first mapping relationship are used to calculate the third coordinates of each key target in the map coordinate system, so as to label the key target in each target image in the three-dimensional map according to the third coordinates. According to the labeling of the key target in the target image in the map, the defect that the laser radar cannot obtain all targets due to the limitation of scanning height can be made up, so as to realize the labeling of each target on the road in the high-precision map, and the content in the high-precision map is more complete. And the automatic labeling of the key target in the map coordinate system can be carried out according to the second coordinates and the first mapping relationship, which reduces the human consumption of manual checking of the position of the key target in the high-precision map, and greatly improves the efficiency of target labeling. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 is a whole flow chart of a target labeling method provided by an embodiment of the present application;

[0019] Figure 2 is a whole flow chart of another target labeling method provided by an embodiment of the present application;

[0020] Figure 3 is a structure schematic diagram of a target labeling device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0021] As described in the background, with the application of unmanned driving technology to more and more scenes, the richness and accuracy of high-precision maps are also very important. The unmanned driving car needs to rely on the high-precision map to expand the boundary of perception and supplement the ability of perception in order to complete accurate driving navigation. In the prior art, when labeling the target, the data corresponding to the target in the high-precision map is labeled according to the data scanned by the laser radar. However, in the case that the laser radar cannot scan the target, the data of the missing target will be lost, which leads to the inability to label the target in the high-precision map, resulting in incomplete content in the high-precision map.

[0022] In the embodiment of the present application, by acquiring positioning data and image data collected at each time, the positioning data includes first coordinates of the positioning device in the map coordinate system and first collection time of the first coordinates, and the image data includes a target image and second collection time of the target image, and the first coordinates collected at the first collection time matched with the second collection time of the target image are determined according to the second collection time of the target image, so that the first coordinates and the target image collected at the same time can be used for accurate labeling of key targets in the map. Then, the second coordinates of each key target in each target image and the first mapping relationship are used to calculate third coordinates of each key target in the map coordinate system, so as to label the key targets in each target image in the three-dimensional map according to the third coordinates. According to the labeling of the key targets in the target image in the map, the defect that the laser radar cannot obtain all targets due to the limitation of scanning height can be made up, so that each target on the road in the high-precision map can be labeled, and the content in the high-precision map is more complete. Moreover, the automatic labeling of the key targets in the map coordinate system can be realized according to the second coordinates and the first mapping relationship, the human consumption for manually checking the positions of the key targets in the high-precision map is reduced, and the efficiency of target labeling is greatly improved.

[0023] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0024] Figure 1 It is a whole flowchart of a target labeling method provided by an embodiment of the present application.

[0025] In a specific implementation, the target labeling method described in the following steps 101 to 104 can be used in a server. The above steps can be executed by the server, can be executed by a chip with data processing function in the server, or can be executed by a chip module containing a chip with data processing function. In one specific embodiment, each step of the target labeling method can be executed by the server.

[0026] Specifically, as shown in Figure 1 , the target labeling method can include the following steps:

[0027] In step 101, positioning data and image data collected at each time are acquired.

[0028] In step 102, for each target image, the first coordinates collected at the first collection time matched with the second collection time of the target image are determined.

[0029] In step 103, for each target image, the second coordinates of each key target in the target image and the first mapping relationship are used to calculate third coordinates of each key target in the map coordinate system.

[0030] In step 104, the third coordinates of the key targets in each target image are marked in a three-dimensional map, which is constructed based on the map coordinate system.

[0031] At this point, the key targets have been marked, and the map with completed target marking can be used for unmanned driving.

[0032] It should be noted that the first mapping relationship can be established in advance before the key targets are marked.

[0033] In one non-limiting embodiment, target recognition can be performed on the target image in advance before the positioning data and image data are obtained, to obtain the key targets in the target image.

[0034] It should be noted that the key targets can be recognized using a neural network model, or recognized using other implementable methods, which are not limited by the present application.

[0035] It should also be noted that the key targets can be any implementable objects in the map, such as traffic lights, overpasses, etc., which are not limited by the present application.

[0036] In specific implementation, positioning data and image data collected at each time are obtained, the positioning data including first coordinates of a positioning device in a map coordinate system and a first collection time of the first coordinates, the image data including target images and a second collection time of the target images, the target images including key targets obtained by pre-target recognition, the key targets having second coordinates in the target images. Specifically, the map coordinate system is a coordinate system of a pre-established high-precision map, the first coordinates are three-dimensional coordinates in the map coordinate system, and the second coordinates are two-dimensional coordinates in a pixel coordinate system.

[0037] In a specific implementation, the second acquisition time of each target image is matched with the first acquisition time to obtain the first coordinate acquired by the matched first acquisition time. Specifically, the positioning data acquired by the first acquisition time consistent with the second acquisition time can be matched to obtain the positioning data matched by the target image; or the positioning data acquired by multiple first acquisition times inconsistent with the second acquisition time can be matched, and the multiple positioning data are interpolated to obtain an estimated value of the positioning data, and the estimated value of the positioning data is taken as the positioning data matched by the target image. For example, the second acquisition time of the target image is the 50th ms, and the matched first acquisition time is the 50th ms. The positioning data acquired at the 50th ms is taken as the positioning data matched by the target image; or the second acquisition time of the target image is the 50th ms, and the matched first acquisition time is the 48th ms and the 52nd ms. The positioning data at the 48th ms and the 52nd ms are interpolated to obtain an estimated value of the positioning data at the 50th ms, and the estimated value of the positioning data at the 50th ms is taken as the positioning data matched by the target image.

[0038] It should be noted that linear interpolation can be used for interpolation calculation of the positioning data, or other implementable interpolation methods can be used for interpolation calculation of the positioning data, and the present application does not limit this.

[0039] In a specific implementation of step 103, the fourth coordinates of the key targets in the camera coordinate system are calculated using the second coordinates of the key targets in the target image and a second mapping relationship. The second mapping relationship represents the mapping relationship between each pixel coordinate in the target image and each three-dimensional coordinate in the camera coordinate system. Specifically, the second mapping relationship is determined according to the intrinsic calibration data of the shooting device, and the camera coordinate system is established with the shooting device as the origin, and the shooting device is used to acquire the target image.

[0040] Further, due to the deviation caused by the manufacturing and assembly process of the shooting device, distortion is introduced, which causes the original image to be distorted, so the distortion parameters of the shooting device need to be considered when calculating the fourth coordinates to restore the original image shot. When the three-dimensional coordinate [x c y c z c ] cam When projected on the imaging plane, the normal coordinate should be And the normalized distortion coordinate after distortion caused by the hardware deviation is [x dis y dis 1] cam .

[0041] In a specific implementation, the mapping relationship between the normal coordinate and the distortion coordinate is as follows:

[0042]

[0043]

[0044]

[0045] wherein k1, k2, k3, p1, p2 are distortion parameters of the shooting device, which can be obtained in advance.

[0046] According to the mapping relationship between the normal coordinates and the distortion coordinates and the intrinsic calibration data, the mapping relationship between each two-dimensional coordinate in the distortion coordinate system and the pixel coordinate system can be obtained as follows:

[0047]

[0048] wherein f x , f y , c x , c y are intrinsic parameters of the shooting device, which can be obtained in advance, is an intrinsic matrix, [u v1] pixel is a two-dimensional coordinate in the pixel coordinate system.

[0049] In a specific implementation, the shooting device and the positioning device have a certain distance, and a third mapping relationship needs to be established according to the extrinsic calibration data between the shooting device and the positioning device to perform three-dimensional coordinate conversion between the vehicle coordinate system and the camera coordinate system, and the third mapping relationship represents the mapping relationship between each three-dimensional coordinate in the camera coordinate system and each three-dimensional coordinate in the vehicle coordinate system.

[0050] In a specific implementation, the third mapping relationship can be represented by the following formula:

[0051]

[0052] wherein [offset_x, offset_y, offset_z] is a translation matrix from the vehicle coordinate system to the camera coordinate system, denoted as is a rotation matrix from the vehicle coordinate system to the camera coordinate system, [x y z] car is a three-dimensional coordinate in the vehicle coordinate system, [x c y c z c ] cam is a three-dimensional coordinate in the camera coordinate system.

[0053] In specific implementations, a fourth mapping relationship between each three-dimensional coordinate in the vehicle coordinate system and each three-dimensional coordinate in the map coordinate system can be calculated according to the positioning data and the coordinates of the positioning device in the vehicle coordinate system. The vehicle coordinate system is a coordinate system with the positioning device as the origin. The fourth mapping relationship is determined according to the rotation relationship and the translation relationship between the vehicle coordinate system and the map coordinate system calculated from the first coordinates. The fourth mapping relationship represents the mapping relationship between each three-dimensional coordinate in the vehicle coordinate system and each three-dimensional coordinate in the map coordinate system. The translation relationship between the vehicle coordinate system and the map coordinate system can be obtained from the first coordinates, and the rotation relationship between the vehicle coordinate system and the map coordinate system can be obtained according to the orientation of the positioning device.

[0054] In specific implementations, the fourth mapping relationship can be represented by the following formula:

[0055]

[0056] where [Δx Δy Δz] is the translation matrix from the map coordinate system to the vehicle coordinate system, denoted as is the rotation matrix from the map coordinate system to the vehicle coordinate system, and [x y z] map is the three-dimensional coordinate in the map coordinate system.

[0057] In this embodiment, the second mapping relationship between each two-dimensional coordinate in the pixel coordinate system and each three-dimensional coordinate in the camera coordinate system can be calculated from the intrinsic calibration data and the distortion parameters of the shooting device. The third mapping relationship between each three-dimensional coordinate in the camera coordinate system and each three-dimensional coordinate in the vehicle coordinate system can be calculated from the extrinsic calibration data of the shooting device and the positioning device. The fourth mapping relationship between each three-dimensional coordinate in the vehicle coordinate system and each three-dimensional coordinate in the map coordinate system can be calculated from the positioning data. According to the second mapping relationship, the third mapping relationship, and the fourth mapping relationship, the first mapping relationship between each two-dimensional coordinate in the pixel coordinate system and each three-dimensional coordinate in the map coordinate system can be calculated.

[0058] In specific implementations, the first mapping relationship can be represented by the following formula:

[0059]

[0060] where D is the achievable distortion matrix, and K is the intrinsic matrix

[0061] Further, each target image can obtain a formula 7 containing specific parameters, and for the same key target, the bundle adjustment (BA) method can be used to calculate an optimal solution [x y z] of a key target from multiple formula 7s. mapSo that the residual of the optimal solution is minimum, and the third coordinate of the key target is more accurate.

[0062] In the embodiment, the second coordinates [u v 1] pixel of each key target in the target image are used to calculate the third coordinates [x y z] map of each key target in the map coordinate system according to the first mapping relationship. map And the key targets in each target image are marked in the map coordinate system according to [x y z] map , so as to realize accurate marking of the key targets. The marking of the targets in combination with the data collected by the positioning device and the shooting device can make up for the defect that the laser radar cannot obtain all the targets around due to the limited elevation angle, so that the targets that cannot be scanned by the laser radar can also be marked on the map. In addition, the key targets that have been marked in the map can also be verified in combination with the target image, so that the coordinates of the key targets are more accurate.

[0063] Figure 2 is a specific flowchart of another target marking method provided by the embodiment of the application.

[0064] Different from the above target marking method, in step 203, for each image, the second coordinates of each key target in the target image are used to calculate the fourth coordinates of each key target in the camera coordinate system according to the second mapping relationship.

[0065] In step 204, for each image, the fourth coordinates of each key target in the camera coordinate system are used to calculate the third coordinates of each key target in the map coordinate system according to the fifth mapping relationship.

[0066] In specific implementation, when the coordinates of the shooting device and the positioning device in the map coordinate system are consistent, that is, the shooting device and the positioning device are deployed at the same position on the vehicle, the second coordinates and the fifth mapping relationship between each three-dimensional coordinate in the camera coordinate system and each three-dimensional coordinate in the map coordinate system can be used to calculate the third coordinates, so as to simplify the calculation process.

[0067] In specific implementation, the fifth mapping relationship can be expressed by the following formula:

[0068]

[0069] Wherein, [Delta x1 Delta y1 Delta z1] is the translation matrix from the map coordinate system to the camera coordinate system, denoted as is the rotation matrix from the map coordinate system to the camera coordinate system, [x y z] map is the three-dimensional coordinate in the map coordinate system, and [x y z] camThe three-dimensional coordinates in the camera coordinate system. Specifically, the translation matrix from the map coordinate system to the camera coordinate system and the rotation matrix from the map coordinate system to the camera coordinate system can be obtained from the positioning data collected by the positioning device. For the conversion between coordinate systems, refer to the related description in step 103, which will not be repeated here.

[0070] In the embodiment of the application, according to the second mapping relationship and the fifth mapping relationship described above, the mapping relationship between each pixel coordinate in the target image and each three-dimensional coordinate in the map coordinate system, i.e., the first mapping relationship, can be obtained.

[0071] When the coordinates of the shooting device and the positioning device in the map coordinate system are consistent, the first mapping relationship can be represented by the following formula:

[0072]

[0073] In this embodiment, in the case where the coordinates of the shooting device and the positioning device in the map coordinate system are consistent, the fifth mapping relationship between each three-dimensional coordinate in the camera coordinate system and each three-dimensional coordinate in the map coordinate system can be established to simplify the conversion process of the camera coordinate system, the vehicle coordinate system and the map coordinate system, and reduce the calculation power consumption when calculating the third coordinate.

[0074] In a specific application scenario, due to the limitation of the scanning height of the laser radar, the laser radar may not be able to scan the traffic light, and the laser radar cannot recognize the color of the traffic light, etc., resulting in that the point cloud data obtained by the laser radar scanning does not contain the traffic light. Then, if only the point cloud data is labeled in the high-precision map, the information of the traffic light will be missing in the map. However, the present application can establish the first mapping relationship between each three-dimensional coordinate in the map coordinate system and each pixel coordinate in the pixel coordinate system according to the positioning data of the positioning device in advance. The image data on the road is collected by the shooting device, and the target image containing the traffic light is selected therefrom. The target recognition is performed on the target image to obtain the second coordinate of the traffic light in the pixel coordinate system of the target image. The third coordinate of the traffic light in the map coordinate system can be calculated through the second coordinate and the first mapping relationship. The third coordinate obtained can be used to accurately label the traffic light in the map, so that the content of the high-precision map is more abundant, and the vehicle can be better assisted to perform unmanned driving in actual application.

[0075] As shown in Figure 3 The target labeling device 30 comprises:

[0076] The acquisition module 301 is configured to acquire positioning data and image data collected at respective time points, the positioning data including first coordinates of a positioning device in a map coordinate system and first collection time of the first coordinates, and the image data including a target image and second collection time of the target image, the target image including key targets, each key target having second coordinates in the target image;

[0077] The first coordinate acquisition module 302 is configured to determine, for each target image, the first coordinates collected at the first collection time matched with the second collection time of the target image.

[0078] The third coordinate acquisition module 303 is configured to calculate, for each target image, third coordinates of each key target in the map coordinate system by using the second coordinates of each key target in the target image and a first mapping relationship, the first mapping relationship representing a mapping relationship between each pixel coordinate in the target image and each three-dimensional coordinate in the map coordinate system.

[0079] The labeling module 304 is configured to label the third coordinates of the key targets in each target image in a three-dimensional map, the three-dimensional map being constructed based on the map coordinate system.

[0080] In specific implementations, the target labeling device described above can correspond to a chip with a target labeling function in a server, such as a SOC (System-On-a-Chip), a baseband chip, etc.; or a chip module including a chip with a target labeling function in a server; or a chip module with a target labeling function chip, or a server.

[0081] For more details about the working principle and working mode of the target labeling device 30, please refer to the related description in Figures 1 to 2 , which will not be repeated here.

[0082] As to each device, product or apparatus described in the embodiments, it can be a product independent of other apparatuses, or a component of another apparatus. By means of example, such component can be a module of another apparatus. By means of example, such device, product or apparatus can be implemented as a chip, which can be a component of another apparatus. By means of example, such chip can be implemented as a module of another apparatus, and the module can be a component of another apparatus. There can be many devices, products, or apparatuses that can representively be a chip, a component of another apparatus, a module of another apparatus, or a component of a module of another apparatus. Any combination of the above can also be a device, product, or apparatus of the present application. As to each device, product or apparatus described in the embodiments, it can be a component of another device, product or apparatus. By means of example, one device, product, or apparatus can be a component of another device, product or apparatus. As to each device, product or apparatus described in the embodiments, it can be a component of another device, product or apparatus. By means of example, one device, product, or apparatus can be a component of another device, product or apparatus. As to each device, product or apparatus described in the embodiments, it can be a component of another device, product or apparatus. By means of example, one device, product, or apparatus can be a component of another device, product or apparatus. As to each device, product or apparatus described in the embodiments, it can be a component of another device, product or apparatus. By means of example, one device, product, or apparatus can be a component of another device, product or apparatus.

[0083] The embodiments of the present application further disclose a storage medium, which is a non-volatile storage medium or a non-transitory storage medium, and stores a computer program. The computer program can perform the steps of the method shown in any of the embodiments of the present application when the computer program is run. Figures 1 to 2 The storage medium can include ROM, RAM, magnetic disk or optical disk, etc. The storage medium can also include non-volatile memory or non-transitory memory, etc.

[0084] The embodiments of the present application further disclose a server, which can include a memory and a processor. The memory stores a computer program, which can be run on the processor. The processor can perform the steps of the method shown in any of the embodiments of the present application when running the computer program. Figures 1 to 2

[0085] The term “plurality” appearing in the embodiments of the present application means two or more.

[0086] ​The first, second, and the like appearing in the embodiments of the present application are only for indicating and distinguishing the described objects, and do not have sequence, and do not represent the special limitation of the number of devices in the embodiments of the present application, and cannot constitute any limitation on the embodiments of the present application.

[0087] It should be understood that, in the embodiments of the present application, the processor can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0088] It should also be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct rambus random access memory (DR RAM).

[0089] The above-described embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, the above-described embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are wholly or partially generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center through a wired or wireless manner. The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center, or the like containing one or more available medium collections. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium can be a solid-state disk.

[0090] It should be understood that the size of the sequence number of each process described above in various embodiments of the present application does not mean the order of execution, and the execution order of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0091] In several embodiments provided in the present application, it should be understood that the disclosed methods, devices and systems can be implemented in other ways. For example, the device embodiments described above are only schematic; for example, the division of the units is only a logical function division, and actual implementation can have another division manner; for example, a plurality of units or components can be combined or integrated into another system, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0092] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0093] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically included separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of hardware plus software function unit.

[0094] The integrated unit realized in the form of software function unit can be stored in a computer readable storage medium. The software function unit is stored in a storage medium, and includes a plurality of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute part of steps of the method according to each embodiment of the present application. The storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various storage medium capable of storing program codes.

[0095] Although the present application is disclosed as above, the present application is not limited to this. Any person skilled in the art can make various modifications and changes without departing from the spirit and scope of the present application, and the protection scope of the present application should be defined by the scope defined in the claims.

Claims

1. A target annotation method, characterized in that, include: The system acquires positioning data and image data collected at various times. The positioning data includes the first coordinate of the positioning device in the map coordinate system and the first acquisition time of the first coordinate. The image data includes the target image and the second acquisition time of the target image. The target image includes key targets, and each key target has a second coordinate in the target image. For each target image, determine the first coordinate acquired at the first acquisition time that matches the second acquisition time of the target image; For each target image, the third coordinates of each key target in the target image are calculated in the map coordinate system using the second coordinates of each key target in the target image and the first mapping relationship. The first mapping relationship represents the mapping relationship between each pixel coordinate in the target image and each three-dimensional coordinate in the map coordinate system. The third coordinates of key targets in each target image are marked in a 3D map, which is constructed based on the map coordinate system.

2. The target annotation method according to claim 1, characterized in that, The step of calculating the third coordinate of the key target in the map coordinate system using the second coordinate of the key target in the target image and the first mapping relationship includes: The fourth coordinates of each key target in the camera coordinate system are calculated using the second coordinates and the second mapping relationship of each key target in the target image. The second mapping relationship represents the mapping relationship between each pixel coordinate in the target image and each three-dimensional coordinate in the camera coordinate system. The fifth coordinates of each key target in the vehicle coordinate system are calculated using the fourth coordinates of each key target in the camera coordinate system and the third mapping relationship. The third mapping relationship represents the mapping relationship between each three-dimensional coordinate in the camera coordinate system and each three-dimensional coordinate in the vehicle coordinate system. The third coordinates of each key target in the map coordinate system are calculated using the fifth coordinates of each key target in the vehicle coordinate system and the fourth mapping relationship. The fourth mapping relationship represents the mapping relationship between each three-dimensional coordinate in the vehicle coordinate system and each three-dimensional coordinate in the map coordinate system.

3. The target annotation method according to claim 2, characterized in that, The second mapping relationship is determined based on the intrinsic parameter calibration data of the shooting device, which is used to acquire the target image.

4. The target annotation method according to claim 2, characterized in that, The third mapping relationship is determined based on the external parameter calibration data between the shooting device and the positioning device, wherein the positioning device is used to collect the first coordinates; the fourth mapping relationship is determined based on the rotation and translation relationships between the vehicle coordinate system and the map coordinate system calculated from the first coordinates.

5. The target annotation method according to claim 2, characterized in that, The first mapping relationship is established in the following manner: The first mapping relationship is determined based on the second mapping relationship, the third mapping relationship, and the fourth mapping relationship.

6. The target annotation method according to claim 1, characterized in that, The step of calculating the third coordinate of the key target in the map coordinate system using the second coordinate of the key target in the target image and the first mapping relationship includes: The fourth coordinates of each key target in the camera coordinate system are calculated using the second coordinates and the second mapping relationship of each key target in the target image. The second mapping relationship represents the mapping relationship between each pixel coordinate in the target image and each three-dimensional coordinate in the camera coordinate system. The third coordinates of each key target in the map coordinate system are calculated using the fourth coordinates of each key target in the camera coordinate system and the fifth mapping relationship. The fifth mapping relationship represents the mapping relationship between each three-dimensional coordinate in the camera coordinate system and each three-dimensional coordinate in the map coordinate system.

7. The target annotation method according to claim 1, characterized in that, Prior to acquiring the positioning data and image data collected at various times, the following steps are included: Target recognition is performed on the target image to obtain the key target in the target image.

8. A target marking device, characterized in that, include: The acquisition module is used to acquire positioning data and image data collected at various times. The positioning data includes the first coordinate of the positioning device in the map coordinate system and the first acquisition time of the first coordinate. The image data includes the target image and the second acquisition time of the target image. The target image includes key targets, and each key target has a second coordinate in the target image. The first coordinate acquisition module is used to determine, for each target image, the first coordinate acquired at the first acquisition time that matches the second acquisition time of the target image; The third coordinate acquisition module is used to calculate the third coordinate of each key target in the map coordinate system for each target image using the second coordinate of each key target in the target image and the first mapping relationship. The first mapping relationship represents the mapping relationship between each pixel coordinate in the target image and each three-dimensional coordinate in the map coordinate system. The annotation module is used to annotate the third coordinates of key targets in each target image in a 3D map, the 3D map being constructed based on the map coordinate system.

9. A server comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor runs the computer program, it performs the steps of the target annotation method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer-readable storage medium is a non-volatile or non-transient storage medium, and the computer program, when executed by a processor, performs the steps of the target annotation method according to any one of claims 1 to 7.

Images