A method and device for labeling perception data in fusion of V2X technology

Abstract

The application discloses a kind of perception data labeling method and device fusing V2X technology, and the present application relates to intelligent network connection technical field, wherein including: obtaining the data to be labeled collected by vehicle-mounted perception terminal when network connection car drives, and V2X message data collected by vehicle-mounted communication terminal, the data to be labeled includes video image data and radar point cloud data;From the V2X message data, the coordinate information of target object, three-dimensional boundary box information and semantic information are extracted;The coordinate information and the three-dimensional boundary box information are mapped to the coordinate system of perception space as label data, to obtain the mapping boundary box of the target object in the data to be labeled;Based on the pixel region and point cloud space corresponding to the mapping boundary box, and the semantic information, the target object is respectively labeled with geometric range and attribute.The present application can improve data labeling precision and ensure data labeling quality.

Description

Technical Field

[0001] This invention relates to the field of intelligent connected vehicle technology, and more specifically, to a method and apparatus for labeling sensing data that integrates V2X technology. Background Technology

[0002] Intelligent connected vehicles are developing towards intelligence and connectivity. The realization of vehicle intelligence relies on highly accurate and reliable perception technologies. The iteration of perception technologies mainly relies on sensor data fusion, and data annotation and training are the key to perception technologies.

[0003] Currently, the common approach is to manually annotate perceived data. However, due to the limited amount of annotation content obtained manually, it is difficult to provide true information that closely approximates the actual target, thus failing to guarantee the accuracy of the annotation and resulting in poor data quality after annotation. Summary of the Invention

[0004] This invention provides a method and apparatus for labeling sensing data that integrates V2X technology, which mainly improves the accuracy of data labeling and ensures the quality of data labeling.

[0005] According to a first aspect of the present invention, a method for annotating perception data integrating V2X technology is provided, comprising:

[0006] The system acquires labeled data collected by the vehicle-mounted sensing terminal during vehicle operation, as well as V2X message data collected by the vehicle-mounted communication terminal. The labeled data includes video image data and radar point cloud data.

[0007] Extract the coordinate information, 3D bounding box information, and semantic information of the target object from the V2X message data;

[0008] The coordinate information and the three-dimensional bounding box information are mapped as label data to the coordinate system of the perceptual space to obtain the mapped bounding box of the target object in the data to be labeled.

[0009] Based on the pixel region and point cloud space corresponding to the mapped bounding box, as well as the semantic information, the target object is labeled with geometric range and attributes respectively.

[0010] According to a second aspect of the present invention, a sensory data annotation apparatus integrating V2X technology is provided, comprising:

[0011] The acquisition unit is used to acquire the data to be labeled collected by the vehicle-mounted sensing terminal when the connected vehicle is driving, as well as the V2X message data collected by the vehicle-mounted communication terminal. The data to be labeled includes video image data and radar point cloud data.

[0012] The extraction unit is used to extract the coordinate information, three-dimensional bounding box information and semantic information of the target object from the V2X message data;

[0013] The mapping unit is used to map the coordinate information and the three-dimensional bounding box information as label data to the coordinate system of the perceptual space to obtain the mapped bounding box of the target object in the data to be labeled.

[0014] The annotation unit is used to perform geometric range annotation and attribute annotation on the target object based on the pixel region and point cloud space corresponding to the mapped bounding box and the semantic information.

[0015] According to a third aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, performs the following steps:

[0016] The system acquires labeled data collected by the vehicle-mounted sensing terminal during vehicle operation, as well as V2X message data collected by the vehicle-mounted communication terminal. The labeled data includes video image data and radar point cloud data.

[0017] Extract the coordinate information, 3D bounding box information, and semantic information of the target object from the V2X message data;

[0018] The coordinate information and the three-dimensional bounding box information are mapped as label data to the coordinate system of the perceptual space to obtain the mapped bounding box of the target object in the data to be labeled.

[0019] Based on the pixel region and point cloud space corresponding to the mapped bounding box, as well as the semantic information, the target object is labeled with geometric range and attributes respectively.

[0020] According to a fourth aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to perform the following steps:

[0021] The system acquires labeled data collected by the vehicle-mounted sensing terminal during vehicle operation, as well as V2X message data collected by the vehicle-mounted communication terminal. The labeled data includes video image data and radar point cloud data.

[0022] Extract the coordinate information, 3D bounding box information, and semantic information of the target object from the V2X message data;

[0023] The coordinate information and the three-dimensional bounding box information are mapped as label data to the coordinate system of the perceptual space to obtain the mapped bounding box of the target object in the data to be labeled.

[0024] Based on the pixel region and point cloud space corresponding to the mapped bounding box, as well as the semantic information, the target object is labeled with geometric range and attributes respectively.

[0025] The innovative aspects of this invention include:

[0026] 1. This invention can extract rich, comprehensive and accurate coordinate information, three-dimensional bounding box information and semantic information of target objects by collecting V2X interaction data, and use them as tag data.

[0027] 2. In this invention, vehicle-mounted communication terminals at different locations and at different times will provide a V2X message sample for observing the target object, thereby providing observation samples from multiple perspectives for a single target object and increasing the diversity of labeled samples.

[0028] 3. Data fusion annotation can be applied on the vehicle or in the cloud, and can be annotated in real time or offline, meaning that this invention can achieve diverse annotation methods.

[0029] This invention provides a perception data annotation method and apparatus integrating V2X technology. Compared with existing technologies, it can acquire data to be annotated collected by the vehicle-mounted perception terminal during the operation of a connected vehicle, as well as V2X message data collected by the vehicle-mounted communication terminal. The data to be annotated includes video image data and radar point cloud data. The method extracts the coordinate information, 3D bounding box information, and semantic information of the target object from the V2X message data. At the same time, the coordinate information and the 3D bounding box information are mapped as label data to the coordinate system of the perception space to obtain the mapped bounding box of the target object in the data to be annotated. Finally, based on the pixel region and point cloud space corresponding to the mapped bounding box, as well as the semantic information, the method performs geometric range annotation and attribute annotation on the target object. Therefore, this invention, in the process of acquiring V2X message data using vehicle-to-everything (V2X) information interaction technology and mapping the high-precision information of the target object extracted from the V2X message data to the perception data, can obtain multi-view observation information of the target object by aggregating V2X broadcast messages from different communication terminals. This results in rich and accurate label data, providing ground truth information close to the actual target for data annotation, thereby improving data annotation accuracy and ensuring the quality of the annotated data. Furthermore, compared to manual annotation methods, this invention can also achieve automated annotation of perception data, improving data annotation efficiency.

[0030] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This diagram illustrates a flow chart of a perception data annotation method integrating V2X technology provided by an embodiment of the present invention.

[0033] Figure 2 This diagram illustrates V2X message data provided in an embodiment of the present invention.

[0034] Figure 3 This diagram illustrates the structure of a sensing data annotation device integrating V2X technology according to an embodiment of the present invention.

[0035] Figure 4 A schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention is shown; Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] It should be noted that the terms "comprising" and "having," and any variations thereof, in the embodiments and drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0038] Manually obtained annotations are limited and it is difficult to provide true information that closely approximates the actual target for data annotation. Therefore, the accuracy of data annotation cannot be guaranteed, and the quality of the annotated data is poor.

[0039] To overcome the above-mentioned shortcomings, embodiments of the present invention provide a method for labeling sensory data that integrates V2X technology, such as... Figure 1 As shown, the method includes:

[0040] Step 101: Obtain the data to be labeled collected by the vehicle-mounted sensing terminal when the connected vehicle is in motion, and the V2X message data collected by the vehicle-mounted communication terminal.

[0041] The data to be labeled includes video image data and radar point cloud data. V2X message data includes Basic Safety Message (BSM), Map Message (MAP), Traffic Light Phase and Timing Message (SPAT), Roadside Information (RSI), and Roadside Safety Message (RSM). Furthermore, the onboard perception terminal includes visual cameras and LiDAR, among others.

[0042] This invention is primarily applicable to scenarios where V2X message data is used to annotate sensing data. The executing entity of this invention is a device or equipment capable of annotating sensing data using V2X message data, such as one installed on the vehicle side or the cloud side.

[0043] Specifically, connected vehicles can collect video image data and radar point cloud data through vision cameras and LiDAR while driving, and use the video image data and radar point cloud data as data to be labeled. In addition, it can also collect the timestamps corresponding to the video image data and radar point cloud data, as well as the location information of the connected vehicle at the corresponding time.

[0044] Step 102: Extract the coordinate information, 3D bounding box information and semantic information of the target object from the V2X message data.

[0045] The target objects primarily include traffic participants and basic transportation infrastructure. Traffic participants include vehicles, pedestrians, bicycles, and electric vehicles, while basic transportation infrastructure includes roads, traffic signs, and traffic lights. Furthermore, the coordinate information of the target objects specifically refers to latitude and longitude coordinates, and the 3D bounding box information specifically refers to the geometric shape and size of the target objects, such as length, width, and height. Semantic information specifically refers to the target objects' color, type, speed, heading angle, and driving status, as well as road topology data, traffic light status, roadside event types, impact range, duration, speed limit information, and driving instruction information. Further, V2X message data can originate from vehicles, roadside RSUs and MECs, and 4G / 5G communication base stations.

[0046] In this embodiment of the invention, after collecting V2X message data, it is necessary to filter out V2X message data associated with the data to be labeled from the collected V2X message data. For this process, the method includes: obtaining the timestamp corresponding to the data to be labeled, and the location information of the connected vehicle at the corresponding time; determining the spatiotemporal range in which the data to be labeled was generated based on the timestamp and the location information of the connected vehicle at the corresponding time; and filtering out V2X message data associated with the data to be labeled based on the spatiotemporal range.

[0047] For example, radar point cloud data, video image data, and V2X message data located on road segment A and within the same time period can be filtered out. The filtered radar point cloud data and video image data are correlated with the V2X message data and belong to the same spatiotemporal range. By using timestamps and vehicle location information, a one-to-one correspondence can be established between the video image data and radar point cloud data and the V2X message data.

[0048] It should be noted that the above filtering and matching process can be carried out in real time on the vehicle, or the data can be uploaded to the cloud and then filtered and matched.

[0049] Furthermore, after filtering out the V2X message data associated with the data to be labeled, the coordinate information, 3D bounding box information, and semantic information of the target object can be extracted from the V2X message data associated with the data to be labeled. For this process, the method includes: parsing each message according to the message type of each message contained in the V2X message data in an appropriate manner to obtain the parsed data corresponding to each message; and determining the coordinate information, 3D bounding box information, and semantic information of the target object based on the parsed data corresponding to each message.

[0050] Further, based on the message type of each message contained in the V2X message data, the messages are parsed in a corresponding manner to obtain the parsed data corresponding to each message, including: determining the three-dimensional bounding box information of the target object based on the coordinate field and shape / size field in the Basic Safety Message (BSM), and using the three-dimensional bounding box information of the target object as the parsed data of the Basic Safety Message (BSM); determining the road topology data around the connected vehicle based on the lane topology information and lane trajectory coordinate information in the Map Message (MAP), and using the road topology data as the parsed data of the Map Message (MAP); and based on the traffic light timing information in the Traffic Light Phase and Timing Message (SPAT) and the map data... The system uses a Mapping Context (MAP) to determine the traffic light status perceived by the connected vehicle at the intersection, and identifies the traffic light status as parsed data for the Traffic Light Phase and Timing Message (SPAT). It also reads the roadside event type, impact range, duration, speed limit information, and driving instruction information from the Roadside Information Resource Sequence (RSI), and identifies these information as parsed data for the RSI. Furthermore, based on the traffic safety information in the Basic Safety Message (BSM), it determines the coordinate information, geometry, and status data of target objects around the connected vehicle, and identifies these information as parsed data for the BSM.

[0051] Specifically, such as Figure 2 As shown, for the Basic Safety Message (BSM), the pos coordinate field and size field data are read to obtain the 3D bounding box information of the target object; for the Map Message (MAP), the lane topology information and lane trajectory coordinate data are read to obtain the road topology data around the connected vehicle; for the Traffic Light Phase and Timing Message (SPAT), the traffic light timing information is read in conjunction with the MAP message data to obtain the traffic light color and timing data perceived by the connected vehicle at the traffic light intersection; for the Roadside Information Message (RSI), the retel data is read, including semantic information such as roadside event type, occurrence range, duration, speed limit, and driving instructions. The position in the Roadside Information Message (RSI) is generally given in the form of reference coordinates refpos and offset, and the actual global coordinate is equal to the offset plus the reference coordinate; for the Roadside Safety Message (RSM), the traffic safety related information is read, including the position and size data of key traffic participants (such as pedestrians and vehicles) around the vehicle, thereby obtaining the position, geometry, and status data of traffic participants around the connected vehicle.

[0052] Furthermore, after obtaining the coordinate information and 3D bounding box information of the target object, they can be uniformly transformed to a global coordinate system through coordinate transformation, and the measurement units can be unified to obtain the label data. Based on this, the method includes: uniformly transforming the coordinate information and 3D bounding box information of the target object to a global coordinate system to obtain the transformed coordinate information and 3D bounding box information.

[0053] Step 103: Map the coordinate information and the three-dimensional bounding box information as label data to the coordinate system of the perceptual space to obtain the mapped bounding box of the target object in the data to be labeled.

[0054] In this embodiment of the invention, after unifying the coordinate information and the 3D bounding box information into the global coordinate system, it is necessary to map the coordinate information and the 3D bounding box information to the 2D image plane and the 3D point cloud space, respectively. For this process, the method includes: determining the position information and attitude data of the connected vehicle based on the V2X message data; determining the mapping relationship between the coordinate system of the perception space and the global coordinate system based on the position information and attitude data of the connected vehicle, and the extrinsic and intrinsic parameters of the vehicle-mounted sensing terminal; and mapping the coordinate information and the 3D bounding box information as label data to the coordinate system of the perception space, thereby obtaining the mapped bounding box of the target object in the data to be labeled.

[0055] Furthermore, based on the location information and attitude data of the connected vehicle, as well as the extrinsic and intrinsic parameters of the vehicle-mounted sensing terminal, the mapping relationship between the coordinate system of the sensing space and the global coordinate system is determined, including: based on the location and attitude data of the connected vehicle, as well as the extrinsic and intrinsic parameters of the vision camera, the mapping relationship between the visual camera coordinate system and the global coordinate system is determined; based on the location and attitude data of the connected vehicle, as well as the extrinsic and intrinsic parameters of the lidar, the mapping relationship between the lidar coordinate system and the global coordinate system is determined.

[0056] Simultaneously, based on the mapping relationship, the coordinate information and the 3D bounding box information are mapped as label data to the coordinate system of the perception space to obtain the mapped bounding box of the target object in the data to be labeled. This includes: based on the mapping relationship between the visual camera coordinate system and the global coordinate system, mapping the coordinate information and the 3D bounding box information as label data to the visual camera coordinate system to obtain the mapped bounding box of the target object in the video image data; and based on the mapping relationship between the lidar coordinate system and the global coordinate system, mapping the coordinate information and the 3D bounding box information as label data to the lidar coordinate system to obtain the mapped bounding box of the target object in the lidar point cloud data.

[0057] Step 104: Based on the pixel region and point cloud space corresponding to the mapped bounding box, and the semantic information, perform geometric range annotation and attribute annotation on the target object respectively.

[0058] In this embodiment of the invention, after obtaining the mapped bounding boxes of the target object in video image data and radar point cloud data, a series of pixels within the mapped bounding boxes can be labeled as two-dimensional geometric data of the target object, and semantic information can be labeled as semantic tags of the target object in the video image data. Similarly, the point cloud space within the mapped bounding boxes can be labeled as three-dimensional geometric data of the target object, and semantic information can be labeled as semantic tags of the target object in the radar point cloud data.

[0059] It should be noted that in a fully connected environment, all elements can be labeled using the above methods; however, in a non-fully connected environment, the above methods can be used as auxiliary labeling methods. Specifically, target objects can be pre-labeled through image recognition or manual labeling, and then supplemented and enhanced by vehicle-to-everything (V2X) labeling methods.

[0060] This invention provides a perception data annotation method integrating V2X technology. In the process of acquiring V2X message data using vehicle-to-everything (V2X) information interaction technology and mapping high-precision information of the target object extracted from the V2X message data to the perception data, this invention aggregates V2X broadcast messages from different communication terminals. Therefore, it can acquire observation information of the target object from multiple perspectives, resulting in rich and accurate label data. This provides ground truth information close to the actual target for data annotation, thereby improving data annotation accuracy and ensuring the quality of the annotated data. Furthermore, compared to manual annotation methods, this invention can also achieve automated annotation of perception data, improving data annotation efficiency.

[0061] Furthermore, as Figure 1 In specific implementation, embodiments of the present invention provide a sensing data annotation device integrating V2X technology, such as... Figure 3 As shown, the device includes: an acquisition unit 31, an extraction unit 32, a mapping unit 33, and a labeling unit 34.

[0062] The acquisition unit 31 can be used to acquire the data to be labeled collected by the vehicle-mounted sensing terminal when the connected vehicle is driving, as well as the V2X message data collected by the vehicle-mounted communication terminal. The data to be labeled includes video image data and radar point cloud data.

[0063] The extraction unit 32 can be used to extract the coordinate information, three-dimensional bounding box information and semantic information of the target object from the V2X message data.

[0064] The mapping unit 33 can be used to map the coordinate information and the three-dimensional bounding box information as label data to the coordinate system of the perceptual space to obtain the mapped bounding box of the target object in the data to be labeled.

[0065] The annotation unit 34 can be used to perform geometric range annotation and attribute annotation on the target object based on the pixel region and point cloud space corresponding to the mapped bounding box and the semantic information.

[0066] In a specific application scenario, the extraction unit 32 includes: a parsing module and a first determining module.

[0067] The parsing module can be used to parse each message according to the message type of each message contained in the V2X message data, and obtain the parsed data corresponding to each message.

[0068] The first determining module can be used to determine the coordinate information, three-dimensional bounding box information and semantic information of the target object based on the parsed data corresponding to each message.

[0069] In specific application scenarios, when the messages include basic safety messages, map messages, traffic light phase and timing messages, roadside information, and roadside safety messages, the parsing module can be specifically used to determine the three-dimensional bounding box information of the target object based on the coordinate and shape / size fields in the basic safety message, and to determine the three-dimensional bounding box information of the target object as the parsing data of the basic safety message; to determine the road topology data around the connected vehicle based on the lane topology information and lane trajectory coordinate information in the map message, and to determine the road topology data as the parsing data of the map message; and to determine the traffic light timing information and the map based on the traffic light phase and timing messages. The system retrieves and interprets traffic light statuses perceived by the connected vehicle at intersections, and uses these statuses as parsing data for the traffic light phase and timing messages. It also reads roadside event types, impact ranges, durations, speed limits, and driving instructions from roadside information, and uses these information as parsing data for the roadside information. Furthermore, based on traffic safety information in the basic safety messages, it determines the coordinates, geometry, and status data of target objects around the connected vehicle, and uses these information as parsing data for the basic safety messages.

[0070] In specific application scenarios, the device further includes a screening unit.

[0071] The filtering unit can be used to obtain the timestamp corresponding to the data to be labeled, and the location information of the connected vehicle at the corresponding time; determine the spatiotemporal range of the data to be labeled based on the timestamp and the location information of the connected vehicle at the corresponding time; and filter out V2X message data associated with the data to be labeled based on the spatiotemporal range.

[0072] The extraction unit 32 can be specifically used to extract the coordinate information, three-dimensional bounding box information and semantic information of the target object from the V2X message data associated with the data to be labeled.

[0073] In specific application scenarios, the device further includes a conversion unit.

[0074] The conversion unit can be used to uniformly convert the coordinate information and three-dimensional bounding box information of the target object to the global coordinate system to obtain the converted coordinate information and three-dimensional bounding box information.

[0075] The mapping unit 33 can be specifically used to map the transformed coordinate information and three-dimensional bounding box information as label data to the coordinate system of the perceptual space, so as to obtain the mapped bounding box of the target object in the data to be labeled.

[0076] In a specific application scenario, the mapping unit 33 includes: a second determining module and a mapping module.

[0077] The second determining module can be used to determine the location information and attitude data of the connected vehicle based on the V2X message data.

[0078] The second determining module can also be used to determine the mapping relationship between the coordinate system of the sensing space and the global coordinate system based on the location information and attitude data of the connected vehicle, as well as the extrinsic and intrinsic parameters of the vehicle-mounted sensing terminal.

[0079] The mapping module can be used to map the coordinate information and the three-dimensional bounding box information as label data to the coordinate system of the perception space based on the mapping relationship, so as to obtain the mapped bounding box of the target object in the data to be labeled.

[0080] Furthermore, the second determining module can be specifically used to determine the mapping relationship between the visual camera coordinate system and the global coordinate system based on the position and attitude data of the connected vehicle and the extrinsic and intrinsic parameters of the visual camera; and to determine the mapping relationship between the lidar coordinate system and the global coordinate system based on the position and attitude data of the connected vehicle and the extrinsic and intrinsic parameters of the lidar.

[0081] The mapping module can be specifically used to map the coordinate information and the 3D bounding box information as label data to the visual camera coordinate system based on the mapping relationship between the visual camera coordinate system and the global coordinate system, thereby obtaining the mapped bounding box of the target object in the video image data; and based on the mapping relationship between the lidar coordinate system and the global coordinate system, to map the coordinate information and the 3D bounding box information as label data to the lidar coordinate system, thereby obtaining the mapped bounding box of the target object in the lidar point cloud data.

[0082] It should be noted that other corresponding descriptions of the functional modules involved in the sensing data annotation device integrating V2X technology provided in this embodiment of the invention can be found in [reference]. Figure 1 The corresponding description of the method shown will not be repeated here.

[0083] Based on the above, Figure 1Correspondingly, this embodiment of the invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the following steps: acquiring data to be labeled collected by an onboard perception terminal during vehicle operation, and V2X message data collected by an onboard communication terminal, wherein the data to be labeled includes video image data and radar point cloud data; extracting coordinate information, three-dimensional bounding box information, and semantic information of a target object from the V2X message data; mapping the coordinate information and the three-dimensional bounding box information as label data to the coordinate system of the perception space to obtain the mapped bounding box of the target object in the data to be labeled; and performing geometric range labeling and attribute labeling on the target object based on the pixel region and point cloud space corresponding to the mapped bounding box, and the semantic information.

[0084] Based on the above, Figure 1 The method shown and as Figure 3 The embodiment of the device shown in the invention also provides a physical structural diagram of an electronic device, such as... Figure 4 As shown, the electronic device includes a processor 41, a memory 42, and a computer program stored in the memory 42 and executable on the processor. Both the memory 42 and the processor 41 are mounted on a bus 43. When the processor 41 executes the program, it performs the following steps: acquiring data to be labeled collected by an onboard perception terminal during vehicle operation, and V2X message data collected by an onboard communication terminal. The data to be labeled includes video image data and radar point cloud data. It extracts coordinate information, 3D bounding box information, and semantic information of the target object from the V2X message data. It maps the coordinate information and the 3D bounding box information as label data to the coordinate system of the perception space to obtain the mapped bounding box of the target object in the data to be labeled. Based on the pixel region and point cloud space corresponding to the mapped bounding box, and the semantic information, it performs geometric range labeling and attribute labeling on the target object.

[0085] This invention utilizes vehicle-to-everything (V2X) message data acquisition technology and maps high-precision target object information extracted from the V2X message data onto sensing data. Because it aggregates V2X broadcast messages from different communication terminals, it can acquire multi-view observation information of the target object, thereby obtaining rich and accurate label data. This provides ground truth information close to the actual target for data annotation, improving annotation accuracy and ensuring the quality of the annotated data. Furthermore, compared to manual annotation methods, this invention can also automate the annotation of sensing data, improving annotation efficiency.

[0086] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing the present invention.

[0087] Those skilled in the art will understand that the modules in the apparatus of the embodiments can be distributed in the apparatus of the embodiments as described in the embodiments, or they can be located in one or more devices different from this embodiment with corresponding changes. The modules of the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0088] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for labeling sensory data integrating V2X technology, characterized in that, include: The system acquires labeled data collected by the vehicle-mounted sensing terminal during vehicle operation, as well as V2X message data collected by the vehicle-mounted communication terminal. The labeled data includes video image data and radar point cloud data. Extract the coordinate information, 3D bounding box information, and semantic information of the target object from the V2X message data; The coordinate information and the three-dimensional bounding box information are mapped as label data to the coordinate system of the perceptual space to obtain the mapped bounding box of the target object in the data to be labeled. Based on the pixel region and point cloud space corresponding to the mapped bounding box, as well as the semantic information, the target object is labeled with geometric range and attributes respectively.

2. The method according to claim 1, characterized in that, Extracting the target object's coordinate information, 3D bounding box information, and semantic information from the V2X message data includes: Based on the message type of each message contained in the V2X message data, each message is parsed in a corresponding manner to obtain the parsed data corresponding to each message; Based on the parsed data corresponding to each message, the coordinate information, 3D bounding box information, and semantic information of the target object are determined.

3. The method according to claim 2, characterized in that, When the messages include basic safety messages, map messages, traffic light phase and timing messages, roadside information, and roadside safety messages, the messages are parsed according to their message types in the V2X message data, using appropriate methods to obtain the parsed data corresponding to each message, including: Based on the coordinate and shape / size fields in the basic security message, the three-dimensional bounding box information of the target object is determined, and the three-dimensional bounding box information of the target object is used as the parsed data of the basic security message. Based on the lane topology information and lane trajectory coordinate information in the map message, the road topology data around the connected vehicle is determined, and the road topology data is used as the parsed data of the map message. Based on the traffic light timing information in the traffic light phase and timing message and the map message, the traffic light status perceived by the connected vehicle at the intersection is determined, and the traffic light status is determined as the parsed data of the traffic light phase and timing message; Read the roadside event type, impact range, duration, speed limit information, and driving instruction information from the roadside information, and determine the roadside event type, impact range, duration, speed limit information, and driving instruction information as the parsed data of the roadside information; Based on the traffic safety information in the basic safety message, the coordinate information, geometric shape, and status data of the target objects around the connected vehicle are determined, and the coordinate information, geometric shape, and status data of the target objects are used as the parsed data of the basic safety message.

4. The method according to any one of claims 1-3, characterized in that, After extracting the coordinate information, 3D bounding box information, and semantic information of the target object from the V2X message data, the method further includes: Obtain the timestamp corresponding to the data to be labeled, and the location information of the connected vehicle at the corresponding time; Based on the timestamp and the location information of the connected vehicle at the corresponding time, the spatiotemporal range of the data to be labeled is determined; Based on the aforementioned spatiotemporal range, V2X message data associated with the data to be labeled are filtered out; The extraction of the target object's coordinate information, 3D bounding box information, and semantic information from the V2X message data includes: Extract the coordinate information, 3D bounding box information, and semantic information of the target object from the V2X message data associated with the data to be labeled.

5. The method according to any one of claims 1-4, characterized in that, After extracting the coordinate information, 3D bounding box information, and semantic information of the target object from the V2X message data, the method further includes: The coordinate information and 3D bounding box information of the target object are uniformly transformed to the global coordinate system to obtain the transformed coordinate information and 3D bounding box information. The step of mapping the coordinate information and the 3D bounding box information as label data to the coordinate system of the perceptual space to obtain the mapped bounding box of the target object in the data to be labeled includes: The transformed coordinate information and 3D bounding box information are used as label data and mapped onto the coordinate system of the perceptual space to obtain the mapped bounding box of the target object in the data to be labeled.

6. The method according to any one of claims 1-5, characterized in that, Mapping the coordinate information and the 3D bounding box information as label data onto the coordinate system of the perceptual space to obtain the mapped bounding box of the target object in the data to be labeled, including: Based on the V2X message data, the location information and attitude data of the connected vehicle are determined; Based on the location information and attitude data of the connected vehicle, as well as the extrinsic and intrinsic parameters of the vehicle-mounted sensing terminal, the mapping relationship between the coordinate system of the sensing space and the global coordinate system is determined. Based on the mapping relationship, the coordinate information and the three-dimensional bounding box information are mapped as label data to the coordinate system of the perception space to obtain the mapped bounding box of the target object in the data to be labeled.

7. The method according to claim 6, characterized in that, The vehicle-mounted sensing terminal includes a visual camera and a lidar. Determining the mapping relationship between the coordinate system of the sensing space and the global coordinate system based on the location information and attitude data of the connected vehicle, as well as the extrinsic and intrinsic parameters of the vehicle-mounted sensing terminal, includes: Based on the position and attitude data of the connected vehicle, as well as the extrinsic and intrinsic parameters of the vision camera, the mapping relationship between the vision camera coordinate system and the global coordinate system is determined. Based on the position and attitude data of the connected vehicle, as well as the extrinsic and intrinsic parameters of the lidar, the mapping relationship between the lidar coordinate system and the global coordinate system is determined; Based on the mapping relationship, the coordinate information and the 3D bounding box information are mapped as label data to the coordinate system of the perceptual space to obtain the mapped bounding box of the target object in the data to be labeled, including: Based on the mapping relationship between the visual camera coordinate system and the global coordinate system, the coordinate information and the three-dimensional bounding box information are mapped as label data to the visual camera coordinate system to obtain the mapped bounding box of the target object in the video image data. Based on the mapping relationship between the lidar coordinate system and the global coordinate system, the coordinate information and the three-dimensional bounding box information are mapped as label data to the lidar coordinate system to obtain the mapped bounding box of the target object in the lidar point cloud data.

8. A sensing data annotation device integrating V2X technology, characterized in that: The acquisition unit is used to acquire the data to be labeled collected by the vehicle-mounted sensing terminal when the connected vehicle is driving, as well as the V2X message data collected by the vehicle-mounted communication terminal. The data to be labeled includes video image data and radar point cloud data. The extraction unit is used to extract the coordinate information, three-dimensional bounding box information and semantic information of the target object from the V2X message data; The mapping unit is used to map the coordinate information and the three-dimensional bounding box information as label data to the coordinate system of the perceptual space to obtain the mapped bounding box of the target object in the data to be labeled. The annotation unit is used to perform geometric range annotation and attribute annotation on the target object based on the pixel region and point cloud space corresponding to the mapped bounding box and the semantic information.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images