Data processing method, computer equipment and computer program product
By displaying perceived instance information on the target management page and generating display images, combined with the true value annotation function, the problem of low efficiency in perceived abnormal data analysis in the prior art is solved, and a faster and more efficient analysis and labeling process is achieved.
Patent Information
- Application Number
- CN202411634165.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-14
- Publication Date
- 2025-03-07
AI Technical Summary
The prior art is less efficient when analyzing vehicle perceived abnormal data, and testers need to analyze each data in turn and mark the true value, which takes a long time.
By displaying instance information of the perceived instance on the target management page, and displaying a display image of the target perceived instance in the second display area in response to a selection operation, the image is generated based on the selected state of the data layer and the layer file. At the same time, testers are allowed to mark the target-aware instance truth value to generate road truth data.
The efficiency of perceived anomaly data analysis is improved, the problem of perceived instances is quickly analyzed by displaying images, and the labeling efficiency is improved by automated truth value annotation process.
Smart Images
Figure CN119576182A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of autonomous driving technology, and in particular to a data processing method, a computer device, and a computer program product. Background Art
[0002] The intelligent development of vehicles is getting faster and faster, and more and more large models are gradually being installed on vehicles. However, even so, actual vehicles still encounter many perception anomaly problems. When the number of users increases, more perception anomalies are reported by vehicles. Operation and maintenance personnel analyze the specific circumstances of perception anomalies through the perception anomaly data reported by users. However, the current method of analyzing perception anomaly data has the problem of low efficiency. Summary of the invention
[0003] The embodiments of the present disclosure at least provide a data processing method, a computer device, and a computer program product.
[0004] In a first aspect, an embodiment of the present disclosure provides a data processing method, including:
[0005] Displaying a target management page, and displaying instance information of at least one perception instance in a first display area of the target management page; the perception instance includes a layer file of at least one data layer generated according to the perception abnormality data reported by the vehicle;
[0006] In response to a selection operation of a target perception instance in the at least one perception instance, a display image corresponding to the target perception instance is displayed in a second display area of the target management page; wherein the display image is generated based on a data layer corresponding to the target perception instance and a selected state corresponding to each layer file;
[0007] In response to performing a truth value labeling operation on the target perception instance, road truth value data corresponding to the target perception instance is generated.
[0008] In a possible implementation, the data layer includes at least one of the following layers: a standard definition SD data layer, a vehicle driving trajectory layer, a visual perception layer, and a bound road layer.
[0009] In a possible implementation manner, the first display area further displays first selection controls corresponding to each of the perception instances;
[0010] Performing a selection operation on a target perception instance in the at least one perception instance, comprising: performing a selection operation on the target perception instance based on a first selection control corresponding to the target perception instance;
[0011] and / or,
[0012] The target management page also displays a target function control;
[0013] The method further comprises: in response to a trigger operation on the target function control, displaying a target operation page at the front end of the target management page; the target operation page comprises a second selection control;
[0014] Performing a selection operation on a target perception instance in the at least one perception instance includes: utilizing the second selection control to perform a selection operation on the target perception instance.
[0015] In a possible implementation manner, the performing a truth value labeling operation on the target perception instance includes:
[0016] Generate a true value layer corresponding to the target perception instance, and superimpose and display the true value layer and the display image corresponding to the target perception instance;
[0017] A truth value annotation operation for the target perception instance is received through the truth value layer.
[0018] In a possible implementation, the method further includes: in response to the target perception instance entering a labelable state, loading a satellite map layer, and superimposing and displaying the satellite map layer and the display image.
[0019] In a possible implementation manner, the method further includes: performing target recognition on the satellite map layer to obtain at least one candidate true value label;
[0020] Overlaying and displaying the candidate true value annotation and the display image;
[0021] In response to a selection operation on the candidate truth value label, determining the candidate truth value label as the road truth data;
[0022] In response to an adjustment operation on the candidate truth value label, the candidate truth value label is updated according to an operation result of the adjustment operation.
[0023] In a possible implementation, the method further includes: in response to performing a layer association operation on the target perception instance, controlling a display image corresponding to the target perception instance to enter a triggerable state;
[0024] After the display image enters the triggerable state, in response to triggering a target position point in the display image, determining an associated object of the target position point from each data layer corresponding to the target perception instance;
[0025] The associated object corresponding to the target location point is highlighted.
[0026] In a possible implementation, the method further includes: in response to a dynamic playback operation on the target perception instance, generating a target video for simulating the vehicle driving process based on each data layer corresponding to the target perception instance;
[0027] Play the target video.
[0028] In a second aspect, an optional implementation of the present disclosure further provides a computer device, a processor, and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is used to execute the machine-readable instructions stored in the memory, and when the machine-readable instructions are executed by the processor, the machine-readable instructions perform the steps in the above-mentioned first aspect, or any possible implementation of the first aspect.
[0029] In a third aspect, an optional implementation of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed, the steps of the above-mentioned first aspect, or any possible implementation of the first aspect are executed.
[0030] In a fourth aspect, an optional implementation of the present disclosure further provides a computer program product, which carries a program code, and the instructions included in the program code can be used to execute the steps of the data processing method as described in the first aspect or any one of the first aspect.
[0031] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of the present disclosure.
[0032] The embodiment of the present disclosure utilizes the first display area of the target management page to display the instance information of at least one perception instance, and in response to the selection operation of the target perception instance, displays the display image corresponding to the target perception instance in the second display area of the target management page, and the display image is generated based on the data layer corresponding to the target perception instance and the selected state corresponding to each layer file. This allows the tester to adjust the selected state of different layer files as needed to control the combination of different data layers into display images with different contents for display, so that the tester can quickly analyze the problems existing in the perception instance through the display image and improve the analysis efficiency of the perception abnormal data; at the same time, on this basis, the tester can perform annotating operations on the target perception instance to generate road true value data corresponding to the perception abnormal data, thereby improving the efficiency of true value annotation. In order to make the above-mentioned purposes, features and advantages of the present disclosure more obvious and easy to understand, the following is a preferred embodiment, and is described in detail in conjunction with the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.
[0034] Figure 1 A flowchart showing a data processing method provided by some embodiments of the present disclosure is shown;
[0035] Figure 2 One of the specific examples of the target management page provided by some embodiments of the present disclosure is shown;
[0036] Figure 3 The second specific example of the target operation page provided by some embodiments of the present disclosure is shown;
[0037] Figure 4 The third specific example of performing true value labeling on a target perceived instance provided by some embodiments of the present disclosure is shown;
[0038] Figure 5 A schematic diagram of a computer device provided by some embodiments of the present disclosure is shown. DETAILED DESCRIPTION
[0039] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the present disclosure claimed for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.
[0040] After research, it was found that when a vehicle has a perception anomaly, it will report the perception anomaly data to the cloud management platform; after obtaining the perception anomaly data, the tester will analyze the perception anomaly data. When analyzing the perception anomaly data, the tester needs to obtain the problem data from the perception anomaly data and mark the true value data corresponding to the problem data. However, when analyzing the current perception anomaly data, the tester needs to analyze each item in the perception anomaly data in turn, and then mark the true value on this basis, which makes the analysis and true value marking process take too much time, and the analysis and processing efficiency of the perception anomaly data is low.
[0041] Based on the above research, the present disclosure provides a data processing method and a method for processing perception instances to improve the efficiency of analyzing perception instances.
[0042] The defects existing in the above solutions are the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the present disclosure for the above problems below should be the contributions made by the inventor to the present disclosure during the disclosure process.
[0043] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.
[0044] The solutions described in this specification and in the examples, if they involve the processing of personal information, will be processed on the premise of having a legal basis (such as obtaining the consent of the subject of personal information, or being necessary for the performance of a contract, etc.), and will only be processed within the scope of regulations or agreements. If a user refuses to process personal information other than the necessary information for basic functions, it will not affect the user's use of basic functions.
[0045] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the type, scope of use, and usage scenarios of the personal information involved in the present disclosure should be informed to the user in an appropriate manner in accordance with relevant laws and regulations, and the user's authorization should be obtained. For example, a prompt message requesting authorization can be sent to the user through a pop-up window on the page, information push, etc., and the above information can be used after the user agrees.
[0046] To facilitate understanding of this embodiment, a data processing method disclosed in the embodiment of the present disclosure is first introduced in detail. The executor of the data processing method provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities. In some possible implementation methods, the data processing method can be implemented by a processor calling computer-readable instructions stored in a memory.
[0047] The data processing method provided by the embodiment of the present disclosure is described below.
[0048] See also Figure 1 FIG. 1 is a flowchart of a data processing method provided by an embodiment of the present disclosure, wherein the method includes steps S101 to S103, wherein:
[0049] S101: Display a target management page, and display instance information of at least one perception instance in a first display area of the target management page; the perception instance includes a layer file of at least one data layer generated according to perception abnormality data reported by a vehicle.
[0050] In a specific implementation, the target management page, for example, is an interactive page displayed when executing the data processing method provided by the embodiment of the present disclosure. The tester can use the target management page to control the specific perception instance (case) to be processed.
[0051] The target management page includes a first display area, in which instance information corresponding to at least one perception instance is displayed. The perception instance includes a layer file of at least one data layer generated according to the perception abnormality data reported by the vehicle. The instance information displayed in the target management page includes, for example, an identifier of the perception instance and identifiers of each layer file.
[0052] Specifically, the perception abnormality data reported by the vehicle is, for example, data related to the perception abnormality reported to the cloud management platform when a perception abnormality occurs in a perception device deployed on the vehicle. The perception device may include, for example, at least one of: a camera, a radar, an ultrasonic sensor, an inertial measurement unit, etc. In addition, the perception device may also be other sensor devices deployed on the vehicle, which is not limited in the embodiments of the present disclosure.
[0053] For different sensing devices, the types of sensing anomalies may also be different. For example, for cameras and radars, the sensing anomaly may include at least one of the following: missing sensing results for certain objects, such as not sensing certain obstacles or certain road facilities; there is a large error between the sensing results for certain objects and the actual situation, such as a large distance between the perceived position of a lane line and the actual position of the lane line, or a certain obstacle is mistakenly perceived as another obstacle. For an inertial measurement unit, the sensing anomaly may include, for example, an abnormality in the vehicle speed, position, heading, etc. determined by the inertial measurement unit, such as binding to the wrong road, or position jump, or the heading is inconsistent with the actual vehicle head direction, etc.
[0054] Different types of perception anomalies will result in different corresponding perception anomaly data. For example, for a case where the type of perception anomaly is a vehicle-bound road anomaly, such as a vehicle under an overpass and the navigation display is above the overpass, the perception anomaly data uploaded by the vehicle includes, for example: the trajectory data of the vehicle within a certain period of time, the visual perception data during that period of time (such as video data obtained through a camera, or radar detection data obtained through a radar, or related data obtained by processing video data and radar detection data to describe the environment near the vehicle, such as a constructed three-dimensional scene, etc.), Standard Definition (SD) map data, and the corresponding bound road data of the vehicle, etc.
[0055] In the case where the type of perception anomaly is lane line detection error, such as the lane line is not detected, or there is a large difference between the position of the detected lane line and the actual lane line position, the perception anomaly data uploaded by the vehicle includes, for example: the vehicle's trajectory data within a certain period of time, visual perception data within that period of time, lane line-related data generated by the vehicle based on visual perception data, SD data, etc.
[0056] In the case where the type of perception anomaly is an error in the detection of road facilities or obstacles, such as errors in the detected position or type of other vehicles, failure to detect nearby road facilities, etc., the perception anomaly data uploaded by the vehicle may include: visual perception data of the vehicle within a certain period of time, road facility or obstacle detection result data generated based on visual perception data, etc.
[0057] After the perceived abnormal data is uploaded to the cloud management platform, the cloud management platform will convert the perceived abnormal data into layer files corresponding to each data item, and each layer file will be displayed as a corresponding data layer on the display.
[0058] The data layer includes at least one of the following layers:
[0059] Standard definition SD data layer, vehicle trajectory layer, visual perception layer and bound road layer.
[0060] The above data layer may be generated by directly converting the perceived abnormal data into a specific data format.
[0061] There may be at least one data layer for each type. Taking the SD data layer as an example, it may include: road layer, intersection layer, etc. Taking the visual perception layer as an example, it may include: lane line visual perception layer, stop line visual perception layer, different types of obstacles visual perception layer, etc.; taking the driving trajectory layer as an example, it may include: original positioning trajectory layer, repositioning trajectory layer, the closest distance layer of the original positioning trajectory from the SD road, the closest distance layer of the repositioning trajectory from the SD road, etc.; wherein the repositioning trajectory is, for example, the positioning trajectory obtained by optimizing the original positioning trajectory.
[0062] In addition, map-related data, such as SD data, etc., may not be uploaded by the vehicle to the cloud management platform. The vehicle uploads the vehicle's positioning information to the cloud management platform. The cloud management platform determines the SD data that matches the positioning information based on the positioning information, and generates a corresponding layer file based on the determined SD data.
[0063] These layer files constitute a perception instance, and the corresponding instance information is displayed in the target management page.
[0064] like Figure 2 As shown, a specific example of a target management page is shown, in which the target management page includes a first display area s1, in which instance information of two perception instances is displayed, and the instance information of each perception instance includes the name of the perception instance and the name of the layer file. The two perception instances include: case1 and case2; wherein, taking case1 as an example, case1 corresponds to 5 layer files, namely: the relocated track layer rebind_track, the layer org_track_bind of the closest distance from the original positioning track to sdlink, the layer rebind_track_bind of the closest distance from the relocated track to sdlink, the road layer ll_link of SD data, and the intersection layer ll_junc of SD data. Among them, sdlink represents a road represented in SD form within a certain area where the vehicle is located. Case2 is similar to case1 and will not be repeated here.
[0065] In addition, there can be multiple perception instances in the target management page. In the case of limited first display area, in many cases it is impossible to display all perception instances through the display interface. Therefore, you can also set
[0066] Following the above S101, the interactive control method provided by the embodiment of the present disclosure further includes:
[0067] S102: In response to a selection operation on a target perception instance among the at least one perception instance, a display image corresponding to the target perception instance is displayed in a second display area of the target management page; wherein the display image is generated based on a data layer corresponding to the target perception instance and a selected state corresponding to each layer file.
[0068] In a specific implementation, the selection operation of the target perception instance may include at least one of the following methods a1 and a2:
[0069] a1: the first display area also displays first selection controls corresponding to each of the perception instances;
[0070] Performing a selection operation on a target perception instance in the at least one perception instance includes: performing a selection operation on the target perception instance based on a first selection control corresponding to the target perception instance.
[0071] Specifically, the target management page may display a first selection control corresponding to each perception instance. When a perception instance is selected using the first selection control, the selected perception instance may be used as the target perception instance.
[0072] In addition, in another embodiment of the present disclosure, selection controls corresponding to each layer file corresponding to the target perception instance can also be displayed in the target management page; after a certain perception instance is determined as a target perception instance, the selection state corresponding to each layer file in the target perception instance can be adjusted through the selection controls corresponding to each layer file to determine the data layer to be displayed, and then the data layer to be displayed is superimposed in a certain superposition manner to obtain a display image, and the display image is displayed in the second display area of the target management page. Here, the display image can be regarded as an image including multiple channels, each channel corresponding to a data layer. The selection state corresponding to any layer file is used to indicate whether the layer file is selected or not selected.
[0073] a2: The target management page also displays a target function control;
[0074] The method further comprises: in response to a trigger operation on the target function control, displaying a target operation page at the front end of the target management page; the target operation page comprises a second selection control;
[0075] Performing a selection operation on a target perception instance in the at least one perception instance includes: utilizing the second selection control to perform a selection operation on the target perception instance.
[0076] In a specific implementation, the target function control is used to call up a target operation page for specifically controlling the perception instance. The target operation page may include at least one control control for the target perception instance, such as the second selection control mentioned above, through which the target perception instance can be selected from multiple perception instances.
[0077] like Figure 3 As shown, the embodiment of the present disclosure shows a specific example of a target operation page, which includes: two parts: a second selection control s3 and s4, wherein the second selection control s3 is used to filter a list of perception instances of a block to be processed from multiple perception instance lists; the list of perception instances to be processed includes multiple perception instances to be processed; the second selection control s4 is used to determine a target perception instance from the multiple perception instances included in the list of perception instances to be processed.
[0078] In addition, in another embodiment of the present disclosure, when the first selection control and the target function control are displayed in the target management page, if a target perception instance A has been selected using the first selection control before the target operation page is opened using the target function control, the target perception instance A will not change after the target operation page is opened using the target function control. At this time, the relevant controls included in the target operation page can be used to control the perception instance A accordingly; in addition, the target perception instance B can also be re-determined through the second selection control. In this case, there may be any of the following possibilities: the first possibility is that the target perception instance B and the perception instance A are different perception instances. After the target perception instance B is re-determined, the selection state of the perception instance A is adjusted from selected to unselected, and the state of the first selection control is changed, so that the first selection control corresponding to the perception instance A is changed from the selected state to the unselected state, and the first selection control corresponding to the perception instance B is changed from the unselected state to the selected state. The second possibility is that there are multiple target perception instances A; target perception instance B is one of target perception instances A; at this time, in target perception instance A, the first selection controls corresponding to other target perception instances except target perception instance B are changed from the selected state to the unselected state; the state of the first selection control corresponding to target perception instance B does not change.
[0079] After the selection operation of the target perception instance is triggered by any of the above methods, a display image corresponding to the target perception instance will be displayed in the second display area of the target management page; wherein, the display image is generated based on the data layer corresponding to the target perception instance and the selected status corresponding to each layer file.
[0080] like Figure 2In the example shown, the first selection controls s10 corresponding to case1 and case2 are both selected as target perception instances; at the same time, the selection state of the selection control s11 corresponding to each layer file of case1 indicates that all layer files corresponding to case1 are selected, while the selection state of the selection control s13 corresponding to each layer file of case2 indicates that all layer files corresponding to case2 are not selected. In this case, the second display area s2 of the target management page displays a display image obtained by superimposing all layer files corresponding to case1.
[0081] In addition, in another embodiment of the present disclosure, after displaying the display image corresponding to the target perception instance in the second display area, the method may further include:
[0082] In response to an operation of changing the selected state of any layer file corresponding to the target perception instance, based on the operation result of the change operation, the display image corresponding to the target perception instance displayed in the second display area is updated.
[0083] Here, the operation of changing the selection status of any layer file can, for example, be to change the selection status of any layer file from selected to unselected. At this time, when the display image is updated, the data layer corresponding to the layer file will be deleted from the display image to obtain a new display image, and the new display image will be displayed in the second display area.
[0084] The operation of changing the selection status of any layer file can also be to change the selection status of any layer file from unselected to selected. At this time, when the display image is updated, the data layer corresponding to the layer file will be superimposed on the display image to obtain a new display image, and the new display image will be displayed in the second display area.
[0085] In this way, the user can adjust which perception instance is used as the target perception instance by adjusting the selection status corresponding to different perception instances, and can display the display images under different data layer combinations in the second display area by adjusting the selection status of each layer file corresponding to the target perception instance, thereby facilitating analysis and comparison through this combination of different data layers to determine the specific abnormal situation of the perception instance.
[0086] In the embodiment of the present disclosure, there may be at least one target perception instance determined. When there are multiple target perception instances, the tester may generate a display image including data layers of different target perception instances by controlling the selected state of the layer files of different target perception instances, so as to realize the combination of multiple target perception instances and analyze the common abnormalities existing in different target perception instances.
[0087] For example, for a certain area, different vehicles A and B both experienced perception anomalies when passing through the area, and reported the perception anomaly data to the cloud management platform. The cloud management platform can display the instance information of the perception instances corresponding to the two vehicles in the target management page. The tester can select both perception instances as target perception instances and display a display image containing the data layers corresponding to the two perception instances in the second display area.
[0088] Following the above S102, the data processing method provided by the embodiment of the present disclosure further includes:
[0089] S103: In response to performing a truth value labeling operation on the target perception instance, generating road truth value data corresponding to the target perception instance.
[0090] In a specific implementation, the truth value labeling operation is performed on the target perception instance, specifically including:
[0091] Generate a true value layer corresponding to the target perception instance, and superimpose and display the true value layer and the display image corresponding to the target perception instance;
[0092] A truth value annotation operation for the target perception instance is received through the truth value layer.
[0093] In a specific implementation, when generating a truth value layer corresponding to a target perception instance, for example, a corresponding truth value layer file can be generated, and the relevant information of the truth value layer file can be loaded into a position corresponding to the target perception instance displayed in the first display area, and at the same time, the selected state corresponding to the truth value layer file is adjusted to selected, thereby realizing the superposition display of the display image corresponding to the truth value layer and the target perception instance.
[0094] Through the truth value layer, the truth value annotation operation of the tester is received, and the road truth value data is obtained according to the truth value annotation operation, and then the road truth value data is displayed in the truth value layer.
[0095] In another embodiment of the present disclosure, before generating a truth value layer corresponding to a target perception instance, for example, the target perception instance can be controlled to enter an annotable state first. Exemplarily, for example, the target perception instance can be controlled to enter an annotable state by a truth value annotation control set in a target management page or a target operation page. After controlling the target perception instance to enter an annotable state, the user can directly perform a truth value annotation operation on the target perception instance in the second display area; in addition, in response to the target perception instance entering an annotable state, a target annotation page can be displayed at the front end of the target management page, and the display image and the truth value layer of the target perception instance can be superimposed and displayed on the target annotation page, so that the tester can perform an annotation operation on the target perception instance in the target annotation page.
[0096] Specifically, the tester can control the target perception instance to enter the labeling state by clicking the truth value labeling control, and control the target perception instance to exit the labeling state by clicking the truth value labeling control again. After the target perception instance enters the labeling state, the tester can label the target perception instance by performing the truth value labeling operation in the second display area.
[0097] In addition, after the target perception instance enters the annotable state, the first selection control for selecting different perception instances can be set to an invalid state that cannot be triggered. In this state, the tester can no longer adjust other non-target perception instances to target perception instances, or cannot adjust the target perception instance to a non-target perception instance. In addition, in this state, the selection control for selecting each layer file corresponding to the target perception instance is still in a triggerable valid state. The tester can adjust the specific data layer included in the display image according to the annotation needs of the target perception instance, which is convenient for the tester's annotation operation.
[0098] After exiting the labelable state of the target perception instance, the road truth data corresponding to the perception anomaly data is saved to facilitate the tester to export the road truth data, or to label the same target perception instance again based on the previous labeling.
[0099] In addition, if there are multiple target perception instances, the display image displayed in the second display area may include data layers corresponding to multiple target perception instances. When the true value layer and the display image corresponding to the target perception instance are superimposed and displayed, the display image displayed in the second display area may not be adjusted. In addition, the data layers corresponding to other target perception instances in the display image except the target perception instance may be deleted from the target display layer. For example, the target perception instances include case_1 and case_2. The display image P1 includes the data layers t1, t2 and t3 of case_1, and the data layers t4, t5 and t6 of case_2. After case1 is used as the target perception instance, the true value layer corresponding to case_1 can be directly superimposed with the display image P1, or P4, P5 and P6 can be deleted from the display image to obtain the display image P2 corresponding to case_1, and the true value layer and the display image P2 are superimposed.
[0100] When receiving a truth value annotation operation for a target truth value instance through the truth value layer, the truth value annotation operation includes, for example, a gesture operation in the second display area; by detecting the user's gesture operation on the second area, and determining specific road truth value data according to the operation track of the gesture operation. Exemplarily, the gesture operation may be, for example, an operation of adjusting the position of an object in the display image, a marking operation, an operation of adding description content, etc. The specific method of the annotation operation may be set according to actual needs, and the embodiments of the present disclosure do not limit this.
[0101] Specifically, through gesture operation, for example, any object in the display image is selected, and the object can be an object included in any data layer, such as a track point in the track layer, an obstacle, lane line, stop line, etc. in the visual perception layer. After selecting an object in the display image, another gesture operation can be performed on the object, such as adding a label, adjusting the position, etc. The gesture operation on the object will be converted into the corresponding road truth data, which is carried in the truth layer and saved in the form of a layer.
[0102] Here, adjusting the position of an object will not change the position of the object in the corresponding data layer, but will generate a copy object corresponding to the original object and add the copy object to the true value layer according to the adjusted position.
[0103] In addition, gesture operations can also be performed directly on the truth layer. For example, a gesture operation can be used to add an object to the truth layer, such as adding a lane line or adding the correct driving path of a vehicle.
[0104] In another embodiment of the present disclosure, in response to the target perception instance entering a labelable state, a satellite map layer may be loaded, and the satellite map layer and the display image may be superimposed and displayed.
[0105] The satellite map layer usually includes a map obtained by using a satellite to take real-life photos of a certain area. Various objects in the area can be more intuitively reflected through the satellite map layer; further, in the embodiment of the present disclosure, the satellite map layer and the display image are superimposed and displayed, and by comparing the satellite map layer with each data layer, it can be more intuitive to reflect whether there are abnormalities in each data layer.
[0106] In order to further improve the efficiency of truth value annotation, in another embodiment of the present disclosure, the following may also be included:
[0107] Performing target recognition on the satellite map layer to obtain at least one candidate true value label;
[0108] Overlaying and displaying the candidate true value annotation and the display image;
[0109] In response to a selection operation on the candidate truth value label, determining the candidate truth value label as the road truth data;
[0110] In response to an adjustment operation on the candidate truth value label, the candidate truth value label is updated according to an operation result of the adjustment operation.
[0111] In a specific implementation, when performing target recognition on a satellite map layer, for example, a pre-trained target recognition model may be used to perform target detection processing on the satellite map layer, and at least one candidate true value annotation may be obtained based on the result of the target detection processing. The candidate true value annotation may include, for example, the category of the target object, the position of the target object in the satellite map layer, etc. The target object may include, for example, lane lines, roads, stop lines, and other objects that can be detected by the visual perception of the target vehicle.
[0112] The alternative true value annotation can, for example, form a separate layer and be displayed superimposed on the display image; in addition, the alternative true value annotation can, for example, be displayed in the true value layer; when the tester selects an alternative true value annotation, the alternative true value annotation is determined as the road true value data and stored in the true value layer file corresponding to the true value layer; if an alternative true value annotation is not selected, it can be deleted after the true value annotation is completed.
[0113] In addition, the tester can also adjust the candidate truth value annotation, for example, adjust the content of the candidate truth value annotation, such as adjusting the category of the target object, the position of the target object, etc. In response to the adjustment operation on the candidate truth value annotation, the candidate truth value annotation is updated according to the operation result.
[0114] like Figure 4 As shown in FIG. 1 , a specific example of performing true value annotation on a target-aware instance is provided. In this example, Figure 4 As shown in a, in the first display area, in addition to the instance information corresponding to the target perception instance, the layer information s13 corresponding to the true value layer file is also displayed. In this example, the layer information of the true value layer is "hmm_data_vec", and the layer information s14 corresponding to the satellite map layer is displayed. In this example, the layer information of the satellite map layer is "gaode_site"; Figure 4 As shown in b, in the second display area, the satellite map layer, the display image corresponding to the target perception instance, and the candidate true value annotation obtained by target recognition of the satellite map layer are displayed ( Figure 4 yellow and green lines in the middle).
[0115] In another embodiment of the present disclosure, in order to further facilitate the user's annotation operation on the target perception instance, a layer association control can also be displayed in the target operation page.
[0116] The data processing method provided by the embodiment of the present disclosure may further include: in response to performing a layer association operation on the target perception instance, controlling the display image corresponding to the target perception instance to enter a triggerable state;
[0117] After the display image enters the triggerable state, in response to triggering a target position point in the display image, determining an associated object of the target position point from each data layer corresponding to the target perception instance;
[0118] The associated object corresponding to the target location point is highlighted.
[0119] In a specific implementation, the layer association operation can be implemented, for example, by setting a layer association control in a target management page or a target operation page. After a certain perception instance is determined as a target perception instance, the layer association control is set to a trigger-valid state; at this time, the tester can trigger the layer association operation through the layer association control.
[0120] When the layer association operation is triggered, the display image corresponding to the target perception instance enters a triggerable state; in this triggerable state, at least some of the position points in the display image can be triggered; these position points, for example, include the position points of each object included in the display image; when any of these position points is triggered, the associated area of the target position point can be determined from each data layer corresponding to the target perception instance, and the associated area can be highlighted.
[0121] When determining the associated area, for example, any of the following methods may be used:
[0122] Method 1: According to the triggered position point, a first associated object corresponding to the position point can be determined from the target data layer, and according to the position of the first associated object in the target data layer, a second associated object whose distance to the first associated object is within a preset distance range can be determined from other data layers;
[0123] The first associated object and the second associated object are used as associated objects of the target location point.
[0124] Here, the target data layer can be, for example, a pre-selected data layer; for example, before performing a layer association operation on the target perception instance, in response to receiving a selection operation on any data layer, the data layer is used as the target data layer; in response to performing a layer association operation on the target perception instance, when controlling the display image corresponding to the target perception instance to enter a triggerable state, for example, only the objects in the target data layer can be controlled to enter a triggerable state.
[0125] In addition, after the display image corresponding to the target perception instance enters a triggerable state, the target data layer can also respond to a selection operation on any data layer corresponding to the target perception instance and determine the any data layer as the target data layer, so that the object in the target data layer enters a triggerable state.
[0126] However, the objects in non-target data layers will not enter the triggerable state.
[0127] Exemplarily, when the data layer includes the following layers: standard definition SD data layer, vehicle driving trajectory layer, and bound road layer, the objects in the SD data layer may include, for example, lane lines, stop lines, etc.; the objects in the vehicle driving trajectory layer may include, for example, multiple trajectory points of the vehicle; the bound road layer may include, for example, the road on which the vehicle is traveling; when the position point corresponding to a certain trajectory point is triggered, the trajectory point is determined according to the position point, and the lane lines and stop lines within a preset distance range from the trajectory point are determined from the vehicle driving trajectory layer, and the road sections within a preset distance range from the trajectory point are determined from the roads included in the road binding data layer. The above-determined trajectory points, lane lines, stop lines, and road sections are highlighted as highlighted associated objects in the display image. In this way, the tester can more clearly identify whether these interrelated objects have abnormal conditions by highlighting certain interrelated objects. For example, in the event of a bound road anomaly, assuming that there are two associated road sections, the track point should be located on one of them, but the result obtained from the highlighted associated object is that the track point is located on the other one. After highlighting the associated object, the anomaly can be more clearly reflected in the displayed image.
[0128] Method 2: Each data layer corresponding to the perception instance usually includes data corresponding to multiple time points. For example, the bound road layer includes bound roads corresponding to multiple time points. When a position point in the display image is triggered, the first associated object corresponding to the position point can be determined from the target data layer through the position point, such as a bound road, a trajectory point, a lane line, etc. After determining the first associated object corresponding to the penalized position point, determine the second associated object whose timestamp is consistent with the timestamp of the first associated object from other data layers according to the timestamp corresponding to the first associated object, and determine the first associated object and the second associated object as the associated objects of the target position point.
[0129] For example, when the target location point is triggered, the first associated object determined based on the target location point is a certain track point; the timestamp corresponding to the track point is T, and then the second associated object with the timestamp of T is determined from other data layers, such as the bound road, the lane line on the road detected by visual perception at the time T, etc.
[0130] In addition, when determining associated objects, a combination of timestamp (method two) and location (method one above) can be used, and associated objects determined by different methods can be highlighted in different display methods, so that testers can better determine the location of perceived anomalies based on associated objects determined by different methods.
[0131] Taking the bound road anomaly as an example, when there is no bound road anomaly, there will not be too large a difference between the associated objects determined by timestamp and location respectively; if there is a large difference, it indicates that a bound road anomaly has occurred.
[0132] In another embodiment of the present disclosure, the data processing method provided by the embodiment of the present disclosure further includes:
[0133] In response to a dynamic playback operation on the target perception instance, a target video for simulating the vehicle driving process is generated based on each data layer corresponding to the target perception instance; and the target video is played.
[0134] In a specific implementation, the dynamic playback operation of the target perception instance can be achieved, for example, by setting a dynamic playback control in a target management page or a target operation page; the tester can trigger the dynamic playback operation of the target perception instance through the dynamic playback control.
[0135] When generating a target video for simulating a vehicle driving process based on each data layer corresponding to the target perception instance, the following method can be used:
[0136] Determine a target data layer from multiple data layers of the target perception instance; the information carried by the target data layer is a data sequence with a time sequence;
[0137] For each data in the data sequence, determine the associated object associated with the data from each data layer;
[0138] Generate a video frame corresponding to each data according to each data and an associated object associated with each data;
[0139] The order of each data in the data sequence is combined by combining the video frames corresponding to each data in the data sequence to form a target video.
[0140] Here, the target data layer may include, for example, a vehicle trajectory layer, which carries the vehicle trajectory, which is composed of multiple trajectory points, and the multiple trajectory points are a data sequence with a time order. The method for determining the associated object is similar to the method for determining the associated object in the above embodiment, and will not be repeated here.
[0141] like Figure 3 As shown, the target operation page provided by the embodiment of the present disclosure includes, in addition to the second selection control, a layer association control s5, a dynamic playback control s6, an exception confirmation control s7, a truth value labeling control s8, and a truth value labeling result export control s9.
[0142] Among them, identify, that is, the layer association control s5, after selecting this function, click a position point in the track layer to highlight the layer data associated with the position point. For example, for a bad case of vehicle binding error type, click a track point to highlight the road section bound to the track point in the vehicle binding data layer and the corresponding frame data in the perception data layer corresponding to this track point, which is convenient for visualizing all related data of this frame and problem analysis.
[0143] Dynamic playback control s6: used to trigger the dynamic playback operation of the target perception instance. After selecting this function, different data layers are linked to generate a target video for simulating the driving process of the vehicle, and the target video is automatically played. Taking the perception instance of the wrong type of road binding as an example, when playing the target video, if the road binding starts and the result is bound to a road very close to the vehicle, a jump will occur in the middle and the road will be bound to a road far away. This situation is obviously a problem point, and the key to the problem lies in the location of the jump.
[0144] The abnormality confirmation control s7, when confirming that the target perception instance is a perception instance with an abnormality, can, for example, determine the target perception instance as an abnormal instance according to the abnormality confirmation control s7. In a possible implementation, after the target perception instance is confirmed as an abnormal instance by the abnormality confirmation control s7, the state of the truth value labeling control s8 and the truth value labeling result export control s9 changes from a non-triggerable state to a triggerable state.
[0145] Truth value labeling control s8. After clicking the truth value labeling control s8, a new truth value layer will be automatically created under the perception information of the target perception instance displayed in the first display area, and the satellite map layer will be automatically loaded. The truth value layer, satellite map layer and display image of the target perception instance are superimposed and displayed to assist the tester in labeling the truth value and facilitate the subsequent regression verification of the problem. In order to improve the efficiency of labeling, the trained model is used in the background to extract and identify multiple types of ground signs on the satellite layer where the target perception instance is located, such as lane lines, zebra crossings, stop lines, ground drivable areas, etc. The tester can directly click on the recognition result as the truth value, or adjust and optimize the geometric shape of the recognized result as the truth value.
[0146] like Figure 4 As shown, Figure 4 Where a is the instance information of the target perception instance. Figure 4 In the middle, b is the visualization interface of the layer. You can see that the map is a satellite map with SD data layer and trajectory layer superimposed on it. After the annotator clicks the start annotation button, the background will automatically call the trained model satellite layer to extract and identify multiple types of ground signs. The recognition results will be displayed on the satellite layer. Then, the annotator can choose to draw points, lines, and surfaces on the true value layer, or click on the model recognition results for fine-tuning. After confirming that there are no errors, click Save to improve the efficiency of annotation.
[0147] After marking the true value, click "Export Results" in the true value labeling result export control s9 to convert the true value layer into csv format data, and automatically upload the target perception instance and the true value layer to the verification set in the cloud. Subsequently, you can directly pull batches of verification sets from the cloud for evaluation and compare them with the true value to determine whether the verification is passed.
[0148] The disclosed embodiment utilizes the first display area of the target management page to display the instance information of at least one perception instance, and in response to the selection operation of the target perception instance, displays the display image corresponding to the target perception instance in the second display area of the target management page, and the display image is generated based on the data layer corresponding to the target perception instance and the selected state corresponding to each layer file. This allows the tester to adjust the selected state of different layer files as needed to control the combination of different data layers into display images with different contents for display, so that the tester can quickly analyze the problems existing in the perception instance through the display image, and improve the analysis efficiency of the perception abnormality data; at the same time, on this basis, the tester can perform annotating operations on the target perception instance to generate road truth data corresponding to the perception abnormality data, thereby improving the efficiency of truth value annotation.
[0149] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.
[0150] The present disclosure also provides a computer device, such as Figure 5 FIG. 1 is a schematic diagram of a computer device structure provided in an embodiment of the present disclosure, including:
[0151] A processor 51 and a memory 52; the memory 52 stores machine-readable instructions executable by the processor 51, and the processor 51 is used to execute the machine-readable instructions stored in the memory 52. When the machine-readable instructions are executed by the processor 51, the processor 51 performs the following steps:
[0152] Displaying a target management page, and displaying instance information of at least one perception instance in a first display area of the target management page; the perception instance includes a layer file of at least one data layer generated according to the perception abnormality data reported by the vehicle;
[0153] In response to a selection operation of a target perception instance in the at least one perception instance, a display image corresponding to the target perception instance is displayed in a second display area of the target management page; wherein the display image is generated based on a data layer corresponding to the target perception instance and a selected state corresponding to each layer file;
[0154] In response to performing a truth value labeling operation on the target perception instance, road truth value data corresponding to the target perception instance is generated.
[0155] The above-mentioned memory 52 includes an internal memory 521 and an external memory 522; the internal memory 521 here is also called an internal memory, which is used to temporarily store the calculation data in the processor 51, as well as the data exchanged with the external memory 522 such as a hard disk. The processor 51 exchanges data with the external memory 522 through the internal memory 521.
[0156] The specific execution process of the above instructions can refer to the steps of the data processing block method described in the embodiment of the present disclosure, which will not be repeated here.
[0157] The present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the data processing method described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.
[0158] The embodiments of the present disclosure also provide a computer program product, which carries a program code. The instructions included in the program code can be used to execute the steps of the data processing method described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.
[0159] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).
[0160] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.
[0161] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0162] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0163] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0164] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the aforementioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the aforementioned embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be based on the protection scope of the claims.
Claims
1. A data processing method, characterized in that: include: Displaying a target management page, and displaying instance information of at least one perception instance in a first display area of the target management page; The perception instance includes a layer file of at least one data layer generated according to the perception abnormality data reported by the vehicle; In response to a selection operation of a target perception instance in the at least one perception instance, a display image corresponding to the target perception instance is displayed in a second display area of the target management page; wherein the display image is generated based on a data layer corresponding to the target perception instance and a selected state corresponding to each layer file; In response to performing a truth value labeling operation on the target perception instance, road truth value data corresponding to the target perception instance is generated.
2. The method according to claim 1, characterized in that: The data layer includes at least one of the following layers: a standard definition SD data layer, a vehicle driving trajectory layer, a visual perception layer, and a bound road layer.
3. The method according to claim 1 or 2, characterized in that: The first display area also displays first selection controls corresponding to each of the perception instances; Performing a selection operation on a target perception instance in the at least one perception instance, comprising: performing a selection operation on the target perception instance based on a first selection control corresponding to the target perception instance; and / or, The target management page also displays a target function control; The method further comprises: in response to a trigger operation on the target function control, displaying a target operation page at the front end of the target management page; the target operation page comprises a second selection control; Performing a selection operation on a target perception instance in the at least one perception instance includes: utilizing the second selection control to perform a selection operation on the target perception instance.
4. The method according to any one of claims 1 to 3, characterized in that: The performing a true value labeling operation on the target perception instance includes: Generate a true value layer corresponding to the target perception instance, and superimpose and display the true value layer and the display image corresponding to the target perception instance; A truth value annotation operation for the target perception instance is received through the truth value layer.
5. The method according to claim 4, characterized in that The method further includes: in response to the target perception instance entering a labelable state, loading a satellite map layer, and superimposing and displaying the satellite map layer and the display image.
6. The method according to claim 5, characterized in that The method further comprises: performing target recognition on the satellite map layer to obtain at least one candidate true value label; Overlaying and displaying the candidate true value annotation and the display image; In response to a selection operation on the candidate truth value label, determining the candidate truth value label as the road truth data; In response to an adjustment operation on the candidate truth value label, the candidate truth value label is updated according to an operation result of the adjustment operation.
7. The method according to any one of claims 1 to 6, characterized in that: The method further includes: in response to performing a layer association operation on the target perception instance, controlling a display image corresponding to the target perception instance to enter a triggerable state; After the display image enters the triggerable state, in response to triggering a target location point in the display image, determining an associated object of the target location point from each data layer corresponding to the target perception instance; The associated object corresponding to the target location point is highlighted.
8. The method according to any one of claims 1 to 7, characterized in that: The method further includes: in response to a dynamic playback operation on the target perception instance, generating a target video for simulating the vehicle driving process based on each data layer corresponding to the target perception instance; Play the target video.
9. A computer device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is used to execute the machine-readable instructions stored in the memory. When the machine-readable instructions are executed by the processor, the processor executes the steps of the data processing method according to any one of claims 1 to 8.
10. A computer program product, characterized in that The computer program product carries a program code, and the program code includes instructions that can be used to execute the steps of the data processing method according to any one of claims 1 to 8.