Data processing method and device, equipment, storage medium and product

Abstract

The application discloses a data processing method, device and equipment, a storage medium and a product, relates to the technical field of automatic driving, and the data processing method comprises the following steps: acquiring a data segment associated with a driving scene; if it is determined that an abnormal frame in which an object in the data segment does not exist an associated lane, it is judged whether the driving scene where the abnormal frame is located is a target driving scene; if the driving scene is the target driving scene, based on map information in the abnormal frame, a data frame used for model training is determined, wherein the object in the target driving scene corresponding frame does not need to be associated with a lane. The application can avoid the problem that in some special scenes, part of the map is missing, the effective lane associated with the ego vehicle cannot be determined, and the diversity of training data is insufficient.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a data processing method and device, equipment, storage medium and product. BACKGROUND

[0002] In the training of a model, in order to ensure the strong generalization ability of the model, the training data needs to be diverse and varied, but the widely sourced data, while improving the generalization ability of the model, will also introduce various abnormalities. The introduction of abnormal data into model training can cause model training disaster, or the model learns abnormal information, so that the model performs abnormally in some scenarios. Therefore, abnormal information needs to be filtered before model training.

[0003] In related technologies, in the screening process of data, a single index is usually used for data abnormality judgment, but in some special scenarios, although the single index of some data is abnormal, the data can still be used for model training. If only a single index is used for data abnormality judgment, the model will miss the training data in special scenarios, resulting in a lack of diversity of training data. SUMMARY

[0004] The main purpose of the present application is to provide a data processing method, device, equipment, storage medium and product, which aims to solve the technical problem that using only a single index for data abnormality judgment will cause the model to miss training data in special scenarios, resulting in a lack of diversity of training data.

[0005] To achieve the above purpose, the present application provides a data processing method, which comprises:

[0006] obtaining a data segment associated with a driving scene;

[0007] if it is determined that the data segment has an abnormal frame in which an object does not exist in an associated lane, determining whether the driving scene in which the abnormal frame is located is a target driving scene;

[0008] if the driving scene is the target driving scene, determining a data frame for model training based on map information in the abnormal frame, wherein the object in the frame corresponding to the target driving scene does not need to be associated with a lane.

[0009] In an embodiment, the step of determining whether the driving scene in which the abnormal frame is located is a target driving scene if it is determined that the data segment has an abnormal frame in which an object does not exist in an associated lane comprises:

[0010] if it is determined that the data segment has an abnormal frame in which an object does not exist in an associated lane, obtaining map information in the abnormal frame;

[0011] determine whether a driving scene where the abnormal frame is located is a target driving scene based on the map information.

[0012] In an embodiment, if the driving scene is the target driving scene, the step of determining the data frame for model training based on the map information in the abnormal frame comprises:

[0013] determining whether the map information in the abnormal frame is valid if the driving scene is the target driving scene;

[0014] if the map information is valid, regarding the abnormal frame and the data frame with the associated lane in the data segment as the data frame for model training.

[0015] In an embodiment, if the map information is valid, the step of regarding the abnormal frame and the data frame with the associated lane in the data segment as the data frame for model training comprises:

[0016] traversing each frame in the data segment, and regarding the traversed frame as a current frame;

[0017] determining the number of lane center lines based on the map information, and determining whether there is a target frame in the data segment after a preset number of frames from the current frame if the number of lane center lines is greater than or equal to a preset center line number threshold;

[0018] if the target frame exists, determining an angle change value between a first heading angle of the ego vehicle in the target frame and a second heading angle of the ego vehicle in the current frame, and normalizing the angle change value to obtain a normalized angle change value;

[0019] if the normalized angle change value is within a preset change value threshold range, traversing all data frames between the current frame and the target frame to determine whether there is a target obstacle in the all data frames;

[0020] if the target obstacle exists, determining that the map information in the abnormal frame is valid, and regarding the abnormal frame and the data frame with the associated lane as the data frame for model training.

[0021] In an embodiment, the step of determining whether a driving scene where the abnormal frame is located is a target driving scene based on the map information if it is determined that the abnormal frame in the data segment has no object with the associated lane comprises:

[0022] calculating lane information associated with the object in the current frame;

[0023] screening the object in the current frame to obtain a target object of a preset type, wherein the target object includes the ego vehicle.

[0024] determine whether the target object has an associated lane based on the lane information;

[0025] If the target object does not have an associated lane, determine the current frame as an abnormal frame in which no object has an associated lane in the data segment.

[0026] In an embodiment, the step of calculating lane information associated with an object in the current frame comprises:

[0027] obtaining map information in the current frame, and extracting a lane center line from the map information in the current frame;

[0028] calculating a minimum projection distance of the object in the current frame to each lane center line;

[0029] determining an associated lane of the object based on the minimum projection distance;

[0030] determining lane information based on the minimum projection distance and the associated lane.

[0031] In an embodiment, the step of determining whether the target object has an associated lane based on the lane information comprises:

[0032] If the minimum projection distance of the target object of the preset type is equal to a preset first projection threshold or the minimum projection distance is greater than a preset second projection threshold, the target object does not have an associated lane.

[0033] In addition, to achieve the above object, the present application further provides a data processing device, which comprises:

[0034] an acquisition module configured to acquire a data segment associated with a driving scene;

[0035] a determination module configured to determine whether a driving scene in which an abnormal frame in which no object has an associated lane is located is a target driving scene;

[0036] a determination module configured to determine a data frame for model training based on map information in the abnormal frame if the driving scene is the target driving scene, wherein the object in the target driving scene does not need to have an associated lane.

[0037] In addition, to achieve the above object, the present application further provides a data processing device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the data processing method as described above.

[0038] In addition, to achieve the above object, the present application also provides a storage medium, which is a computer readable storage medium, and a computer program is stored on the storage medium, and the computer program is executed by a processor to implement the steps of the data processing method.

[0039] In addition, to achieve the above object, the present application also provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of the data processing method.

[0040] The one or more technical solutions provided by the present application have at least the following technical effects:

[0041] In the related art, in the process of automatic driving vehicle trajectory planning, it is necessary to ensure that the vehicle drives on a certain lane. If there is no associated lane during driving, the data associated with the current driving time is determined as an anomaly. However, in some special scenarios, part of the map is missing, and the effective lane associated with the vehicle cannot be determined, resulting in lack of diversity of training data. Compared with the prior art, the present application acquires a data segment associated with a driving scene, determines an abnormal frame in which an associated lane does not exist in the data segment, judges whether the driving scene in which the abnormal frame is located is a target driving scene, and if the driving scene is the target driving scene, determines a data frame for model training based on map information in the abnormal frame. It can be understood that, after acquiring the data segment associated with the driving scene and determining the abnormal frame in which the associated lane does not exist in the data segment, the present application judges the driving scene in which the abnormal frame is located. If the driving scene is the target driving scene in which the object in the abnormal frame does not need to be associated with the lane, the data frame for model training is determined based on the map information in the abnormal frame. This can avoid the problem that using a single indicator to judge data anomalies may cause the model to miss training data in special scenarios, resulting in lack of diversity of training data. BRIEF DESCRIPTION OF DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and serve to explain the principles of the present application together with the specification.

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0044] Figure 1A flowchart provided by the data processing method embodiment one of the present application;

[0045] Figure 2 A brief flowchart of the data processing method provided by the embodiment one of the present application;

[0046] Figure 3 A flowchart provided by the data processing method embodiment two of the present application;

[0047] Figure 4 A module structure diagram of the data processing device of the embodiment of the present application;

[0048] Figure 5 A device structure diagram of the hardware running environment involved in the data processing method in the embodiment of the present application.

[0049] The object implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0050] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application, and are not used to limit the present application.

[0051] In order to better understand the technical solutions of the present application, the following will be described in detail in combination with the drawings and specific embodiments of the specification.

[0052] The main solution of the embodiment of the present application is: obtaining a data segment associated with a driving scene; if it is determined that there is no abnormal frame of associated lane in the object in the data segment, it is judged whether the driving scene where the abnormal frame is located is a target driving scene; if the driving scene is the target driving scene, based on the map information in the abnormal frame, a data frame for model training is determined, wherein the object in the frame corresponding to the target driving scene does not need to be associated with a lane.

[0053] In the related art, in the data screening process, a single index is usually used for data anomaly judgment, but in some special scenes, although the single index of some data is abnormal, the data can still be used for model training. If only a single index is used for data anomaly judgment, the model will miss the training data in the special scene, resulting in lack of diversity of the training data.

[0054] After obtaining the data segment associated with the driving scene and determining the abnormal frame in which the associated lane does not exist in the data segment, the driving scene in which the abnormal frame is located is judged. If the driving scene is a target driving scene in which the object in the frame does not need to be associated with the lane, the data frame for model training is determined based on the map information in the abnormal frame, which can avoid the use of a single indicator for data anomaly judgment, which may lead to the model missing training data in special scenes, resulting in lack of diversity of training data.

[0055] It should be noted that the execution subject of the embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions. The following will take the data processing device as an example to explain the embodiment and the following embodiments.

[0056] Based on this, the embodiment of the present application provides a data processing method, which refers to Figure 1 , Figure 1 The flowchart of the first embodiment of the data processing method of the present application is shown in the figure.

[0057] In the embodiment, the data processing method includes steps S100-S300:

[0058] Step S100, obtaining a data segment associated with a driving scene;

[0059] It should be noted that the execution subject of the embodiment is a data processing device. The data processing device is provided with a sensor that collects information associated with the driving scene of the ego vehicle and generates a data segment. The data segment is composed of continuous frame data. The data processing device obtains the data segment composed of continuous frame data.

[0060] Step S200, if it is determined that the data segment has an abnormal frame in which the object does not exist in the associated lane, it is determined whether the driving scene in which the abnormal frame is located is a target driving scene;

[0061] It can be understood that the target driving scene is a turning scene at an intersection or other special scene in which part of the map may be missing, for example, a scene in which the map data is incomplete or inaccurate due to construction area and temporary road change. After detecting the abnormal frame in which the object does not exist in the associated lane in the data segment, the data processing device will determine whether the driving scene in which the abnormal frame is located is a turning scene at an intersection or other special scene in which part of the map may be missing, for example, a scene in which the map data is incomplete or inaccurate due to construction area and temporary road change.

[0062] Additionally, it needs to be noted that, since the ego vehicle turns at the intersection, there is no connection or only a virtual connection between the current lane where the ego vehicle is located and the target lane to be turned to, which can cause the map to be lost during the turning process. Therefore, in the judgment of the abnormal frame in which the object in the data segment does not have an associated lane, misjudgment can occur. Therefore, step S200 is performed to avoid misjudgment of abnormal frames in special scenarios, thereby improving the diversity of training data.

[0063] In step S300, if the driving scenario is the target driving scenario, data frames for model training are determined based on the map information in the abnormal frame, wherein the object in the target driving scenario corresponding frame does not need to be associated with a lane.

[0064] It can be understood that the data frames not only reflect normal driving conditions, but also include special conditions in the target driving scenario, so as to improve the generalization ability of the model in different environments. The data processing device allows the object in the current target driving scenario associated frame to not be strictly associated with a lane, so in the case where the target driving scenario associated frame is determined to be an abnormal frame, further judgment needs to be made on whether the associated frame is abnormal based on the map information of the associated frame, and then data frames for model training are obtained. The data frames not only reflect normal driving conditions, but also include special conditions in the target driving scenario, so as to improve the generalization ability of the model in different environments.

[0065] In a possible implementation, step S200 includes steps S210-S220.

[0066] In step S210, if an abnormal frame in which the object in the data segment does not have an associated lane is determined, map information in the abnormal frame is obtained.

[0067] It needs to be noted that the map information includes lane center line information, traffic signs and signal lights, and other map-related information. The data processing device extracts the map information including lane center line information, traffic signs and signal lights, and other map-related information for each frame in the abnormal frame when confirming the abnormal frame in which there is no associated lane.

[0068] In step S220, based on the map information, it is judged whether the driving scenario in which the abnormal frame is located is a target driving scenario.

[0069] It can be understood that the data processing device will judge whether the driving scenario in which the abnormal frame is located is a target driving scenario according to the map information extracted for the abnormal frame.

[0070] Specifically, the data processing device is pre-provided with an ego_lane_center_need_exist parameter, which is used to indicate whether the ego vehicle must be associated with a valid lane. Generally, in the process of trajectory planning of the ego vehicle, the data processing device needs to ensure that the ego vehicle is driving on a determined lane, i.e., the ego_lane_center_need_exist is set to True, indicating that the ego vehicle needs to be associated with a valid lane, and by default, the ego_lane_center_need_exist is set to True. In some special scenarios, when the valid lane associated with the ego vehicle cannot be determined, it is further needed to determine whether the map is valid and the data is valid, i.e., the ego_lane_center_need_exist is set to False.

[0071] In a feasible implementation, the step S200 comprises the following steps:

[0072] The lane information associated with the object in the current frame is calculated.

[0073] It should be noted that the lane information includes associated lane information, minimum projection distance and associated lane type. The data processing device calculates the minimum projection distance of each object appearing in the current frame with respect to each lane center line, respectively, to ensure that each object can be correctly associated with the closest lane, and records the projection type and the minimum projection distance.

[0074] The objects in the current frame are screened to obtain target objects of a preset type, wherein the target objects include the ego vehicle.

[0075] It can be understood that the preset type can be pre-set, but at least the ego vehicle is included in the preset type. The target object is a pre-selected obstacle that needs to be associated with a lane. The data processing device does not consider all obstacles in the current frame, so the data processing device screens the objects in the current frame to obtain target objects of a preset type.

[0076] Specifically, the data processing device is pre-provided with the following parameters:

[0077] check_map_only_ego: indicating whether to check only the ego vehicle, when the check_map_only_ego is set to True, only the ego vehicle is checked, and other obstacles are skipped, and when the check_map_only_ego is set to False, all objects are checked;

[0078] check_map_not_pedestrian: indicating whether to filter pedestrians, and when the check_map_not_pedestrian is set to False, non-motor vehicles are not filtered;

[0079] check_map_not_bicycle: indicates whether to check if the line of non-motor vehicles, including bicycles and tricycles, needs to be filtered, and is set to False to allow the ego vehicle to temporarily leave the lane in some special scenarios;

[0080] Further, the data processing device checks each obstacle obj_id in each frame of data one by one, and determines whether to perform lane association checking according to the above parameter settings. If check_map_only_ego is set to True, only the lane association of the ego vehicle is checked, and other obstacles are skipped. If check_map_only_ego is set to False, the remaining parameters are checked. If check_map_only_ego is set to True, all objects identified as pedestrians are skipped. If check_map_not_bicycle is set to True, all objects identified as non-motor vehicles (such as bicycles, tricycles, etc.) are skipped.

[0081] Based on the lane information, it is determined whether the target object has an associated lane.

[0082] It should be noted that the data processing device determines whether the target object has an associated lane according to the associated lane information in the lane information.

[0083] If the target object does not have an associated lane, the current frame is determined to be an abnormal frame in which there is no associated lane in the data segment.

[0084] It can be understood that the data processing device performs a preliminary judgment on the target object. If the current frame has a target object that does not have an associated lane, the current frame is identified as an abnormal frame.

[0085] In a possible implementation, the step of calculating the lane information associated with the objects in the current frame includes:

[0086] Obtaining map information in the current frame, and extracting lane centerlines from the map information in the current frame;

[0087] It should be noted that the lane centerline can be regarded as a plurality of line segments connected by a plurality of points. The data processing device extracts lane centerline data from the map information in the current frame, which is usually represented in the form of a series of points.

[0088] Calculating the minimum projection distance of the object in the current frame to each of the lane centerlines;

[0089] It can be understood that the data processing device calculates the minimum projection distance of each object in the current frame to each lane centerline.

[0090] Specifically, the data processing device initializes variables before data processing, and the specific variables are as follows:

[0091] distance_small is initialized to a large number (default 10000), which is used to store the projection distance of the object to the lane center line;

[0092] project_type_small (default -1) is used to represent the projection type of the object to the lane center line:

[0093] project_type_small=0, indicating that the object projection point falls on the lane;

[0094] project_type_small=1, indicating that the object projection point falls in front of the lane;

[0095] project_type_small=2, indicating that the object projection point falls behind the lane;

[0096] cur_lane_id (default -1) represents the current minimum projection distance corresponding to the lane center line ID;

[0097] Further, the data processing device first calculates the position coordinates of the current object obj_id_x and obj_id_y, and for each lane center line, calculates the projection distance of the object to the lane center line, and finds the smallest distance. For each lane center line lane_id, for each two consecutive points p1 and p2 on the lane center line lane_id, according to the position of the projection point, the projection type project_type is judged, and the shortest projection distance dist_to_line of the object to the straight line p1-p2 is calculated by using the geometric method.

[0098] Further, if the currently calculated distance dist_to_line is less than the recorded minimum distance distance_small, then update:

[0099] distance_small is dist_to_line;

[0100] project_type_small is project_type;

[0101] cur_lane_id is the current lane_id;

[0102] After the data processing device traverses all the lane center lines, distance small, project type small and cur lane id contain the preliminary associated lane, projection type and minimum projection distance of the object.

[0103] In particular, the data processing device can further analyze the relationship between the ego vehicle and the lane line according to the projection type.

[0104] Based on the minimum projection distance, determine the associated lane of the object;

[0105] It should be noted that the data processing device determines the lane corresponding to the minimum projection distance of each object as the associated lane of the object.

[0106] Based on the minimum projection distance and the associated lane, determine the lane information.

[0107] It can be understood that the data processing device determines the lane information of the current frame based on the minimum projection distance and the associated lane.

[0108] In a possible implementation, based on the lane information, determining whether the target object exists in the associated lane includes the following steps:

[0109] If the minimum projection distance of the target object of the preset type is equal to a preset first projection threshold or the minimum projection distance is greater than a preset second projection threshold, the target object does not exist in the associated lane.

[0110] It should be noted that the preset first projection threshold is -1m, indicating that no valid lane center line is found, which may be due to missing map information or incomplete lane data. The preset second projection threshold is 3m, indicating that the distance between the ego vehicle and the nearest lane center line is large, which may not be on the lane or deviate from the lane far away. If the data processing device detects that the minimum projection distance of the target object is greater than or equal to 3m or -1m, it is considered that the ego vehicle is not associated with a valid lane.

[0111] In this embodiment, the data processing device further analyzes the abnormal frame in the special scene to avoid misjudgment of the abnormal frame in the special scene, and can accurately determine the relationship between the object and the lane by calculating the minimum projection distance of the object to each lane center line, which is not only suitable for the ego vehicle, but also suitable for other vehicles, pedestrians and non-motor vehicles, ensuring that the method can be applied to more scenes.

[0112] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as the above embodiment one can refer to the above introduction, and the following will not be repeated. On this basis, please refer to Figure 3The step S300 of the data processing method further includes steps A100-S200.

[0113] In step A100, if the driving scene is the target driving scene, it is determined whether the map information in the abnormal frame is valid.

[0114] It should be noted that if the data processing device determines that the driving scene of the current frame is the target driving scene, it further checks the validity of the map information of each abnormal frame to ensure that the map data in the abnormal frame obtained by the preliminary determination is still reliable.

[0115] In step A200, if the map information is valid, the abnormal frame and the data frame with the associated lane in the data segment are used as the data frame for model training.

[0116] It can be understood that if the data processing device determines that the map information of the abnormal frame is valid, the abnormal frame and the data frame with the associated lane can be used as the data frame for model training. These data frames increase the diversity of model training data and can help the model learn how to handle special cases.

[0117] In an available implementation, if the map information is valid, the step of using the abnormal frame and the data frame with the associated lane in the data segment as the data frame for model training includes:

[0118] Traverse each frame in the data segment, and take the traversed frame as the current frame.

[0119] It should be noted that the data processing device traverses the data segment frame by frame, and takes the currently traversed frame as the current frame.

[0120] Based on the map information, the number of lane center lines is determined, and if the number of lane center lines is greater than or equal to a preset center line number threshold, it is determined whether there is a target frame in the data segment after a preset number of frames from the current frame.

[0121] It can be understood that the preset center line number threshold is 1, which can be modified according to the actual scene. The preset center line number threshold is used to determine whether the current frame contains enough lane center lines. The preset number of frames is 60 frames, which can be modified according to the actual scene. The data processing device extracts and counts the number of lane center lines according to the map information of the current frame. If the number of lane center lines of the current frame is greater than or equal to 1, the next step is continued. From the current frame, the frame after the preset number of frames is searched, and it is checked whether the frame exists. If there is a target frame within the preset number of frames after the current frame, the next step is continued, otherwise, the frame is skipped and the next frame is traversed.

[0122] If the target frame exists, an angle change value between a first heading angle of the ego vehicle in the target frame and a second heading angle of the ego vehicle in the current frame is determined, and the angle change value is normalized to obtain a normalized angle change value;

[0123] It should be noted that the preset change value threshold range is [π / 6, 11π / 6]. When the target frame exists in the data segment, the data processing device extracts the heading angle of the ego vehicle from the target frame, extracts the heading angle of the ego vehicle from the current frame, and obtains the heading angle of the ego vehicle in different frames. After obtaining the heading angle of the ego vehicle in different frames, the difference between the first heading angle in the target frame and the second heading angle in the current frame is calculated to obtain the angle change value.

[0124] Specifically, the data processing device determines that the current frame ego vehicle heading angle is curr_ego_theta, calculates the heading angle of the ego vehicle in the future 60th frame (if it exists) future_ego_theta, and calculates the angle change value theta_diff of future_ego_thetacurr_ego_thet, and is normalized to [0, 2π]. The calculation formula is as follows:

[0125] ;

[0126] Further, after the heading angle difference is normalized, the data processing device determines whether the heading angle difference is between [π / 6, 11π / 6]. If theta_diff is less than π / 6 or theta_diff is greater than 11π / 6, the frame map is invalid.

[0127] If the normalized angle change value is within the preset change value threshold range, all data frames between the current frame and the target frame are traversed to determine whether the target obstacle exists in the all data frames.

[0128] It can be understood that the target obstacle is a pedestrian crossing or a traffic signal, which can be modified according to the actual scene. After obtaining the normalized angle change value, if the normalized angle change value is within the preset change value threshold range, all data frames between the current frame and the target frame are traversed to check whether the pedestrian crossing or the traffic signal exists in these frames. If the pedestrian crossing or the traffic signal is detected in any one of the frames between the current frame and the target frame, the next step is continued; otherwise, the frame is skipped and the next frame is traversed.

[0129] If the target obstacle exists, the map information in the abnormal frame is determined to be valid, and the abnormal frame and the data frame with the associated lane are used as data frames for model training.

[0130] It should be noted that the data processing device confirms that the target obstacle exists in the current frame and the frames before and after the current frame, and the change of the heading angle of the ego vehicle is within a reasonable range, and it is considered that the map information in the abnormal frame is valid. After confirming the validity, the abnormal frame and the data frame in which the associated lane exists are regarded as data frames for model training.

[0131] Further, the data processing device determines the validity of the data segment according to different scene requirements after determining the abnormal frame, and the specific determination rules are as follows:

[0132] If a frame of map abnormality appears in the scene segment, it is considered that the segment data is abnormal;

[0133] If the proportion of the map abnormal frames in the scene segment to the entire data segment frame number is greater than a threshold value, it is considered that the segment data is abnormal;

[0134] If the number of continuous normal frames in the scene segment accounts for more than a threshold value of the entire data segment frame number, it is considered that the segment is normal.

[0135] In the embodiment, the data processing device determines the validity of the map information in the associated frame in the special scene, which not only ensures the diversity of the training data, but also ensures the accuracy of the training data. In addition, the validity of the data segment can be further determined according to different scene requirements.

[0136] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the data processing method of the present application. More simple transformations based on the technical concept are within the protection scope of the present application.

[0137] The present application also provides a data processing device, please refer to Figure 4 , the data processing device comprises:

[0138] The acquisition module 10 is configured to acquire a data segment associated with a driving scene;

[0139] The determination module 20 is configured to determine whether the driving scene in which the abnormal frame is located is a target driving scene if it is determined that there is an abnormal frame in which the object does not exist in the associated lane in the data segment.

[0140] The determination module 30 is configured to determine data frames for model training based on the map information in the abnormal frame if the driving scene is the target driving scene, wherein the object in the frame corresponding to the target driving scene does not need to be associated with a lane.

[0141] Optionally, the determination module comprises:

[0142] The determining sub-module is configured to: if it is determined that the object in the data segment does not exist in the abnormal frame of the associated lane, acquire map information in the abnormal frame; and determine whether the driving scene in which the abnormal frame is located is the target driving scene based on the map information.

[0143] The associated lane determining sub-module is configured to: calculate lane information associated with the object in the current frame; filter the object in the current frame to obtain a target object of a preset type, wherein the target object includes the ego vehicle; determine whether the target object exists in the associated lane based on the lane information; and if the target object does not exist in the associated lane, determine the current frame as the abnormal frame in which the data segment does not exist in the associated lane.

[0144] Optionally, the associated lane determining sub-module comprises:

[0145] The computing unit is configured to: acquire map information in the current frame, and extract a lane center line from the map information in the current frame; calculate a minimum projection distance of the object in the current frame to each lane center line; determine the associated lane of the object based on the minimum projection distance; and determine lane information based on the minimum projection distance and the associated lane.

[0146] The threshold determining unit is configured to: if the minimum projection distance of the target object of the preset type is equal to a preset first projection threshold or the minimum projection distance is greater than a preset second projection threshold, the target object does not exist in the associated lane.

[0147] Optionally, the determining module comprises:

[0148] The effective determining sub-module is configured to: if the driving scene is the target driving scene, determine whether the map information in the abnormal frame is effective; and if the map information is effective, take the abnormal frame and the data frame in which the data segment exists in the associated lane as a data frame for model training.

[0149] Optionally, the effective determining sub-module comprises:

[0150] The normalization unit is configured to traverse each frame in the data segment, take the traversed frame as a current frame, determine a number of lane center lines based on the map information, and if the number of lane center lines is greater than or equal to a preset center line number threshold, determine whether there is a target frame in the data segment after a preset number of frames of the current frame. If there is the target frame, determine an angle change value between a first heading angle of the ego vehicle in the target frame and a second heading angle of the ego vehicle in the current frame, normalize the angle change value to obtain a normalized angle change value, if the normalized angle change value is within a preset change value threshold range, traverse all data frames between the current frame and the target frame, and determine whether there is a target obstacle in the all data frames. If there is the target obstacle, determine that the map information in the abnormal frame is valid, and take the abnormal frame and the data frame having the associated lane as data frames for model training.

[0151] The data processing device provided in the present application adopts the data processing method in the above embodiments, and can solve the technical problem of data processing. Compared with the prior art, the data processing device provided in the present application has the same beneficial effects as the data processing method provided in the above embodiments, and other technical features in the data processing device are the same as the features disclosed in the above embodiments, which will not be repeated here.

[0152] The present application provides a data processing device, comprising: at least one processor; and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the data processing method in Embodiment I.

[0153] Reference will now be made to the following description Figure 5 , which shows a structural schematic diagram of a data processing device suitable for implementing the embodiments of the present application. The data processing device in the embodiments of the present application can include but is not limited to mobile terminals such as mobile phones, notebook computers, tablet computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 5 The data processing device shown is only an example, and should not impose any limitation on the functions and use range of the embodiments of the present application.

[0154] As Figure 5As shown, the data processing device can include a processing apparatus 1001 (e.g., a central processor, a graphics processor, etc.) that can perform various appropriate actions and processes according to a program stored in a read only memory (ROM) 1002 or a program loaded from a storage apparatus 1003 into a random access memory (RAM) 1004. Various programs and data required for operation of the data processing device are also stored in the RAM 1004. The processing apparatus 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input apparatus 1007 including, for example, a touch panel, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output apparatus 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage apparatus 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication apparatus 1009. The communication apparatus 1009 can allow the data processing device to communicate with other devices wirelessly or by wire to exchange data. Although the data processing device having various systems is shown in the figure, it should be understood that all the shown systems are not required to be implemented or possessed. More or less systems can be alternatively implemented or possessed.

[0155] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by a communication apparatus, or installed from the storage apparatus 1003, or installed from the ROM 1002. When the computer program is executed by the processing apparatus 1001, the above-mentioned functions defined in the methods of embodiments of the present disclosure are performed.

[0156] The data processing device provided by the present disclosure adopts the data processing method in the above-mentioned embodiments, and can solve the technical problem of data processing. Compared with the prior art, the data processing device provided by the present disclosure has the same beneficial effects as the data processing method provided by the above-mentioned embodiments, and other technical features in the data processing device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0157] It should be understood that various parts of the present application can be realized with hardware, software, firmware, or a combination thereof. In the above description of embodiments, specific functional configurations, structures, materials or characteristics can be combined in any appropriate manner in one or more embodiments or examples.

[0158] The above description is merely that of specific embodiments of the present application, but the scope of the present application is not limited thereto. Any person skilled in the art can easily conceive various changes or substitutions within the technical scope of the present application, and all such changes and substitutions should be encompassed within the scope of the present application. Therefore, the scope of the present application should be decided based on the scope of the claims.

[0159] The present application provides a computer-readable storage medium having stored thereon computer-readable program instructions (i.e., a computer program) for performing the data processing method in the above-described embodiments.

[0160] The computer-readable storage medium provided by the present application may, for example, be a U disk, but is not limited to an electric, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination thereof. More specific examples of the computer-readable storage medium can include, but are not limited to, an electric connection having one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination thereof. In the present embodiment, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium can be transmitted by any appropriate medium, including but not limited to an electric wire, an optical cable, an RF (Radio Frequency), etc., or any appropriate combination thereof.

[0161] The above-described computer-readable storage medium can be contained in a data processing device, or can exist separately without being assembled into the data processing device.

[0162] The computer readable storage medium described above carries one or more programs, when the one or more programs are executed by a data processing device, cause the data processing device to: acquire a data segment associated with a driving scene; if it is determined that an abnormal frame in which an object in the data segment does not exist a relevant lane, determine whether the driving scene where the abnormal frame is located is a target driving scene; if the driving scene is the target driving scene, determine a data frame for model training based on map information in the abnormal frame, wherein the object in the target driving scene corresponding frame does not need a relevant lane.

[0163] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0164] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0165] The modules involved in the embodiments of the present application can be implemented in a software manner or in a hardware manner. In some cases, the name of the module does not constitute a limitation on the module itself.

[0166] The readable storage medium provided by the application is a computer readable storage medium, which stores computer readable program instructions (i.e. computer programs) for executing the above-mentioned data processing method, and can solve the technical problem of data processing. Compared with the prior art, the beneficial effects of the computer readable storage medium provided by the application are the same as those of the data processing method provided by the above-mentioned embodiments, which will not be repeated here.

[0167] The application further provides a computer program product comprising a computer program, which, when executed by a processor, implements the steps of the data processing method as described above.

[0168] The computer program product provided by the application can solve the technical problem of data processing. Compared with the prior art, the beneficial effects of the computer program product provided by the application are the same as those of the data processing method provided by the above-mentioned embodiments, which will not be repeated here.

[0169] The above-mentioned is only part of the embodiments of the application, and does not limit the patent scope of the application, and any equivalent structural transformation made by using the content of the application specification and drawings, or direct / indirect application in other related technical fields is included in the patent protection scope of the application.

Claims

1. A data processing method, characterized in that: The data processing method includes: Obtaining data segments associated with the driving scenario; If it is determined that there is no abnormal frame associated with the lane in the data segment, determining whether the driving scene in which the abnormal frame is located is the target driving scene; If the driving scene is the target driving scene, determining a data frame for model training based on the map information in the abnormal frame, wherein the object in the frame corresponding to the target driving scene does not need to be associated with a lane; If the driving scene is the target driving scene, the step of determining a data frame for model training based on the map information in the abnormal frame includes: If the driving scene is the target driving scene, determining whether the map information in the abnormal frame is valid; If the map information is valid, the abnormal frame and the data frame with the associated lane in the data segment are used as data frames for model training; If the map information is valid, the step of using the abnormal frame and the data frame with the associated lane in the data segment as data frames for model training includes: Traversing each frame in the data segment and taking the traversed frame as the current frame; Determining the number of lane centerlines based on the map information, and if the number of lane centerlines is greater than or equal to a preset centerline number threshold, determining whether a target frame exists within a preset number of frames after the current frame; If the target frame exists, determining an angle change between a first heading angle of the ego vehicle in the target frame and a second heading angle of the ego vehicle in the current frame, and normalizing the angle change to obtain a normalized angle change value; If the normalized angle change value is within a preset change value threshold range, traversing all data frames between the current frame and the target frame to determine whether there is a target obstacle in all data frames; If the target obstacle exists, the map information in the abnormal frame is determined to be valid, and the abnormal frame and the data frame with the associated lane are used as data frames for model training.

2. The data processing method according to claim 1, wherein: If it is determined that there is no abnormal frame associated with the lane in the data segment, the step of determining whether the driving scene in which the abnormal frame is located is the target driving scene includes: If it is determined that there is no abnormal frame associated with the lane in the data segment, obtaining map information in the abnormal frame; Based on the map information, it is determined whether the driving scene in which the abnormal frame is located is a target driving scene.

3. The data processing method according to claim 1, wherein: If it is determined that there is no abnormal frame associated with the lane in the data segment, the step of determining whether the driving scene in which the abnormal frame is located is the target driving scene includes: Calculating lane information associated with the object in the current frame; Filtering the objects in the current frame to obtain target objects of a preset type, wherein the target objects include the vehicle; Based on the lane information, determining whether the target object has an associated lane; If the target object does not have an associated lane, the current frame is determined as an abnormal frame in the data segment that does not have an associated lane.

4. The data processing method according to claim 3, wherein: The step of calculating and obtaining lane information associated with the object in the current frame includes: Obtaining map information in the current frame, and extracting a lane centerline from the map information in the current frame; Calculating the minimum projection distance from the object in the current frame to the center line of each lane; determining an associated lane of the object based on the minimum projection distance; Lane information is determined based on the minimum projection distance and the associated lane.

5. The data processing method according to claim 4, wherein: The step of determining whether the target object has an associated lane based on the lane information includes: If the minimum projection distance of the target object of the preset type is equal to a preset first projection threshold or the minimum projection distance is greater than a preset second projection threshold, the target object has no associated lane.

6. A data processing device, characterized in that: The device comprises: An acquisition module, configured to acquire data segments associated with the driving scenario; a judgment module, configured to, if it is determined that there is no abnormal frame associated with the lane in the data segment, determine whether the driving scene in which the abnormal frame is located is a target driving scene; a determination module configured to determine, if the driving scene is the target driving scene, a data frame for model training based on map information in the abnormal frame, wherein the object in the frame corresponding to the target driving scene does not need to be associated with a lane; The determining module includes: a validity judgment submodule, configured to, if the driving scene is the target driving scene, determine whether the map information in the abnormal frame is valid; and if the map information is valid, use the abnormal frame and the data frame with the associated lane in the data segment as data frames for model training; The validity judgment submodule includes: A normalization unit is configured to traverse each frame in the data segment and use the traversed frame as the current frame; determine the number of lane centerlines based on the map information, and if the number of lane centerlines is greater than or equal to a preset centerline number threshold, determine whether a target frame exists within a preset number of frames after the current frame; if the target frame exists, determine an angle change value between a first heading angle of the vehicle in the target frame and a second heading angle of the vehicle in the current frame, and normalize the angle change value to obtain the normalized angle change value; if the normalized angle change value is within a preset change value threshold, traverse all data frames between the current frame and the target frame to determine whether a target obstacle exists in all data frames; if the target obstacle exists, determine that the map information in the abnormal frame is valid, and use the abnormal frame and the data frame with the associated lane as data frames for model training.

7. A data processing device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the data processing method according to any one of claims 1 to 5.

8. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the data processing method according to any one of claims 1 to 5 are implemented.

9. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the data processing method according to any one of claims 1 to 5 are implemented.

Images