Autonomous driving storage system, method, device and storage medium

Abstract

The application discloses an automatic driving storage system, method, equipment and storage medium, and the system comprises a data processing module and a screening module which are communicatively connected; the data processing module is used for acquiring an original frame sequence, correcting the original frame sequence and obtaining a corrected frame sequence; the screening module is used for screening the corrected frame sequence and obtaining a target key frame set; wherein the screening unit comprises a distribution unit and a calculation unit which are communicatively connected; the distribution unit is used for obtaining an original sample set based on the corrected frame sequence, and obtaining an updated distribution result matched with each original sample based on the original sample set; and the calculation unit is used for calculating each updated distribution result, obtaining a calculation result and comparing the calculation result with a threshold value of the calculation result. The technical scheme of the application can assist a perception device and the like in discovering potential risks.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving technology, and in particular relates to an autonomous driving storage system, method, device and storage medium. Background Technology

[0002] Currently, autonomous driving storage systems at Level 3 and above require the accumulation of a large amount of data during the early development and later mass production processes. Therefore, it is crucial to effectively utilize storage resources and store different data for different functions.

[0003] However, current autonomous driving storage systems directly store the acquired data without correcting or filtering it, causing errors in the data to be ignored. As a result, the reliability of the decisions obtained is low, which in turn reduces the safety of autonomous driving. Summary of the Invention

[0004] This invention aims to at least partially solve one of the technical problems in the related art. Therefore, one object of this invention is to provide an autonomous driving storage system, method, device, and storage medium.

[0005] To address the aforementioned technical problems, embodiments of the present invention provide the following technical solutions:

[0006] An autonomous driving storage system includes:

[0007] The data processing module and filtering module for communication connection;

[0008] The data processing module is used to acquire the original frame sequence and correct the original frame sequence to obtain a corrected frame sequence.

[0009] The filtering module is used to filter the corrected frame sequence to obtain a target keyframe set;

[0010] The filtering unit includes a distribution unit and a calculation unit connected by communication. The distribution unit is used to obtain an original sample set based on the correction frame sequence and to obtain an updated distribution result matching each original sample based on the original sample set. The calculation unit is used to calculate each updated distribution result, obtain a calculation result, and compare the calculation result with a threshold of the calculation result. If the calculation result is greater than the threshold of the calculation result, the original sample corresponding to the calculation result is determined as the target keyframe.

[0011] Optionally, the data processing module includes a scene annotation unit and a verification unit;

[0012] The scene annotation unit is used to annotate each of the original frames to obtain an annotated frame sequence;

[0013] The inspection unit is used to acquire the labeled frame sequence and inspect the labeled frame sequence to obtain an inspection result; the inspection unit classifies the labeled frame sequence based on the inspection result to obtain a first labeled frame sequence, a second labeled frame sequence, and a third labeled frame sequence.

[0014] Optionally, the data processing module further includes a first correction unit;

[0015] The first correction unit is used to acquire the first labeled frame sequence and correct the first labeled frame sequence to obtain a first corrected frame sequence.

[0016] Optionally, the data processing module further includes a second correction unit;

[0017] The second correction unit is used to acquire the second labeled frame sequence and perform secondary labeling on the second labeled sequence to obtain the second corrected frame sequence.

[0018] Optionally, the correction frame sequence includes a first correction frame sequence, a second correction frame sequence, and a third annotation frame sequence; wherein the first correction frame sequence, the second correction frame sequence, and the third annotation frame sequence are arranged based on time sequence.

[0019] Optionally, the distribution unit obtains the original sample set distribution result of the original sample set based on the correction sequence; wherein the original sample set includes N original samples; wherein N≥1, and N is an integer;

[0020] The distribution unit uses the Kth original sample as the Kth parameter and the remaining N-1 original samples in the original sample set as the Kth reference sample set.

[0021] The distribution unit extracts the Kth original sample from the distribution result of the original sample set, and determines the remaining N-1 original samples in the distribution result of the original sample set as the Kth reference sample set distribution result M. K Where 1 ≤ K ≤ N, and K is an integer;

[0022] The distribution unit traverses the Kth reference sample set to generate the Kth updated distribution result L. K .

[0023] Optionally, the computing unit obtains the Kth updated distribution result L. K and the distribution result M of the Kth reference sample set K And for the Kth update distribution result L K and the distribution result M of the Kth reference sample set KPerform the calculation to obtain the Kth result;

[0024] The Kth calculation result is compared with the threshold of the calculation result;

[0025] If the Kth calculation result is greater than the threshold of the calculation result, then the Kth original sample is determined as the Kth target keyframe.

[0026] Embodiments of the present invention also provide an autonomous driving storage method, comprising:

[0027] Obtain the original frame sequence and correct the original frame sequence to obtain the corrected frame sequence;

[0028] The original sample set is obtained based on the corrected frame sequence, and the updated distribution result matching each original sample is obtained based on the original sample set.

[0029] For each of the updated distribution results, a calculation is performed to obtain a calculation result, and the calculation result is compared with a threshold of the calculation result; if the calculation result is greater than the threshold of the calculation result, the original sample corresponding to the calculation result is determined as the target keyframe.

[0030] Embodiments of the present invention also provide an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the system as described above.

[0031] Embodiments of the present invention also provide a computer-readable storage medium comprising a stored computer program, wherein the computer program, when executed, controls the device containing the computer-readable storage medium to execute the system as described above.

[0032] The embodiments of the present invention have the following technical effects:

[0033] The above-mentioned technical solution of the present invention 1) uses a funnel strategy to filter data layer by layer. Specifically, the original frame sequence is corrected by the data processing module to obtain a corrected frame sequence. Then, the corrected frame sequence is calculated by the filtering module to obtain the calculation result. Based on the calculation result, the corrected frame sequence is filtered to obtain a target key frame set. Subsequently, the target key frame set is provided to the perception device, etc., which can assist the perception device in discovering potential risks, improve the sensitivity of the perception device, and thus improve the safety of autonomous driving.

[0034] 2) After obtaining the target keyframe set, the data that was previously misdetected or missed by the system can be labeled manually or automatically based on the target keyframes in the target keyframe set, and applied to multiple application scenarios such as perception (e.g., recognition algorithm or detection algorithm), planning and control (e.g., geometric method, deep learning and simulator, etc.) in order to improve the computational accuracy of various subsequent algorithms.

[0035] 3) Based on the first correction unit and the second correction unit, the erroneous labels in the annotation frame sequence generated by the scene annotation unit are corrected, which realizes the separation of the offset frame. This makes up for the defects of the existing autonomous driving system that directly stores the acquired data source and fails to detect certain erroneous data, thereby improving the accuracy of data calculation and reducing the error of subsequent selection of target key frames.

[0036] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the structure of an autonomous driving storage system provided in an embodiment of the present invention;

[0038] Figure 2 This is an example of an autonomous driving storage system provided in an embodiment of the present invention;

[0039] Figure 3 This is the first example of a data processing module provided in the embodiments of the present invention;

[0040] Figure 4 This is a second example of a data processing module provided in an embodiment of the present invention;

[0041] Figure 5 This is a third example of a data processing module provided in the embodiments of the present invention;

[0042] Figure 6 This is a flowchart illustrating an autonomous driving storage method provided in an embodiment of the present invention. Detailed Implementation

[0043] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0044] To facilitate understanding of the embodiments by those skilled in the art, some terms are explained in detail:

[0045] (1)CNN: Convolutional Neural Networks.

[0046] (2) Transformer: A transformation model that computes representations of inputs and outputs based on self-attention.

[0047] Currently, some autonomous driving storage systems directly store all the data they acquire without correction or filtering. Applying erroneous data directly to these original data sources significantly reduces the computational accuracy of subsequent algorithms that call upon them. For example, if the original data source contains erroneous frame data, subsequent algorithms may ignore potential driving risks based on this data, leading to incorrect decisions. For instance, when encountering a 10cm diameter rock, the autonomous driving system might ignore it due to insufficient detection accuracy (it can only detect rocks larger than 10cm) or erroneous frame data. Therefore, when driving based on decisions made by the autonomous driving system, the vehicle will not avoid the rock, greatly reducing the comfort and safety of the autonomous driving experience.

[0048] To solve the above technical problems, such as Figure 1 As shown, an embodiment of the present invention provides an autonomous driving storage system, comprising:

[0049] The data processing module and filtering module for communication connection;

[0050] The data processing module is used to acquire the original frame sequence and correct the original frame sequence to obtain a corrected frame sequence.

[0051] The filtering module is used to filter the corrected frame sequence to obtain a target keyframe set;

[0052] The filtering unit includes a distribution unit and a calculation unit connected by communication. The distribution unit is used to obtain an original sample set based on the correction frame sequence and to obtain an updated distribution result matching each original sample based on the original sample set. The calculation unit is used to calculate each updated distribution result, obtain a calculation result, and compare the calculation result with a threshold of the calculation result. If the calculation result is greater than the threshold of the calculation result, the original sample corresponding to the calculation result is determined as the target keyframe.

[0053] In embodiments of the present invention, the target key frame is to obtain potential risk frame data, which may specifically include frame data that affects the accuracy or safety of autonomous driving, such as false detection frame data and missed detection frame data.

[0054] like Figure 2 As shown, embodiments of the present invention may further include a data acquisition module. In practical application scenarios, the autonomous driving storage system is installed on the target vehicle. The data acquisition module and the sensing device are connected via a network to achieve communication, thereby enabling the data acquisition module to obtain data uploaded by the sensing device via the network.

[0055] Among them, sensing devices include, but are not limited to, various sensors, vehicle cameras, positioning devices, and radar;

[0056] Specifically, based on the aforementioned sensing devices, real-time status data of the target vehicle can be obtained, including but not limited to sensing data uploaded to the target vehicle's cloud, image data collected by all onboard cameras, radar point cloud data, map data, positioning data, and driving behavior data generated during the target vehicle's operation; it may also include data from shadow mode, etc.

[0057] In addition, it also includes non-real-time status data uploaded to the cloud of the target vehicle, such as data uploaded to the cloud on a daily basis and data in shadow mode.

[0058] It should be noted that in the embodiments of the present invention, real-time status data and non-real-time status data are defined (annotated) using the same data structure, but the differences in data streams arise due to the different granularity of the upload time. The real-time status data and non-real-time status data can complement each other to improve the system security of the embodiments of the present invention.

[0059] In an optional embodiment of the present invention, after obtaining the target keyframe set, data that was previously misdetected or missed by the system can be labeled manually or automatically based on the target keyframes in the target keyframe set. This labeled data can then be applied to various application scenarios in the subsequent system, such as perception (e.g., recognition or detection algorithms) and control (e.g., geometric methods, deep learning, and simulators), to improve the computational accuracy of various subsequent algorithms. For example, if the target keyframe set obtained in the embodiment of the present invention is applied to the control algorithm, then the error data or related data that the control algorithm previously failed to detect can be labeled manually or by the system based on the target keyframe set. When the control algorithm performs subsequent trajectory prediction, it can refer to this supplementary data (e.g., when generating a new predicted trajectory, it will avoid stones with a diameter of 10cm), thereby improving the safety of autonomous driving.

[0060] In an optional embodiment of the present invention, the working cycle of the screening module can be preset, for example, different working cycles such as 24 hours or one week;

[0061] In practical applications, whenever the filtering module has a preset working cycle, it retrieves some data from the data processing module. The preset working cycle can reduce the repetitive processing of some data and can also be used to specifically correct or improve the accuracy of the system, avoiding the waste of computing power by blindly filtering the labeled frame sequence without a specific purpose.

[0062] Furthermore, the filtering module can preset the retrieval threshold, that is, the filtering module retrieves the data in the data processing module each time according to the retrieval threshold, thereby obtaining the original sample set. In this way, when obtaining the target key frame set each time, it is not necessary to filter all the labeled frame sequences in the system, which greatly reduces the computing power required in the filtering process and improves the working efficiency of the filtering module.

[0063] In embodiments of the present invention, a funnel strategy is used to filter data layer by layer. Specifically, the original frame sequence is corrected by the data processing module to obtain a corrected frame sequence. Then, the corrected frame sequence is calculated by the filtering module to obtain the calculation result. Based on the calculation result, the corrected frame sequence is filtered to obtain a target key frame set. Subsequently, the target key frame set is provided to perception devices, which can assist perception devices in discovering potential risks, improve the sensitivity of perception devices, and thus improve the safety of autonomous driving.

[0064] like Figure 3 As shown, in an optional embodiment of the present invention, the data processing module includes a scene annotation unit and a verification unit;

[0065] The scene annotation unit is used to annotate each of the original frames to obtain an annotated frame sequence;

[0066] The inspection unit is used to acquire the labeled frame sequence and inspect the labeled frame sequence to obtain an inspection result; the inspection unit classifies the labeled frame sequence based on the inspection result to obtain a first labeled frame sequence, a second labeled frame sequence, and a third labeled frame sequence.

[0067] In practical application scenarios, taking the sensor to collect data as an example, the sensor can collect the state data of the target vehicle in real time. For example, if one frame of data is collected every 0.1 seconds, then 10 frames of data can be collected in 1 second. After collecting for a period of time, the original frame sequence can be obtained. The acquisition module inputs the original frame sequence into the scene annotation unit in chronological order, and then the scene annotation unit annotates each frame of data received.

[0068] The scenarios can include driving scenarios and traffic scenarios, including but not limited to road structure, traffic lights, construction and targets (other vehicles or pedestrians, etc.), whether lane changes occur, whether other vehicles cut in or out, whether to follow or stop, and environmental characteristics of other target vehicles during their driving process.

[0069] For example, taking weather as an example, consecutive frames of data are labeled as sunny, cloudy, or rainy according to time sequence; the scene labeling unit stores the obtained labeled frame sequence for later retrieval.

[0070] In an embodiment of the present invention, when the number of labeled frame sequences in the scene labeling unit exceeds a threshold (e.g., 10,000 frames), the scene labeling unit sends the number of labeled frame sequences to the verification unit for verification. That is, the verification unit performs intermittent verification based on the data size of the labeled frame sequences in the scene labeling unit, without having to work continuously, thereby reducing the system's computing power.

[0071] Specifically, the verification is based on the following benchmarks:

[0072] First, preset a first proportional threshold and a second proportional threshold;

[0073] The labeled sequence is inspected based on the inspection unit, which is a segment. For example, each segment may include 10 or 20 frames.

[0074] The embodiments of the present invention are explained and illustrated using an example where each segment includes 10 frames:

[0075] Specifically, taking weather as an example, the inspection unit judges the weather labels of 10 consecutive frames of images. If there are no conflicts in the weather labels, that is, all of them are correct, then the inspection continues to the next 10 frames of images, and so on, until all the inspections of the acquired labeled frame sequence are completed.

[0076] If an incorrect label appears in the weather labels of a certain 10-frame image (based on the labeling sequence, such as: labeling result is correct, incorrect, correct, incorrect, etc.), then the actual proportion of the incorrect label in the 10-frame image is obtained, and then the actual proportion is compared with the first proportion threshold and the second proportion threshold.

[0077] If the actual proportion is less than the first proportion threshold, then these 10 images are determined to be part of the first labeled frame sequence. This process is repeated until all the labels in the obtained labeled frame sequence are verified, and then the first labeled frame sequence of the labeled frame sequence can be obtained.

[0078] If the first ratio threshold ≤ the actual ratio ≤ the second ratio threshold, then these 10 images are determined to be part of the second annotation frame sequence. This process continues until all the obtained annotation frame sequences have been checked, and then the second annotation frame sequence of the annotation frame sequence can be obtained.

[0079] For segments without erroneous labels, they are stored directly in chronological order as part of the third labeled frame sequence. This process continues until all the labels in the acquired labeled frame sequence have been checked, at which point the third labeled frame sequence is obtained.

[0080] Furthermore, in embodiments of the present invention, the first proportional threshold can be 2 / 5, and the second proportional threshold can be 1 / 2.

[0081] Furthermore, during system operation, the values ​​of the first proportional threshold, the second proportional threshold, and the number of frames included in each segment can be adjusted in real time according to the requirements for system accuracy, etc.

[0082] like Figure 4 As shown in an optional embodiment of the present invention, the data processing module further includes a first correction unit;

[0083] The first correction unit is used to acquire the first labeled frame sequence and correct the first labeled frame sequence to obtain a first corrected frame sequence.

[0084] In practical application scenarios, when the inspection unit detects the existence of a certain segment and the actual proportion is less than the first proportion threshold, the inspection unit sends the segment to the first correction unit. Based on the first correction unit, the frame with the erroneous label in the segment is corrected for the erroneous label. After the correction is completed, the segment is sent to the scene annotation unit and stored again in the time sequence for subsequent use, and the continuity of the annotation frame sequence stored in the scene annotation unit is guaranteed.

[0085] like Figure 4 As shown in an optional embodiment of the present invention, the data processing module further includes a second correction unit;

[0086] The second correction unit is used to acquire the second labeled frame sequence and perform secondary labeling on the second labeled sequence to obtain the second corrected frame sequence.

[0087] In practical application scenarios, when the inspection unit detects the existence of a certain segment, and the first ratio threshold ≤ the actual ratio ≤ the second ratio threshold, the inspection unit sends the segment to the second correction unit. Based on the second correction unit, all frames of the segment are re-annotated to generate a new segment, and the new segment is sent to the scene annotation unit and stored in chronological order for later retrieval.

[0088] Specifically, in an embodiment of the present invention, the second correction unit can take a CNN+transformer model as an example, input the second labeled frame sequence into the CNN+transformer model in a temporal order, re-label it based on the CNN+transformer model, and then output the updated second labeled frame sequence;

[0089] The CNN+transformer model sends the output CNN+transformer model to the scene annotation unit and stores it in time sequence for later use.

[0090] Furthermore, the CNN+transformer model involved in the embodiments of the present invention is trained multiple times based on a large amount of historical state data of the target vehicle until its transformation accuracy reaches the expected level, at which point training stops; however, the specific training method of the CNN+transformer model is not within the scope of protection of the present invention, and therefore will not be described in detail.

[0091] like Figure 5 As shown, the second correction unit can also be set independently, that is, the second correction unit is set outside the autonomous driving storage system and interacts with the scene annotation unit through the network.

[0092] In embodiments of the present invention, erroneous labels in the annotation frame sequence generated by the scene annotation unit are corrected based on the first correction unit and the second correction unit. This achieves the separation of offset frames, overcoming the shortcomings of existing autonomous driving systems that directly store the acquired data source, leading to the failure to detect certain erroneous data. This improves the accuracy of data calculation and reduces the error in subsequent selection of target keyframes. In an optional embodiment of the present invention, the correction frame sequence includes a first correction frame sequence, a second correction frame sequence, and a third annotation frame sequence; wherein the first correction frame sequence, the second correction frame sequence, and the third annotation frame sequence are arranged in temporal order.

[0093] In embodiments of the present invention, the scene annotation unit arranges the acquired frame data in chronological order to ensure the continuity and effectiveness of the data called by subsequent algorithms.

[0094] In an optional embodiment of the present invention, the distribution unit obtains the original sample set distribution result of the original sample set based on the correction sequence; wherein the original sample set includes N original samples; wherein N≥1, and N is an integer;

[0095] The distribution unit uses the Kth original sample as the Kth parameter and the remaining N-1 original samples in the original sample set as the Kth reference sample set.

[0096] The distribution unit extracts the Kth original sample from the distribution result of the original sample set, and determines the remaining N-1 original samples in the distribution result of the original sample set as the Kth reference sample set distribution result M. K Where 1 ≤ K ≤ N, and K is an integer;

[0097] The distribution unit traverses the Kth reference sample set to generate the Kth updated distribution result L. K .

[0098] In an embodiment of the present invention, the frame data in the original sample set is processed based on the principle of loss function. The original sample set is regarded as an original sample set distribution result. A certain original sample is removed, that is, a point in the initial distribution set is deleted. The distribution state of the remaining original samples remains unchanged and is used as a reference sample set distribution result that matches a certain original sample.

[0099] Then, iterate through the remaining original samples and generate an updated distribution based on the remaining original samples.

[0100] Specifically, we take a certain original sample in the original sample set as the variable x, and traverse all the original samples in the original sample set except for the variable to form the updated distribution result f(x) of the mapping of the variable.

[0101] In practical applications, let x = Z K Where Z represents the original sample set, that is, the set of samples Z. K Z can be obtained by taking the union of the reference sample set and the reference sample set.

[0102] Then Z should be used K When Z is a variable, i.e. a parameter, then we can obtain Z. K The updated distribution of the mapping is: f(x) = L K .

[0103] For example, the original sample set includes N original samples, and each original sample is numbered from 1 to N to facilitate differentiation.

[0104] When K=1, the parameter is Z1. The distribution of the original sample set after removing Z1 is taken as the first reference sample set distribution M1. The original samples with indices 2 to N are traversed to generate the first updated distribution L1.

[0105] When K=2, the parameter is Z2. The distribution of the original sample set after removing Z2 is taken as the second reference sample set distribution M2. The original samples with indices 1, 3 to N are traversed to generate the second updated distribution L2.

[0106] When K=3, the parameter is Z3. The distribution of the original sample set after removing Z3 is taken as the third reference sample set distribution result M3. The original samples with indices 1, 2, 4 to N are traversed to generate the third updated distribution result L3.

[0107] ...

[0108] By analogy, multiple reference sample set distribution results are obtained: M4, M5, M6...M N ;

[0109] At the same time, multiple update distribution results are obtained: L4, L5, L6...L N .

[0110] In an optional embodiment of the present invention, the computing unit obtains the Kth update distribution result L. K and the distribution result M of the Kth reference sample set K And for the Kth update distribution result L K and the distribution result M of the Kth reference sample set K Perform the calculation to obtain the Kth result;

[0111] The Kth calculation result is compared with the threshold of the calculation result;

[0112] If the Kth calculation result is greater than the threshold of the calculation result, then the Kth original sample is determined as the Kth target keyframe.

[0113] In an embodiment of the present invention, in order to determine whether each original sample in the original sample set is a target keyframe, the frames determined to be target keyframes are then integrated to form a target keyframe set; specifically, in an embodiment of the present invention, the update distribution result corresponding to each parameter and the reference sample set are calculated to obtain the calculation result;

[0114] In order to filter the original sample set based on the calculation results, the embodiments of the present invention set a threshold for the calculation results, compare each calculation result with the threshold for the calculation results to obtain the comparison results, and then filter the original sample set according to the comparison results;

[0115] If the calculation result of a certain original sample is greater than the threshold of the calculation result, then the original sample is determined as the target keyframe.

[0116] If the calculation result of a certain original sample is less than or equal to the threshold of the calculation result, then the original sample will not be determined as the target keyframe.

[0117] By analogy, each original sample in the original sample set is confirmed, and then the target keyframe set is obtained based on the screening of the original sample set.

[0118] Specifically, for example, calculate the KL divergence between each updated distribution result and the corresponding reference sample set distribution result, and preset a threshold for the KL divergence;

[0119] Furthermore, the KL divergence between each updated distribution result and the corresponding reference sample set distribution result can be calculated based on the following formula:

[0120]

[0121] In the formula, L Z This is the distribution result of the original sample set, that is: for z K and M K Taking the union of the sets yields L. Z .

[0122] Where, when K=1, then

[0123] When K=2, then

[0124] When K = 3, then

[0125] ...

[0126] Similarly, the KL divergence value corresponding to each original sample is calculated; then, the KL divergence value is filtered by a threshold, for example: the threshold for the KL divergence value = 0.46;

[0127] That is, each original sample with a KL divergence value greater than 0.46 is identified as a target keyframe, and multiple target keyframes are integrated to obtain a target keyframe set.

[0128] In embodiments of the present invention, the closer the KL divergence value is to 0, the stronger the M... K and L K The more consistent the distribution, the larger the KL divergence value, indicating that M... K and L K The greater the difference in distribution states, that is, the more z is removed. K For M K The greater the impact, the larger the value of the loss function. Therefore, in the embodiments of the present invention, the value of KL divergence is used to represent the amount of information carried by the frame data of the original sample. The larger the value of KL divergence, the larger the amount of information carried by the frame data of the original sample; the smaller the value of KL divergence, the smaller the amount of information carried by the frame data of the original sample.

[0129] In embodiments of the present invention, the original sample set is filtered by a filtering module to obtain a target keyframe set. The target keyframe set can be used to represent potential risks that have not been discovered during the operation of the target vehicle. When these potential risks are discovered based on embodiments of the present invention, the training accuracy of a large number of different models of the autonomous driving system can be greatly improved, and the data storage and computing pressure can be reduced.

[0130] In an optional embodiment of the present invention, for non-real-time status data, for example, the non-real-time status data of the target vehicle is uploaded to the cloud, i.e., the acquisition module, every day;

[0131] For these non-real-time state data, embodiments of the present invention still rely on the data processing module to annotate and correct the frames, and output the corrected frame sequence of the non-real-time state data.

[0132] Since the non-real-time state data is uploaded once a day, the above-mentioned corrected frame sequence based on the system prediction can be verified based on the actual system operation data (ground-truth, the correct labeled frame sequence, i.e., the reference labeled frame sequence). During the verification process, if it is found that the labeled data of some frames in the predicted corrected frame sequence are different from those in the reference labeled frame sequence, these frame data are extracted from the predicted corrected frame sequence and used as a supplement to the target keyframe set to improve the reliability of the target keyframe set.

[0133] like Figure 6 As shown, embodiments of the present invention also provide an autonomous driving storage method, applied to the above-mentioned system, comprising:

[0134] Step S1: Obtain the original frame sequence and correct the original frame sequence to obtain the corrected frame sequence;

[0135] Step S2: Obtain the original sample set based on the corrected frame sequence, and obtain the updated distribution result matching each original sample based on the original sample set;

[0136] Step S3: Calculate for each of the updated distribution results, obtain the calculation result, and compare the calculation result with the threshold of the calculation result; if the calculation result is greater than the threshold of the calculation result, then the original sample corresponding to the calculation result is determined as the target keyframe.

[0137] In an optional embodiment of the present invention, the step of obtaining the original frame sequence and correcting the original frame sequence to obtain a corrected frame sequence includes:

[0138] Each of the original frames is labeled to obtain a labeled frame sequence;

[0139] The labeled frame sequence is examined to obtain the examination results;

[0140] Based on the test results, the labeled frame sequence is classified to obtain a first labeled frame sequence, a second labeled frame sequence, and a third labeled frame sequence.

[0141] In an optional embodiment of the present invention, obtaining the first corrected frame sequence includes:

[0142] The first labeled frame sequence is obtained, and the first labeled frame sequence is corrected to obtain the first corrected frame sequence.

[0143] In an optional embodiment of the present invention, obtaining the second corrected frame sequence includes: obtaining the second labeled frame sequence and performing secondary labeling on the second labeled sequence to obtain the second corrected frame sequence.

[0144] In an optional embodiment of the present invention, the correction frame sequence includes a first correction frame sequence, a second correction frame sequence, and a third annotation frame sequence; wherein the first correction frame sequence, the second correction frame sequence, and the third annotation frame sequence are arranged based on time sequence.

[0145] In an optional embodiment of the present invention, obtaining the original sample set based on the corrected frame sequence, and obtaining the updated distribution result matching each of the original samples based on the original sample set, includes:

[0146] The original sample set distribution result is obtained based on the correction sequence; wherein, the original sample set includes N original samples; wherein, N≥1, and N is an integer;

[0147] The Kth original sample is used as the Kth parameter, and the remaining N-1 original samples in the original sample set are used as the Kth reference sample set;

[0148] The Kth original sample is extracted from the original sample set distribution result, and the remaining N-1 original samples from the original sample set distribution result are determined as the Kth reference sample set distribution result M. K Where 1 ≤ K ≤ N, and K is an integer;

[0149] Traverse the Kth reference sample set to generate the Kth updated distribution result L. K .

[0150] In an optional embodiment of the present invention, the step of calculating each of the updated distribution results to obtain a calculation result, and comparing the calculation result with a threshold of the calculation result; if the calculation result is greater than the threshold of the calculation result, then the original sample corresponding to the calculation result is determined as the target keyframe, including:

[0151] Obtain the Kth update distribution result L K and the distribution result M of the Kth reference sample set K And for the Kth update distribution result L K and the distribution result M of the Kth reference sample set K Perform the calculation to obtain the Kth result;

[0152] The Kth calculation result is compared with the threshold of the calculation result;

[0153] If the Kth calculation result is greater than the threshold of the calculation result, then the Kth original sample is determined as the Kth target keyframe.

[0154] Embodiments of the present invention also provide an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the system as described above.

[0155] Embodiments of the present invention also provide a computer-readable storage medium comprising a stored computer program, wherein the computer program, when executed, controls the device containing the computer-readable storage medium to execute the system as described above.

[0156] Furthermore, other configurations and functions of the apparatus in the embodiments of the present invention are known to those skilled in the art, and will not be described in detail here to reduce redundancy.

[0157] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0158] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0159] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0160] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0161] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0162] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0163] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0164] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. An autonomous driving storage system, characterized in that, include: The data processing module and filtering module for communication connection; The data processing module is used to acquire the original frame sequence and correct the original frame sequence to obtain a corrected frame sequence. The filtering module is used to filter the corrected frame sequence to obtain a target keyframe set; The filtering module includes a distribution unit and a calculation unit connected by communication. The distribution unit is used to obtain an original sample set based on the correction frame sequence, and to obtain an updated distribution result matching each original sample based on the original sample set. The calculation unit is used to calculate each updated distribution result, obtain a calculation result, and compare the calculation result with a threshold of the calculation result. If the calculation result is greater than the threshold of the calculation result, the original sample corresponding to the calculation result is determined as the target keyframe. The distribution unit obtains the original sample set distribution result of the original sample set based on the correction frame sequence; wherein, the original sample set includes N original samples; wherein, N≥1, and N is an integer; The distribution unit uses the Kth original sample as the Kth parameter and the remaining N-1 original samples in the original sample set as the Kth reference sample set. The distribution unit extracts the Kth original sample from the original sample set distribution result, and determines the remaining N-1 original samples from the original sample set distribution result as the Kth reference sample set distribution result. Where 1 ≤ K ≤ N, and K is an integer; The distribution unit traverses the Kth reference sample set to generate the Kth updated distribution result. .

2. The system according to claim 1, characterized in that, The data processing module includes a scene annotation unit and a verification unit; The scene annotation unit is used to annotate each of the original frames to obtain an annotated frame sequence; The inspection unit is used to acquire the labeled frame sequence, inspect the labeled frame sequence, and obtain inspection results. The inspection unit classifies the labeled frame sequence based on the inspection results to obtain a first labeled frame sequence, a second labeled frame sequence, and a third labeled frame sequence.

3. The system according to claim 2, characterized in that, The data processing module further includes a first correction unit; The first correction unit is used to acquire the first labeled frame sequence and correct the first labeled frame sequence to obtain a first corrected frame sequence.

4. The system according to claim 2, characterized in that, The data processing module further includes a second correction unit; The second correction unit is used to acquire the second labeled frame sequence and perform secondary labeling on the second labeled frame sequence to obtain the second corrected frame sequence.

5. The system according to claim 2, characterized in that, The correction frame sequence includes a first correction frame sequence, a second correction frame sequence, and a third annotation frame sequence; wherein the first correction frame sequence, the second correction frame sequence, and the third annotation frame sequence are arranged based on time sequence.

6. The system according to claim 1, characterized in that, The computing unit obtains the Kth updated distribution result. and the distribution results of the Kth reference sample set And update the distribution result for the Kth time. and the distribution results of the Kth reference sample set Perform the calculation to obtain the Kth result; The Kth calculation result is compared with the threshold of the calculation result; If the Kth calculation result is greater than the threshold of the calculation result, then the Kth original sample is determined as the Kth target keyframe.

7. An autonomous driving storage method, characterized in that, include: Obtain the original frame sequence and correct the original frame sequence to obtain the corrected frame sequence; The original sample set is obtained based on the corrected frame sequence, and the updated distribution result matching each original sample is obtained based on the original sample set. For each of the updated distribution results, a calculation is performed to obtain a calculation result, and the calculation result is compared with the threshold of the calculation result; If the calculation result is greater than the threshold of the calculation result, then the original sample corresponding to the calculation result is determined as the target keyframe; The original sample set distribution result is obtained based on the corrected frame sequence; wherein the original sample set includes N original samples; wherein N≥1, and N is an integer; The distribution unit uses the Kth original sample as the Kth parameter and the remaining N-1 original samples in the original sample set as the Kth reference sample set; The distribution unit extracts the Kth original sample from the original sample set distribution result, and determines the remaining N-1 original samples from the original sample set distribution result as the Kth reference sample set distribution result. Where 1 ≤ K ≤ N, and K is an integer; The distribution unit traverses the Kth reference sample set to generate the Kth updated distribution result. .

8. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the system as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the system as described in any one of claims 1 to 6.

Images