Method and device for processing data compatibility in a deep learning algorithm

Abstract

The present application provides a method and device for processing data compatibility in a deep learning algorithm. The processing method includes: obtaining a plurality of pre-processed video data to be trained; the video data to be trained is data related to the field of intelligent driving; for each video data to be trained, performing sample construction processing on the video data to be trained to obtain a plurality of sample video data of the video data to be trained; for each sample video data, performing data enhancement processing on the sample video data to generate a plurality of target sample data with different frequencies including the sample video data; using the plurality of target sample data of the plurality of video data to be trained to perform model training, and obtaining a target processing model for processing video data of different frequencies. In this way, the technical solution of the present application can effectively solve the problem of data incompatibility in the deep learning algorithm, thereby improving the accuracy of the output results.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method and device for processing data compatibility in a deep learning algorithm. Background Art

[0002] Intelligent driving technology provides a large amount of raw data for subsequent algorithms and designs through the data acquisition system. The system mainly includes a collection vehicle and the different sensors installed on it (cameras, radars, etc.). The use of advanced algorithms in the field of computer vision can accurately detect information such as vehicles, pedestrians, lane lines, traffic signs, etc. near the collection vehicle. However, the data collected by the collection vehicle will have a large number of general scenes (such as no surrounding vehicles or surrounding vehicles at a uniform speed). For the simulation system, this type of data is of little significance and cannot effectively test the capabilities of the autonomous driving algorithm. Therefore, it is necessary to use the above information to extract key scenes (such as cutting in, cutting out, overtaking, blind spot occlusion, intersection turn avoidance, etc.) for generalized simulation rendering and then testing.

[0003] In existing technology, sampling extraction methods typically include: 1. Rule-based, such as determining entry and exit scenarios in each frame: The lane of the preceding vehicle in the current frame and the lane of the same vehicle in the previous n frames are used to determine whether the preceding vehicle has changed lanes. Similarly, different scenarios are defined and extracted by manually setting corresponding standard parameters. 2. Based on machine learning / deep learning algorithms, time series features are constructed based on information such as surrounding obstacles and lane markings. Different algorithmic frameworks are then used to train and extract different scenarios in a supervised or unsupervised manner.

[0004] However, rule-based scene extraction technology has the following disadvantages: it requires humans to define specific scenarios and manually set parameters; the parameter settings are not very generalizable and difficult to calibrate, and different restrictions need to be added to consider different situations (for example, errors in the input of the upstream detection algorithm require corresponding conditional judgments).

[0005] The shortcomings of existing scene extraction technologies based on machine learning / deep learning algorithms include: generally using a fixed n seconds of data for training and testing, requiring the training and test data dimensions to be consistent (that is, the data size is (n*m)*f, where m is the frequency of the data and f is the length of the feature). However, in actual situations, m is often inconsistent (due to different acquisition hardware, docking with different data sources, etc.), affecting the algorithm accuracy and even making the algorithm unapplicable. Summary of the Invention

[0006] In view of this, the purpose of this application is to provide a method and device for processing data compatibility in deep learning algorithms, which can effectively solve the problem of data incompatibility in deep learning algorithms, thereby improving the accuracy of output results.

[0007] The present invention provides a method for processing data compatibility in a deep learning algorithm, the method comprising:

[0008] Acquire a plurality of pre-processed video data to be trained; the video data to be trained is data related to the field of intelligent driving;

[0009] For each video data to be trained, performing sample construction processing on the video data to be trained to obtain multiple sample video data of the video data to be trained;

[0010] For each sample video data, performing data enhancement processing on the sample video data to generate a plurality of target sample data with different frequencies including the sample video data;

[0011] Model training is performed using a plurality of target sample data of a plurality of video data to be trained to obtain a target processing model for processing video data of different frequencies.

[0012] Optionally, the preprocessing includes: lane line splicing processing, main vehicle, environment participants, lane lines, traffic light feature construction processing, duplicate frame deletion, missing frame linear interpolation completion processing, and feature normalization processing.

[0013] Optionally, performing data enhancement processing on the sample video data to generate a plurality of target sample data with different frequencies for the sample data includes:

[0014] Performing upsampling processing on the sample video data to generate at least one first sample data having a frequency higher than that of the sample video data;

[0015] Downsampling the sample video data to generate at least one second sample data with a frequency lower than that of the sample video data;

[0016] The first sample data, the second sample data and the sample video data are determined as target sample data.

[0017] Optionally, the method of performing model training using a plurality of target sample data of a plurality of video data to be trained to obtain a target processing model for processing video data of different frequencies includes:

[0018] Setting different weights for the original data and the data generated after data enhancement processing in the plurality of target sample data;

[0019] The target sample data after setting different weights are used to perform supervised model training or unsupervised model training to obtain a target processing model for processing video data of different frequencies.

[0020] Optionally, performing sample construction processing on the video data to be trained to obtain a plurality of sample video data of the video data to be trained includes:

[0021] The video data to be trained is sequentially segmented according to a preset duration to obtain a plurality of sub-videos with the same duration, wherein the plurality of sub-videos are a plurality of sample video data of the video data to be trained.

[0022] Optionally, the multiple video data to be trained are data with the same frequency.

[0023] The present application also provides a device for processing data compatibility in a deep learning algorithm, the device comprising:

[0024] An acquisition module is used to acquire a plurality of pre-processed video data to be trained; the video data to be trained is data related to the field of intelligent driving;

[0025] A sample construction module is used to perform sample construction processing on each video data to be trained, and obtain multiple sample video data of the video data to be trained;

[0026] A data enhancement module is used to perform data enhancement processing on each sample video data to generate a plurality of target sample data with different frequencies including the sample video data;

[0027] The training module is used to perform model training using multiple target sample data of multiple video data to be trained, so as to obtain a target processing model for processing video data of different frequencies.

[0028] Optionally, the processing device also includes a preprocessing module, which is used for: lane line splicing processing, main vehicle, environmental participants, lane line, traffic light feature construction processing, duplicate frame deletion, missing frame linear interpolation completion processing, and feature normalization processing.

[0029] Optionally, when the data enhancement module is used to perform data enhancement processing on the sample video data to generate a plurality of target sample data with different frequencies for the sample data, the data enhancement module is used to:

[0030] Performing upsampling processing on the sample video data to generate at least one first sample data having a frequency higher than that of the sample video data;

[0031] Downsampling the sample video data to generate at least one second sample data with a frequency lower than that of the sample video data;

[0032] The first sample data, the second sample data and the sample video data are determined as target sample data.

[0033] Optionally, when the training module is used to perform model training using a plurality of target sample data of a plurality of video data to be trained to obtain a target processing model for processing video data of different frequencies, the training module is used to:

[0034] Setting different weights for the original data and the data generated after data enhancement processing in the plurality of target sample data;

[0035] The target sample data after setting different weights are used to perform supervised model training or unsupervised model training to obtain a target processing model for processing video data of different frequencies.

[0036] Optionally, when the sample construction module is used to perform sample construction processing on the video data to be trained and obtain a plurality of sample video data of the video data to be trained, the sample construction module is used to:

[0037] The video data to be trained is sequentially segmented according to a preset duration to obtain a plurality of sub-videos with the same duration, wherein the plurality of sub-videos are a plurality of sample video data of the video data to be trained.

[0038] Optionally, the multiple video data to be trained are data with the same frequency.

[0039] An embodiment of the present application also provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the processing method described above are performed.

[0040] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the processing method described above are executed.

[0041] An embodiment of the present application provides a method and device for processing data compatibility in a deep learning algorithm, the processing method comprising: obtaining a plurality of pre-processed video data to be trained; the video data to be trained is data related to the field of intelligent driving; for each video data to be trained, performing sample construction processing on the video data to be trained to obtain a plurality of sample video data of the video data to be trained; for each sample video data, performing data enhancement processing on the sample video data to generate a plurality of target sample data with different frequencies including the sample video data; using the plurality of target sample data of the plurality of video data to be trained to perform model training to obtain a target processing model for processing video data of different frequencies.

[0042] In this way, through the technical solution provided by this application, data sources of different frequencies are adapted during the training and testing stages, thereby enhancing the generalization of the target processing model obtained through training; through data enhancement, upsampling and downsampling processing can be performed, and data inputs of different frequencies can be flexibly simulated, thereby enhancing the accuracy of subsequent algorithms.

[0043] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0045] Figure 1 A flowchart of a method for processing data compatibility in a deep learning algorithm provided in an embodiment of the present application;

[0046] Figure 2 A flowchart of the training and testing process of a processing model provided in this application;

[0047] Figure 3 This is one of the structural schematic diagrams of a data-compatible processing device in a deep learning algorithm provided in an embodiment of the present application;

[0048] Figure 4 This is a second structural diagram of a data-compatible processing device in a deep learning algorithm provided in an embodiment of the present application;

[0049] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for which protection is claimed, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work falls within the scope of protection of the present application.

[0051] Intelligent driving technology provides a large amount of raw data for subsequent algorithms and designs through the data acquisition system. The system mainly includes a collection vehicle and the different sensors installed on it (cameras, radars, etc.). The use of advanced algorithms in the field of computer vision can accurately detect information such as vehicles, pedestrians, lane lines, traffic signs, etc. near the collection vehicle. However, the data collected by the collection vehicle will have a large number of general scenes (such as no surrounding vehicles or surrounding vehicles at a uniform speed). For the simulation system, this type of data is of little significance and cannot effectively test the capabilities of the autonomous driving algorithm. Therefore, it is necessary to use the above information to extract key scenes (such as cutting in, cutting out, overtaking, blind spot occlusion, intersection turn avoidance, etc.) for generalized simulation rendering and then testing.

[0052] In existing technology, sampling extraction methods typically include: 1. Rule-based, such as determining entry and exit scenarios in each frame: The lane of the preceding vehicle in the current frame and the lane of the same vehicle in the previous n frames are used to determine whether the preceding vehicle has changed lanes. Similarly, different scenarios are defined and extracted by manually setting corresponding standard parameters. 2. Based on machine learning / deep learning algorithms, time series features are constructed based on information such as surrounding obstacles and lane markings. Different algorithmic frameworks are then used to train and extract different scenarios in a supervised or unsupervised manner.

[0053] However, rule-based scene extraction technology has the following disadvantages: it requires humans to define specific scenarios and manually set parameters; the parameter settings are not very generalizable and difficult to calibrate, and different restrictions need to be added to consider different situations (for example, errors in the input of the upstream detection algorithm require corresponding conditional judgments).

[0054] The shortcomings of existing scene extraction technologies based on machine learning / deep learning algorithms include: generally using a fixed n seconds of data for training and testing, requiring the training and test data dimensions to be consistent (that is, the data size is (n*m)*f, where m is the frequency of the data and f is the length of the feature). However, in actual situations, m is often inconsistent (due to different acquisition hardware, docking with different data sources, etc.), affecting the algorithm accuracy and even making the algorithm unapplicable.

[0055] Based on this, the embodiments of the present application provide a method and device for processing data compatibility in deep learning algorithms, which can effectively solve the problem of data incompatibility in deep learning algorithms, thereby improving the accuracy of output results.

[0056] See also Figure 1 , Figure 1 This is a flow chart of a method for processing data compatibility in a deep learning algorithm provided in an embodiment of the present application. Figure 1 As shown in , the processing method provided in the embodiment of the present application includes:

[0057] S101: Acquire a plurality of pre-processed video data to be trained.

[0058] Here, the video data to be trained is data related to the field of intelligent driving, and the multiple video data to be trained are data with the same frequency.

[0059] For example, the video data to be trained can be data related to scene recognition. The video data to be trained has the same frequency. The video data to be trained has a length of (n*m)*f, where n is the number of seconds, m is the frequency of the data, and f is the length of the feature. The value of n can be different or the same for different video data to be trained, and the values of m and f are generally the same.

[0060] The video data to be trained includes at least the following information: main vehicle information (position, speed, heading angle, etc.), environment vehicle information (position, speed, etc.), lane line information (position or fitted curve), and traffic representation information (category, position, etc.).

[0061] The video data to be trained may be labeled data, which is used for supervised training; the video data to be trained may also be unlabeled data, which is used for unsupervised training.

[0062] For example, when the video data to be trained is relevant data for scene recognition, and the video data to be trained is labeled data, the labels of the video data to be trained may be multiple labels, and the multiple labels include: scene sequence number (eventID), start frame number (startFrame), end frame number (endFrame), start time (startTime), end time (endTime), scene description (description), etc.

[0063] In one embodiment, preprocessing the video data to be trained is a process of cleaning and integrating the video data to be trained. The specific preprocessing content includes at least the following: lane line splicing processing; main vehicle, environmental participants, lane line, and traffic light feature construction processing; duplicate frame deletion and missing frame linear interpolation completion processing; and feature normalization processing.

[0064] S102: For each video data to be trained, perform sample construction processing on the video data to be trained to obtain multiple sample video data of the video data to be trained.

[0065] Here, the sample video data is data with the same time, frequency and feature length.

[0066] The sample construction process for the video data to be trained is to convert the video data to be trained into a time series.

[0067] In one embodiment, the sample construction processing of the video data to be trained to obtain multiple sample video data of the video data to be trained includes: sequentially dividing the video data to be trained according to a preset duration to obtain multiple sub-videos with the same duration, wherein the multiple sub-videos are multiple sample video data of the video data to be trained.

[0068] As an example, the sample construction process is explained through the following content. Assuming that the video data to be trained is 10s long and the preset duration is 2s, after the sample construction processing of the video data to be trained, the video data to be trained is converted into a time series {x_1, x_2, x_3, x_4, x_5}, that is, 5 sub-videos are obtained, where the dimension size of x_i is (a*m)*f, where a is 2 seconds, m is the frequency of the data, f is the length of the feature, and x_i[u,v] represents the value of the vth feature of the uth frame.

[0069] S103 : For each sample video data, perform data enhancement processing on the sample video data to generate a plurality of target sample data with different frequencies including the sample video data.

[0070] In one embodiment provided in the present application, the data enhancement processing is performed on the sample video data to generate multiple target sample data with different frequencies for the sample data, including: upsampling the sample video data to generate at least one first sample data with a frequency higher than the sample video data; downsampling the sample video data to generate at least one second sample data with a frequency lower than the sample video data; and determining the first sample data, the second sample data, and the sample video data as target sample data.

[0071] Here, the data enhancement processing of the sample video data is actually upsampling and downsampling of the sample video data. The methods adopted include linear interpolation and polynomial interpolation, etc. In this way, the data of different frequencies are simulated by upsampling and downsampling to input into the model training module to improve the generalization of the model.

[0072] For example, assuming that the sample video data is 60 frames per second for 2 seconds, after data enhancement processing, multiple data of 10 frames, 20 frames...100 frames per second for 2 seconds can be obtained, that is, multiple target sample data with different frequencies can be obtained.

[0073] For example, here is a method for calculating enhanced sample data. Assume that there is sample video data x_i of the m*ath frame (a seconds), which is enhanced to data y_i of the p*ath frame (a seconds). Then the data of the qth frame after enhancement is calculated as follows:

[0074] For feature v, let t1 = [q*m / p], t2 = q*m / p-t1, where [.] indicates rounding.

[0075] If t1=0, y_i[q,v]=x_i[1,v];

[0076] If t2=0, y_i[q,v]=x_i[t1,v];

[0077] the remaining,

[0078] For linear transformation: y_i[q,v]=(1-t2)x_i[t1,v]+t2x_i[t1+1,v];

[0079] For the quadratic transformation: y_i[q,v]=x_i[t1,v]+t2*(x_i[t1,vv] / m)+t2*t2*(x_i[t1+1,v]-x_i[t1,v]-x_i[t1,vv] / m);

[0080] The feature vv is the derivative of feature v with respect to time. For example, the derivative of distance with respect to time is velocity.

[0081] For example, if 60*2 frames (2 seconds) of data x_i are enhanced to 100*2 (frames) of data y_i, the calculation process of the enhanced 152nd frame data is as follows:

[0082] So t1 = [152*60 / 100] = 91, t2 = 152*60 / 100-t1 = 0.2.

[0083] For the characteristic velocity we use linear transformation: y_i[152, velocity] = 0.8*x_i[91, velocity] + 0.2*x_i[92, velocity].

[0084] For the distance feature, we use the quadratic transformation: y_i[152, distance] = x_i[91, distance] + 0.2*(x_i[91, speed] / 60) + 0.2*0.2*(x_i[92, distance] - x_i[91, distance] - x_i[91, speed] / 60).

[0085] The rest is similar, but here we assume that the object moves in a uniformly accelerated straight line within a very short period of time between two frames. This way, data enhancement can be completed.

[0086] S104: Perform model training using a plurality of target sample data of a plurality of video data to be trained, and obtain a target processing model for processing video data of different frequencies.

[0087] Here, the target processing model may be a scene extraction model, a scene recognition model, a driving safety recognition model, or other models that can be used to process video data.

[0088] In one embodiment provided in the present application, the model training is performed using multiple target sample data of multiple video data to be trained to obtain a target processing model for processing video data of different frequencies, including: setting different weights for the original data and the data generated after data enhancement processing in the multiple target sample data; and using the target sample data after setting different weights to perform supervised model training or unsupervised model training to obtain a target processing model for processing video data of different frequencies.

[0089] Here, the original data is sample video data that has not been subjected to data enhancement processing. The purpose of setting different weights for the original data and the data generated after data enhancement processing in the plurality of target sample data is to avoid underfitting of the original data.

[0090] For supervised learning, where samples have known labels, training can be performed using the LSTM (Long Short-Term Memory) framework. For unsupervised learning, where samples have no labels, training can be performed using the LSTM-AutoEnCoder (AutoEncoder Long Short-Term Memory) framework. During training, data with the same frame rate is combined into the same batch for training, while data from different batches may have different frame rates. The loss function is optimized based on the encoding-decoding structure to obtain the target processing model.

[0091] In addition, for unsupervised learning, it is necessary to additionally cluster the output sample points of the target processing model through the K-Means++ (k-means++) clustering model to output categories.

[0092] For examples, see Figure 2 , Figure 2 This is a flowchart of the training and testing process of a processing model provided in this application. Figure 2 As shown, the dotted line is the testing phase and the solid line is the training phase.

[0093] An embodiment of the present application provides a method for processing data compatibility in a deep learning algorithm, the method comprising: obtaining a plurality of pre-processed video data to be trained; the video data to be trained is data related to the field of intelligent driving; for each video data to be trained, performing sample construction processing on the video data to be trained to obtain a plurality of sample video data of the video data to be trained; for each sample video data, performing data enhancement processing on the sample video data to generate a plurality of target sample data with different frequencies including the sample video data; and performing model training using the plurality of target sample data of the plurality of video data to be trained to obtain a target processing model for processing video data of different frequencies.

[0094] In this way, through the technical solution provided by this application, data sources of different frequencies are adapted during the training and testing stages, thereby enhancing the generalization of the target processing model obtained through training; through data enhancement, upsampling and downsampling processing can be performed, and data inputs of different frequencies can be flexibly simulated, thereby enhancing the accuracy of subsequent algorithms.

[0095] See also Figure 3 、 Figure 4 , Figure 3 This is one of the structural diagrams of a data-compatible processing device in a deep learning algorithm provided in an embodiment of the present application. Figure 4 This is a second structural diagram of a data compatible processing device in a deep learning algorithm provided in an embodiment of the present application. Figure 3 As shown in , the processing device 300 includes:

[0096] An acquisition module 310 is configured to acquire a plurality of pre-processed video data to be trained; the video data to be trained is data related to the field of intelligent driving;

[0097] The sample construction module 320 is used to perform sample construction processing on each video data to be trained, and obtain multiple sample video data of the video data to be trained;

[0098] The data enhancement module 330 is configured to perform data enhancement processing on each sample video data to generate a plurality of target sample data with different frequencies including the sample video data;

[0099] The training module 340 is used to perform model training using a plurality of target sample data of a plurality of video data to be trained, so as to obtain a target processing model for processing video data of different frequencies.

[0100] Optional, such as Figure 4 As shown, the processing device 300 also includes a preprocessing module 350, which is used for: lane line splicing processing, main vehicle, environment participants, lane line, traffic light feature construction processing, duplicate frame deletion, missing frame linear interpolation completion processing, and feature normalization processing.

[0101] Optionally, when the data enhancement module 330 is used to perform data enhancement processing on the sample video data to generate a plurality of target sample data with different frequencies for the sample data, the data enhancement module 330 is used to:

[0102] Performing upsampling processing on the sample video data to generate at least one first sample data having a frequency higher than that of the sample video data;

[0103] Downsampling the sample video data to generate at least one second sample data with a frequency lower than that of the sample video data;

[0104] The first sample data, the second sample data and the sample video data are determined as target sample data.

[0105] Optionally, when the training module 340 is used to perform model training using a plurality of target sample data of a plurality of video data to be trained to obtain a target processing model for processing video data of different frequencies, the training module 340 is used to:

[0106] Setting different weights for the original data and the data generated after data enhancement processing in the plurality of target sample data;

[0107] The target sample data after setting different weights are used to perform supervised model training or unsupervised model training to obtain a target processing model for processing video data of different frequencies.

[0108] Optionally, when the sample construction module is used to perform sample construction processing on the video data to be trained and obtain a plurality of sample video data of the video data to be trained, the sample construction module is used to:

[0109] The video data to be trained is sequentially segmented according to a preset duration to obtain a plurality of sub-videos with the same duration, wherein the plurality of sub-videos are a plurality of sample video data of the video data to be trained.

[0110] Optionally, the multiple video data to be trained are data with the same frequency.

[0111] See also Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown in FIG, the electronic device 500 includes a processor 510, a memory 520 and a bus 530.

[0112] The memory 520 stores machine-readable instructions executable by the processor 510. When the electronic device 500 is running, the processor 510 communicates with the memory 520 via the bus 530. When the machine-readable instructions are executed by the processor 510, the above-mentioned Figure 1 as well as Figure 2 The specific implementation of the steps of the method in the method embodiment shown can be found in the method embodiment, and will not be repeated here.

[0113] The embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 as well as Figure 2 The specific implementation of the steps of the method in the method embodiment shown can be found in the method embodiment, and will not be repeated here.

[0114] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0115] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. There may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interface, indirect coupling or communication connection of devices or units, which may be electrical, mechanical or other forms.

[0116] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0117] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0118] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0119] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The scope of protection of the present application is not limited thereto. Although the present application has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-mentioned embodiments within the technical scope disclosed in the present application, or perform equivalent replacements for some of the technical features thereof. These modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for processing data compatibility in a deep learning algorithm, characterized in that: The processing method comprises: Acquire a plurality of pre-processed video data to be trained with the same frequency; the video data to be trained is data related to the field of intelligent driving; For each video data to be trained, sample construction processing is performed on the video data to be trained to obtain multiple sample video data with the same time, frequency and feature length for the video data to be trained; For each sample video data, performing data enhancement processing on the sample video data to generate a plurality of target sample data with different frequencies including the sample video data; the target sample data includes at least one first sample data with a higher frequency than the sample video data, at least one second sample data with a lower frequency than the sample video data, and the sample video data; Model training is performed using a plurality of target sample data of a plurality of video data to be trained to obtain a target processing model for processing video data of different frequencies.

2. The processing method according to claim 1, characterized in that The preprocessing includes: lane line splicing processing, main vehicle, environment participants, lane line, traffic light feature construction processing, duplicate frame deletion, missing frame linear interpolation completion processing, and feature normalization processing.

3. The processing method according to claim 1, characterized in that The performing data enhancement processing on the sample video data to generate a plurality of target sample data with different frequencies for the sample data includes: Performing upsampling processing on the sample video data to generate at least one first sample data having a frequency higher than that of the sample video data; Downsampling the sample video data to generate at least one second sample data with a frequency lower than that of the sample video data; The first sample data, the second sample data and the sample video data are determined as target sample data.

4. The processing method according to claim 1, characterized in that The method of performing model training using a plurality of target sample data of a plurality of video data to be trained to obtain a target processing model for processing video data of different frequencies includes: Setting different weights for the original data and the data generated after data enhancement processing in the plurality of target sample data; The target sample data after setting different weights are used to perform supervised model training or unsupervised model training to obtain a target processing model for processing video data of different frequencies.

5. The processing method according to claim 1, characterized in that The step of performing sample construction processing on the video data to be trained to obtain a plurality of sample video data of the video data to be trained includes: The video data to be trained is sequentially segmented according to a preset duration to obtain a plurality of sub-videos with the same duration, wherein the plurality of sub-videos are a plurality of sample video data of the video data to be trained.

6. The processing method according to claim 1, characterized in that The plurality of video data to be trained are data with the same frequency.

7. A data compatibility processing device in a deep learning algorithm, characterized in that: The processing device comprises: An acquisition module is used to acquire a plurality of pre-processed video data to be trained with the same frequency; the video data to be trained is data related to the field of intelligent driving; A sample construction module is used to perform sample construction processing on each video data to be trained, and obtain a plurality of sample video data with the same time, frequency and feature length for the video data to be trained; a data enhancement module configured to perform data enhancement processing on each sample video data to generate a plurality of target sample data having different frequencies including the sample video data; the target sample data including at least one first sample data having a higher frequency than the sample video data, at least one second sample data having a lower frequency than the sample video data, and the sample video data; The training module is used to perform model training using multiple target sample data of multiple video data to be trained, so as to obtain a target processing model for processing video data of different frequencies.

8. The processing device according to claim 7, characterized in that When the data enhancement module is used to perform data enhancement processing on the sample video data to generate a plurality of target sample data with different frequencies for the sample data, the data enhancement module is used to: Performing upsampling processing on the sample video data to generate at least one first sample data having a frequency higher than that of the sample video data; Downsampling the sample video data to generate at least one second sample data with a frequency lower than that of the sample video data; The first sample data, the second sample data and the sample video data are determined as target sample data.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the processing method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the processing method according to any one of claims 1 to 6 are executed.

Images