Data set processing method, device, storage medium and electronic device

Abstract

The present application relates to the field of data processing technology, and specifically to a data set processing method, device and electronic device, the method comprising: obtaining a data set to be processed, the data set to be processed comprising a first type of sample data; randomly selecting at least three first type of sample data of different action categories from the data set to be processed, and splicing the first type of sample data of at least three different action categories into spliced ​​sample data; sampling the spliced ​​sample data to obtain a sampled sample data set; expanding the data set to be processed according to the sampled sample data set to obtain a target data set; the target data set comprises the sampled sample data set and all samples in the data set to be processed. By sampling the spliced ​​sample data, the method can achieve rapid expansion of the data set to be processed with fewer original samples, thereby reducing the cost of sample acquisition, while also ensuring the diversity of data in the target data set.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data set processing method, device, storage medium and electronic device. Background Art

[0002] With the continuous optimization of deep convolutional network models, they have gradually burst out with great energy in the field of action recognition, especially in the field of video surveillance. Real-time perception and early warning can provide strong support for system decision-making. However, the development of deep convolutional neural networks in the field of action recognition is still restricted. The main reason for restricting its development is that the cost of obtaining data sets is too high. In order to train a network model that can perform action recognition, single-action videos are required as data samples, and action categories are used as labels for training. However, in specific scenarios, it is difficult to obtain a large amount of action sample data and mark them. In addition, in the actual detection process, the samples to be tested usually include multiple actions, and the one-hot label encoding method cannot correctly reflect the true action state of the sample. Summary of the invention

[0003] In order to solve the above problems, the present application provides a data set processing method, device, storage medium and electronic device.

[0004] In a first aspect, the present application provides a method for processing a data set, the method comprising:

[0005] Acquire a data set to be processed, wherein the data set to be processed includes first-category sample data, wherein the first-category sample is sample data including a single action;

[0006] Randomly selecting at least three first-category sample data of different action categories from the data set to be processed, and splicing the first-category sample data of the at least three different action categories into spliced ​​sample data;

[0007] Sampling the spliced ​​sample data to obtain a sample data set;

[0008] The to-be-processed data set is expanded according to the sample data set to obtain a target data set; the target data set includes the sample data set and all samples in the to-be-processed data set.

[0009] In the above implementation, at least three sample data of different action categories are randomly selected from the data set to be processed and then spliced ​​into a spliced ​​sample data. After sampling the spliced ​​sample data, the original data set can be quickly expanded to obtain the target data set. By sampling the spliced ​​sample data, the method can use fewer original samples to achieve rapid expansion of the data set to be processed, thereby reducing the cost of sample acquisition, while also ensuring the diversity of data in the target data set.

[0010] According to an embodiment of the present application, optionally, in the above-mentioned data set processing method, obtaining the data set to be processed includes:

[0011] Acquire an original data set, and divide the second type of sample data in the original data set into a plurality of first type samples, to obtain a divided original data set; wherein the second type of sample data is sample data including a plurality of actions;

[0012] Encoding the first category samples according to the action category of the action included in each of the first category samples to obtain a label corresponding to each of the first category samples;

[0013] The data set to be processed is obtained according to the first type of samples and their corresponding labels.

[0014] According to an embodiment of the present application, optionally, in the above-mentioned data set processing method, obtaining an original data set, and dividing the second category sample data in the original data set into a plurality of first category samples to obtain the divided original data set includes:

[0015] Determine the start frame and the end frame of each action in the second type of sample data;

[0016] The second type of sample data is divided into a plurality of first type samples according to the start frame and the end frame.

[0017] According to an embodiment of the present application, optionally, in the above data set processing method, sampling the spliced ​​sample data to obtain a sampled sample data set includes:

[0018] Sampling the spliced ​​sample data to obtain a plurality of sampling samples;

[0019] Encode each sample to obtain a label of the sample;

[0020] A sampling sample data set is obtained according to the multiple sampling samples and the codes.

[0021] According to an embodiment of the present application, optionally, in the above data set processing method, sampling the spliced ​​sample data includes:

[0022] Get the sampling length;

[0023] Determine the starting sampling point and the ending sampling point of the spliced ​​sample data;

[0024] The spliced ​​sample data is subjected to sliding sampling according to the sampling length, the starting sampling point and the ending sampling point.

[0025] According to an embodiment of the present application, optionally, in the above data set processing method, obtaining the sampling length includes:

[0026] Determine a head segment, a middle segment and a tail segment according to the first type of sample data of three different action categories in the spliced ​​sample data, wherein the head segment, the middle segment and the tail segment correspond to different action categories respectively;

[0027] The sampling length is obtained according to the length of the middle segment.

[0028] According to an embodiment of the present application, optionally, in the above data set processing method,

[0029] Encode each sample to obtain a label of the sample, including:

[0030] Determining a relative offset between a center position of the sample and a center position of the intermediate segment;

[0031] Calculating a first confidence level that the sample belongs to the behavior type included in the middle segment according to the relative offset;

[0032] Determining a second confidence level that the sample belongs to other behavior types according to the first confidence level;

[0033] The sampled sample is encoded according to the first confidence level and the second confidence level to obtain a label corresponding to the sampled sample.

[0034] In a second aspect, the present application provides a data set processing device, the device comprising:

[0035] A module for acquiring a data set to be processed, used for acquiring a data set to be processed, wherein the data set to be processed includes first-type sample data, wherein the first-type sample is sample data including a single action;

[0036] A sample splicing module, used for randomly selecting at least three first-category sample data of different action categories from the data set to be processed, and splicing the first-category sample data of the at least three different action categories into spliced ​​sample data;

[0037] A sampling module, used for sampling the spliced ​​sample data to obtain a sample data set;

[0038] The data expansion module is used to expand the data set to be processed according to the sample data set to obtain a target data set; the target data set includes all samples in the sample data set and the data set to be processed.

[0039] According to an embodiment of the present application, optionally, in the above-mentioned data set processing device, the module for obtaining the data set to be processed includes:

[0040] A segmentation unit, used to obtain an original data set, and segment the second type of sample data in the original data set into a plurality of first type samples, to obtain a segmented original data set; wherein the second type of sample data is sample data including a plurality of actions;

[0041] an encoding unit, configured to encode the first category samples according to an action category of the actions included in each of the first category samples, to obtain a label corresponding to each of the first category samples;

[0042] A preprocessing unit is used to obtain the to-be-processed data set according to the first type of samples and their corresponding labels.

[0043] According to an embodiment of the present application, optionally, in the above-mentioned data set processing device, the segmentation unit includes:

[0044] a frame determination unit, used to determine a start frame and an end frame of each action in the second type of sample data;

[0045] The sample segmentation unit is used to segment the second type of sample data into a plurality of first type samples according to the start frame and the end frame.

[0046] According to an embodiment of the present application, optionally, in the above-mentioned data set processing device, the sampling module includes:

[0047] A sampling sample acquisition unit, used for sampling the spliced ​​sample data to obtain a plurality of sampling samples;

[0048] A sampling encoding unit, used for encoding each sample to obtain a label of the sample;

[0049] The sampling sample data set acquisition unit is used to acquire the sampling sample data set according to the multiple sampling samples and codes.

[0050] According to an embodiment of the present application, optionally, in the above-mentioned data set processing device, the sampling sample acquisition unit includes:

[0051] A sampling length obtaining unit, used for obtaining a sampling length;

[0052] A sampling point determination unit, used to determine the starting sampling point and the ending sampling point of the spliced ​​sample data;

[0053] A sampling unit is used to sample the spliced ​​sample data according to the sampling length, the starting sampling point and the ending sampling point.

[0054] According to an embodiment of the present application, optionally, in the above-mentioned data set processing device, the sampling length acquisition unit includes:

[0055] A segment determination subunit, used to determine a head segment, a middle segment and a tail segment according to the first type of sample data of three different action categories in the spliced ​​sample data, wherein the head segment, the middle segment and the tail segment correspond to different action categories respectively;

[0056] The sliding window determination subunit is used to obtain the sampling length according to the length of the middle segment.

[0057] According to an embodiment of the present application, optionally, in the above-mentioned data set processing device, the sampling encoding unit includes:

[0058] a relative offset determination unit, configured to determine a relative offset between a center position of the sample and a center position of the middle segment;

[0059] A first confidence determination unit, configured to calculate, based on the relative offset, a first confidence that the sample belongs to a behavior type included in the middle segment;

[0060] a second confidence determination unit, configured to determine, based on the first confidence, a second confidence that the sample belongs to another behavior type;

[0061] A label acquisition unit is used to encode the sample according to the first confidence level and the second confidence level to obtain a label corresponding to the sample.

[0062] In a third aspect, the present application provides a storage medium, which stores a computer program that can be executed by one or more processors and can be used to implement the data set processing method as described above.

[0063] In a fourth aspect, the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the above-mentioned data set processing method is performed.

[0064] Compared with the prior art, one or more embodiments of the above scheme may have the following advantages or beneficial effects:

[0065] The present application provides a data set processing method, device, storage medium and electronic device, the method comprising: obtaining a data set to be processed, the data set to be processed comprising a first type of sample data, wherein the first type of sample is sample data including a single action; randomly selecting at least three first type of sample data of different action categories from the data set to be processed, and splicing the first type of sample data of at least three different action categories into spliced ​​sample data; sampling the spliced ​​sample data to obtain a sampled sample data set; expanding the data set to be processed according to the sampled sample data set to obtain a target data set; the target data set includes the sampled sample data set and all samples in the data set to be processed. After randomly selecting at least three sample data of different action categories from the data set to be processed, splicing them into a spliced ​​sample data. After sampling the spliced ​​sample data, the original data set can be quickly expanded to obtain the target data set. By sampling the spliced ​​sample data, the method can use fewer original samples to achieve rapid expansion of the data set to be processed, thereby reducing the cost of sample acquisition, while also ensuring the diversity of data in the target data set. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The present application will be described in more detail below based on embodiments and with reference to the accompanying drawings.

[0067] Figure 1 A flowchart of a data set processing method provided in Example 1 of the present application.

[0068] Figure 2 A schematic diagram of sliding sampling provided in Example 3 of the present application.

[0069] Figure 3 This is a structural block diagram of a data set processing device provided in Example 1 of the present application.

[0070] Figure 4 A connection block diagram of an electronic device provided in Example 2 of the present application.

[0071] In the drawings, the same reference numerals are used for the same components, and the drawings are not drawn to scale. DETAILED DESCRIPTION

[0072] The following will describe the implementation methods of the present application in detail with reference to the accompanying drawings and embodiments, so that the implementation process of how the present application applies technical means to solve technical problems and achieve corresponding technical effects can be fully understood and implemented accordingly. The embodiments of the present application and the various features in the embodiments can be combined with each other without conflict, and the technical solutions formed are all within the scope of protection of the present application.

[0073] Embodiment 1

[0074] The present invention provides a data set processing method, please refer to Figure 1 , the method comprises the following steps:

[0075] Step S110: Acquire a data set to be processed, where the data set to be processed includes first type of sample data.

[0076] Among them, the first type of sample is sample data including a single action. When obtaining the data set to be processed, the original data set can be obtained first, and then the original data set is preprocessed to obtain the data set to be processed. The operation of preprocessing the original data set can be carried out in multiple ways or a combination of multiple ways. For example, invalid data in the original data set can be filtered out to obtain an original data set that only includes valid data, wherein invalid data can refer to incomplete data in the data set, and can also refer to data formats that are not supported by this method. Invalid data can be specifically screened according to actual applications. In addition, when preprocessing the original data set, it can also be a process of unifying the format of the data in the original data set, or it can be a process of unifying the form of the data in the original data set.

[0077] Step S120: randomly selecting at least three first-category sample data of different action categories from the data set to be processed, and splicing the first-category sample data of the at least three different action categories into spliced ​​sample data.

[0078] The sample data of at least three different action categories randomly selected from the data set to be processed can be sample data including different action categories, such as, three sample data randomly selected from the data set to be processed are S1, S2 and S3, then the behavior category included in S1 is A, the behavior category included in S2 is B, and the behavior category included in S3 is C. Since the three sample data are randomly selected from the data set to be processed, the randomness of the sample data has been taken into account. Therefore, when the three sample data are spliced ​​into the spliced ​​sample data, they can be spliced ​​randomly or spliced ​​in sequence according to the sampling order. It can be understood that the sample data in the data set to be processed can also be divided into multiple categories according to categories, each of which includes multiple sample data of the same category, and then the multiple categories are sorted in a certain order, and then three categories arranged in order are selected in turn, and a sample data is randomly selected from each category, and finally the selected sample data is randomly spliced. In other words, when selecting sample data from the data set to be processed, it is sufficient to ensure that the action categories of the sample data are diverse.

[0079] Step S130: sampling the spliced ​​sample data to obtain a sample data set.

[0080] When sampling the spliced ​​sample data, a sliding sampling method or a random sampling method can be used. The specific sampling method can be determined according to the actual sampling requirements.

[0081] If the sliding window length of the sliding sampling is n, and the sliding amount of the sliding sampling is 0.05n, the number of sample data that can be obtained theoretically is If random sampling is used for sampling, a sample center can be randomly generated on the spliced ​​sample, and then the sampling range is determined based on the sample center, and random sampling is performed based on the sampling range.

[0082] Step S140: expanding the to-be-processed data set according to the sample data set to obtain a target data set; the target data set includes all samples in the sample data set and the to-be-processed data set.

[0083] Sliding sampling of the spliced ​​sample data can obtain a sampled sample data set, and then the original data set can be quickly expanded based on the sampled sample data set.

[0084] In summary, the present application provides a data set processing method, including: obtaining a data set to be processed, the data set to be processed includes a first type of sample data, wherein the first type of sample is sample data including a single action; randomly selecting at least three first type of sample data of different action categories from the data set to be processed, and splicing the first type of sample data of at least three different action categories into spliced ​​sample data; sampling the spliced ​​sample data to obtain a sampled sample data set; expanding the data set to be processed according to the sampled sample data set to obtain a target data set; the target data set includes the sampled sample data set and all samples in the data set to be processed. After randomly selecting at least three sample data of different action categories from the data set to be processed, splicing them into a spliced ​​sample data. After sampling the spliced ​​sample data, the original data set can be quickly expanded to obtain the target data set. By sampling the spliced ​​sample data, the method can use fewer original samples to achieve rapid expansion of the data set to be processed, thereby reducing the cost of sample acquisition, while also ensuring the diversity of data in the target data set.

[0085] Embodiment 2

[0086] Based on the first embodiment, this embodiment illustrates the method in the first embodiment through a specific implementation case.

[0087] In the above-mentioned data set processing method, in the step of obtaining the original data set and preprocessing the original data set to obtain the data set to be processed, the original data set can be first obtained, and the second-category sample data in the original data set can be divided into multiple first-category samples to obtain the divided original data set; wherein the second-category sample data is sample data including multiple actions. Then, the first-category samples are encoded according to the action category of the action included in each first-category sample to obtain the label corresponding to each first-category sample. Finally, the data set to be processed can be obtained according to the first-category samples and their corresponding labels.

[0088] For example, the original data set includes video data, and the video data includes actions in the process flow. Then the second type of sample data includes multiple different actions and labels in the process flow, while the first type of sample includes a single action in the process flow. The first type of sample data includes the first type of sample and its corresponding label. The action included in the sample can be a manually set action. For example, in the equipment assembly process, a single action can be defined as the installation step of each part. If the installation steps include two steps of removing the fan blade and hitting the nut, a single action can be defined as the action of removing the fan blade, and another single action can be defined as the action of hitting the nut. It can be understood that the sample data included in the original data set can also be data with time series attributes, that is, data whose data labels change dynamically over time. For example, audio data that needs to be recognized by sound, including a continuous audio data of a multi-person conversation, can also be used as the original data set in this method for expansion processing. The sample data included in the original data set can be determined according to the actual application scenario, wherein the first type of sample data can be specifically defined according to the application scenario.

[0089] An original data set is obtained, and the second category sample data in the original data set is divided into multiple first category samples. When the divided original data set is obtained, the starting frame and the ending frame of each action in the second category sample data can be determined first, and then the second category sample data can be divided into multiple first category samples according to the starting frame and the ending frame.

[0090] When determining the start frame and end frame of each preset behavior in the second category of sample data, it can be determined by manual labeling or by image recognition. It is understandable that it can also be determined by image recognition and then verified by manual labeling.

[0091] After the second-category sample data including multiple actions is divided into the first-category sample including a single action, it is encoded according to the type of action included in the first-category sample to obtain the label corresponding to the first-category sample. Specifically, the label corresponding to the second-category sample in the second-category sample data including multiple actions includes the starting frame, the ending frame and the category of the action of each action, so that it can be divided according to the starting frame and the ending frame, and after being divided into the first-category sample, the first-category sample is encoded according to its corresponding action category. For example, the label of the second-category sample data is {[0, 1, A], [2, 4, B], [5, 6, C], [7, 8, D]}, then the action with action category A can be divided into the first first-category sample according to the starting frame 0 and the ending frame 1, and then encoded according to the action category A. Then the action with action category B can be divided into the second first-category sample according to the starting frame 2 and the ending frame 4, and then encoded according to the action category B. The segmentation thereafter is similar, and will not be repeated one by one. It can be understood that the encoding method here can adopt the one-hot label encoding method, or other encoding methods can be adopted. Taking one-hot as an example, if there are a total of 4 action categories [A, B, C, D], then encoding the action of action type A can get [1, 0, 0, 0], encoding the action of action type B can get [0, 1, 0, 0], and so on, the action of action type C is encoded as [0, 0, 1, 0], and the action of action type D is encoded as [0, 0, 0, 1].

[0092] Embodiment 3

[0093] Based on the first embodiment, this embodiment illustrates the method in the first embodiment through a specific implementation case.

[0094] When sampling the spliced ​​sample data to obtain a sample data set, the spliced ​​sample data may be first sampled to obtain a sampled sample, then the sampled sample may be encoded according to the spliced ​​sample data to obtain a label of the sampled sample, and finally the sampled sample data may be obtained according to the sampled sample and the encoding.

[0095] When sampling the spliced ​​sample data, the sampling length may be obtained first, and then the starting sampling point and the ending sampling point of the spliced ​​sample data may be determined; and the spliced ​​sample data may be sampled according to the sampling length, the starting sampling point and the ending sampling point.

[0096] The sampling length can be obtained in the following manner: determining a head segment, a middle segment and a tail segment according to the preprocessed sample data in the spliced ​​sample data, wherein the head segment, the middle segment and the tail segment correspond to different action categories respectively; and obtaining the sampling length according to the length of the middle segment. It can be understood that the sampling length can also be directly set.

[0097] After obtaining the sampled sample, it is necessary to encode the sampled sample according to the spliced ​​sample data to obtain the label of the sampled sample. Determine the relative offset between the center position of the sampled sample and the center position of the middle segment; calculate the first confidence that the sampled sample belongs to the behavior type included in the middle segment according to the relative offset; determine the second confidence that the sampled sample belongs to other behavior types according to the first confidence; encode the sampled sample according to the first confidence and the second confidence to obtain the label corresponding to the sampled sample.

[0098] When sampling the spliced ​​sample data to obtain the sample data set, if the sliding sampling method is used. Figure 2 As shown, the three random segments included in the spliced ​​sample S are Si, Sj, and Sk, and the action categories of the actions included are A, B, and C, respectively. If the spliced ​​sample S is slidingly sampled according to the length of the middle segment Sj. Assuming that the length of the Si video is m, the length of the Sj segment is n, and the length of the Sk segment is o, the length of the S segment is m+n+o. The starting point of the sliding sampling center is m, the length is n, and the sliding amount is 0.05n. The starting frame and the end frame of the sampled kth video segment are [0.05n(k-1)+m-0.5n, 0.05n(k-1)+m+0.5n]. Slide step by step in this way, and cut the video segment as the starting and ending points with the starting frame and the end frame as the sample and save it until 0.05n(k-1)+m+0.5n>m+n, that is, the sampling center reaches the end frame of the segment Sj, thereby obtaining a new set of sampled samples. The starting sampling point is the starting point m of the sliding sampling center, that is, the starting frame of the segment Sj, and the ending sampling point is the ending frame of the segment Sj.

[0099] Theoretically, the closer the center of the sample is to the center of Sj, the higher the confidence that the sample belongs to the action category B should be. However, when the center of the sample shifts to both sides, the confidence should decrease accordingly. Gaussian distribution is a mathematical model that conforms to this scenario. The larger the output value, the closer it is to the center. The label can be processed by Gaussian smoothing. The closer to the action center, the higher the confidence label can be. Here, the standard normal distribution curve is used to smooth x~(0,1). Assuming that the new sample is T, x is the relative offset between the center of T and the center of sample Sj, then:

[0100]

[0101]

[0102]

[0103] Among them, T center is the position of the center of the sample T in the video S, Sj center is the position of the center of sample Sj in video S, conf is the confidence label that the sample T belongs to action B, conf other is the confidence that the sample T belongs to each other action category, and N is the total number of action categories. For example, the confidence of the sample at the sliding start position (x = -0.5) belonging to category B is:

[0104]

[0105] The confidence of other actions to which this sample belongs is (assuming there are 4 types of actions in total):

[0106]

[0107] After encoding the label, we can get the vector [0.22, 0.34, 0.22, 0.22], where the sum of the numbers in the encoded vector must be 1, because it represents the confidence of each action, and the sum of the probabilities of each event is 1. The confidence of other actions to which the sample belongs is an approximate number 0.22, 0.22*3=0.66, so the confidence of the category B to which the sample belongs, 0.35, needs to be approximated to 0.34 to ensure that the sum is 1.

[0108] If a random sampling method is used when sampling the spliced ​​sample data to obtain a sample data set, that is, a sample center T is randomly generated on the segment Sj. center , then the starting frame and the ending frame of the sample are [T center -0.5n,T center +0.5n], the method for obtaining the sample label is the same as the sliding sampling embodiment, which will not be repeated here.

[0109] Embodiment 4

[0110] Please see Figure 3 The present application provides a data set processing device 300, the device comprising:

[0111] The to-be-processed data set acquisition module 310 is used to acquire the to-be-processed data set, wherein the to-be-processed data set includes first-type sample data, wherein the first-type sample is sample data including a single action;

[0112] A sample splicing module 320 is used to randomly select at least three first-category sample data of different action categories from the data set to be processed, and splice the first-category sample data of the at least three different action categories into spliced ​​sample data;

[0113] A sampling module 330 is used to sample the spliced ​​sample data to obtain a sample data set;

[0114] The data expansion module 340 is used to expand the to-be-processed data set according to the sample data set to obtain a target data set; the target data set includes the sample data set and all samples in the to-be-processed data set.

[0115] According to an embodiment of the present application, optionally, in the above-mentioned data set processing device, the to-be-processed data set acquisition module 310 includes:

[0116] A segmentation unit, used to obtain an original data set, and segment the second type of sample data in the original data set into a plurality of first type samples, to obtain a segmented original data set; wherein the second type of sample data is sample data including a plurality of actions;

[0117] an encoding unit, configured to encode the first category samples according to an action category of the actions included in each of the first category samples, to obtain a label corresponding to each of the first category samples;

[0118] A preprocessing unit is used to obtain the to-be-processed data set according to the first type of samples and their corresponding labels.

[0119] According to an embodiment of the present application, optionally, in the above-mentioned data set processing device, the segmentation unit includes:

[0120] a frame determination unit, used to determine a start frame and an end frame of each action in the second type of sample data;

[0121] The sample segmentation unit is used to segment the second type of sample data into a plurality of first type samples according to the start frame and the end frame.

[0122] According to an embodiment of the present application, optionally, in the above data set processing device, the sampling module 330 includes:

[0123] A sampling sample acquisition unit, used for sampling the spliced ​​sample data to obtain a plurality of sampling samples;

[0124] A sampling encoding unit, used for encoding each sample to obtain a label of the sample;

[0125] The sampling sample data set acquisition unit is used to acquire the sampling sample data set according to the multiple sampling samples and codes.

[0126] According to an embodiment of the present application, optionally, in the above-mentioned data set processing device, the sampling sample acquisition unit includes:

[0127] A sampling length obtaining unit, used for obtaining a sampling length;

[0128] A sampling point determination unit, used to determine the starting sampling point and the ending sampling point of the spliced ​​sample data;

[0129] A sampling unit is used to sample the spliced ​​sample data according to the sampling length, the starting sampling point and the ending sampling point.

[0130] According to an embodiment of the present application, optionally, in the above-mentioned data set processing device, the sampling length acquisition unit includes:

[0131] A segment determination subunit, used to determine a head segment, a middle segment and a tail segment according to the first type of sample data of three different action categories in the spliced ​​sample data, wherein the head segment, the middle segment and the tail segment correspond to different action categories respectively;

[0132] The sliding window determination subunit is used to obtain the sampling length according to the length of the middle segment.

[0133] According to an embodiment of the present application, optionally, in the above-mentioned data set processing device, the sampling encoding unit includes:

[0134] a relative offset determination unit, configured to determine a relative offset between a center position of the sample and a center position of the middle segment;

[0135] A first confidence determination unit, configured to calculate, based on the relative offset, a first confidence that the sample belongs to a behavior type included in the middle segment;

[0136] a second confidence determination unit, configured to determine, based on the first confidence, a second confidence that the sample belongs to another behavior type;

[0137] A label acquisition unit is used to encode the sample according to the first confidence level and the second confidence level to obtain a label corresponding to the sample.

[0138] In summary, the present application provides a data set processing device, including: a data set acquisition module 310 for acquiring a data set to be processed, the data set to be processed includes a first type of sample data, wherein the first type of sample is sample data including a single action; a sample splicing module 320, for randomly selecting at least three first type of sample data of different action categories from the data set to be processed, and splicing the first type of sample data of at least three different action categories into spliced ​​sample data; a sampling module 330, for sampling the spliced ​​sample data to obtain a sampled sample data set; a data expansion module 340, for expanding the data set to be processed according to the sampled sample data set to obtain a target data set; the target data set includes the sampled sample data set and all samples in the data set to be processed. After randomly selecting at least three sample data of different action categories from the data set to be processed, they are spliced ​​into a spliced ​​sample data. After sampling the spliced ​​sample data, the original data set can be quickly expanded to obtain the target data set. By sampling the spliced ​​sample data, this method can use fewer original samples to quickly expand the data set to be processed, thereby reducing the cost of sample acquisition while ensuring the diversity of data in the target data set.

[0139] Embodiment 5

[0140] This embodiment also provides a computer-readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a disk, an optical disk, a server, an App application store, etc., on which a computer program is stored. When the computer program is executed by a processor, the method steps described in the above embodiment can be implemented.

[0141] The specific implementation process of the above method steps can be found in Example 1, and this embodiment will not be repeated here.

[0142] Embodiment 6

[0143] The present application embodiment provides an electronic device, which may be a mobile phone, a computer or a tablet computer, etc., including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the data set processing method described in the first embodiment is implemented. It can be understood that Figure 4 As shown, the electronic device 400 may further include: a processor 401 , a memory 402 , a multimedia component 403 , an input / output (I / O) interface 404 , and a communication component 405 .

[0144] The processor 401 is used to execute all or part of the steps in the data set processing method in Embodiment 1. The memory 402 is used to store various types of data, which may include instructions of any application or method in the electronic device, and data related to the application.

[0145] The processor 401 can be an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components, and is used to execute the data set processing method in the above-mentioned embodiment 1.

[0146] The memory 402 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, disk or optical disk.

[0147] The multimedia component 403 may include a screen and an audio component, the screen may be a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone, and the microphone is used to receive external audio signals. The received audio signal may be further stored in a memory or sent via a communication component. The audio component also includes at least one speaker for outputting audio signals.

[0148] The I / O interface 404 provides an interface between the processor 401 and other interface modules, which may be a keyboard, a mouse, buttons, etc. These buttons may be virtual buttons or physical buttons.

[0149] The communication component 405 is used for wired or wireless communication between the electronic device 400 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more of them, so the corresponding communication component 405 can include: Wi-Fi module, Bluetooth module, NFC module.

[0150] In summary, the present application provides a data set processing method, device, storage medium and electronic device, the method comprising: obtaining a data set to be processed, the data set to be processed comprising a first type of sample data, wherein the first type of sample is sample data including a single action; randomly selecting at least three first type of sample data of different action categories from the data set to be processed, and splicing the first type of sample data of the at least three different action categories into spliced ​​sample data; sampling the spliced ​​sample data to obtain a sampled sample data set; expanding the data set to be processed according to the sampled sample data set to obtain a target data set; the target data set includes the sampled sample data set and all samples in the data set to be processed. After randomly selecting at least three sample data of different action categories from the data set to be processed, splicing them into a spliced ​​sample data. After sampling the spliced ​​sample data, the original data set can be quickly expanded to obtain the target data set. By sampling the spliced ​​sample data, the method can use fewer original samples to achieve rapid expansion of the data set to be processed, thereby reducing the cost of sample acquisition, while also ensuring the diversity of data in the target data set.

[0151] In the several embodiments provided in the present application, it should be understood that the disclosed system and method can also be implemented in other ways. The above described system and method embodiments are merely illustrative.

[0152] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0153] Although the implementation methods disclosed in this application are as above, the contents described are only implementation methods adopted for facilitating the understanding of this application, and are not intended to limit this application. Any technician in the technical field to which this application belongs can make any modifications and changes in the form and details of implementation without departing from the spirit and scope disclosed in this application, but the scope of patent protection of this application shall still be based on the scope defined in the attached claims.

Claims

1. A data set processing method, characterized in that: The method comprises: Acquire a data set to be processed, wherein the data set to be processed includes first-category sample data, wherein the first-category sample is sample data including a single action; Randomly selecting at least three first-category sample data of different action categories from the data set to be processed, and splicing the first-category sample data of the at least three different action categories into spliced ​​sample data; Sampling the spliced ​​sample data to obtain a sample data set; Expanding the to-be-processed data set according to the sample data set to obtain a target data set; the target data set includes all samples in the sample data set and the to-be-processed data set; The step of sampling the spliced ​​sample data to obtain a sample data set includes: Determine a head segment, a middle segment and a tail segment according to the first type of sample data of three different action categories in the spliced ​​sample data, wherein the head segment, the middle segment and the tail segment correspond to different action categories respectively; Acquire the sampling length according to the length of the middle segment; Determine that the starting sampling point is the starting frame of the middle segment, and the ending sampling point is the ending frame of the middle segment, and sample the spliced ​​sample data according to the sampling length, the starting sampling point, and the ending sampling point to obtain a plurality of sampling samples; Encode each sample to obtain a label of the sample; Acquire a sample data set according to the multiple sample samples and codes; Encode each sample to obtain a label of the sample, including: Determining a relative offset between a center position of the sample and a center position of the intermediate segment; Calculating a first confidence level that the sample belongs to the behavior type included in the middle segment according to the relative offset; Determining a second confidence level that the sample belongs to other behavior types according to the first confidence level; Encoding the sample according to the first confidence level and the second confidence level to obtain a label corresponding to the sample; The Gaussian smoothing method is used to calculate the first confidence level and the second confidence level using the following formula: The standard normal distribution curve is used for smoothing x~(0,1), the sample is T, x is the relative offset between the center of T and the center of sample Sj, then: Among them, T center is the position of the center of the sample T in the spliced ​​sample S, Sj center is the position of the center of sample Sj in the spliced ​​sample S, conf B is the confidence that the sample T belongs to the behavior type B included in the middle segment, that is, the first confidence, conf other is the confidence that the sample T belongs to each other action category, that is, the second confidence, and N is the total number of action categories of the action.

2. The method according to claim 1, characterized in that Get the data set to be processed, including: Acquire an original data set, and divide the second type of sample data in the original data set into a plurality of first type samples, to obtain a divided original data set; wherein the second type of sample data is sample data including a plurality of actions; Encoding the first category samples according to the action category of the action included in each of the first category samples to obtain a label corresponding to each of the first category samples; The data set to be processed is obtained according to the first type of samples and their corresponding labels.

3. The method according to claim 2, characterized in that An original data set is obtained, and the second type of sample data in the original data set is divided into a plurality of first type samples to obtain a divided original data set, including: Determine the start frame and the end frame of each action in the second type of sample data; The second type of sample data is divided into a plurality of first type samples according to the start frame and the end frame.

4. The method according to claim 2, characterized in that: Before the step of obtaining an original data set and dividing the second type of sample data in the original data set into a plurality of first type samples to obtain the divided original data set, the step further includes: The original data set is preprocessed, wherein the preprocessing includes at least one of filtering invalid data in the original data set and unifying the data format in the original data set.

5. The method according to claim 3, characterized in that: The step of determining the start frame and the end frame of each action in the second type of sample data comprises: A random sampling method is used to randomly generate a sample center T on the segment Sj center ; The starting frame and ending frame of the sample are determined as [T center -0.5n,T center +0.5n].

6. The method according to claim 1, characterized in that In the step of encoding the sample according to the first confidence level and the second confidence level to obtain a label corresponding to the sample, the sample is encoded using a one-hot label encoding method.

7. A data set processing device for implementing the data set processing method according to any one of claims 1 to 6, characterized in that: The device comprises: A module for acquiring a data set to be processed, used for acquiring a data set to be processed, wherein the data set to be processed includes first-type sample data, wherein the first-type sample is sample data including a single action; A sample splicing module, used for randomly selecting at least three first-category sample data of different action categories from the data set to be processed, and splicing the first-category sample data of the at least three different action categories into spliced ​​sample data; A sampling module, used for sampling the spliced ​​sample data to obtain a sample data set; The data expansion module is used to expand the data set to be processed according to the sample data set to obtain a target data set; the target data set includes all samples in the sample data set and the data set to be processed.

8. A storage medium, characterized in that: The computer program stored in the storage medium, when executed by one or more processors, is used to implement the data set processing method as described in any one of claims 1 to 6.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the method for processing a data set according to any one of claims 1 to 6 is executed.

Images