Method, device and storage medium for determining training sample data ratio

Abstract

This invention provides a method, apparatus, and storage medium for determining the allocation of training sample data, relating to the field of training sample classification allocation. The method for determining the allocation of training sample data includes: acquiring multiple training sample data sets; classifying the sample data according to data categories to generate multiple sets of category data, and calculating a first complexity for each set of category data; extracting training samples according to tasks to generate multiple sets of task data; calculating a second complexity for each set of task data; and determining the amount of sample data required for each task based on the first complexity, the second complexity, and the number of category data sets corresponding to the calculation of the first complexity, thereby determining the data allocation ratio among multiple tasks. This invention, by reasonably allocating the amount of training sample data, makes the training sample data more focused on complex tasks, while avoiding the model overfitting problem caused by repeated sampling, effectively improving the recognition accuracy of the training model.

Description

Technical Field

[0001] The present invention relates to the technical field of training sample classification and matching, and in particular to a method, device and storage medium for determining training sample data matching. Background Art

[0002] The multi-category and multi-task training method based on deep learning has been widely used in various fields. When screening data, the sampling ratio of each category and task data will have a great impact on the final model result. For example, if the proportion of a certain category or task data is too large, the accuracy of other tasks and categories will be too low.

[0003] At present, when sampling and matching the amount of data for each category and task, some methods ensure the accuracy of the final training model by ensuring that the amount of training data for each category and task is the same during data sampling. However, in this implementation, when the number of certain categories is too small or there are a large number of similar pictures, repeated sampling will result in overfitting of the model. In addition, different problems have different difficulties, which will also lead to low accuracy for complex tasks.

[0004] Therefore, in order to improve the accuracy of the training model, how to propose a method to determine the ratio of training sample data has become an urgent problem to be solved. Summary of the invention

[0005] The present invention aims to solve or improve the technical problems in the prior art that, when training a model, it is impossible to provide accurate matching values ​​between various training sample data, resulting in too few categories or multiple repeated sampling when there are a large number of similar pictures, which will cause over-fitting of the model and too low recognition accuracy of complex tasks, affecting the accuracy of recognition of the training model.

[0006] The first object of the present invention is to provide a method for determining the ratio of training sample data.

[0007] The second object of the present invention is to provide a device for determining the ratio of training sample data.

[0008] A third object of the present invention is to provide an electronic device.

[0009] A fourth object of the present invention is to provide a storage medium.

[0010] To achieve the above-mentioned purpose, the technical solution of the first aspect of the present invention provides a method for determining the ratio of training sample data, and the confirmation method includes the following steps: obtaining multiple training sample data; classifying the multiple training sample data according to data categories, generating multiple groups of category data, and obtaining the number of each group of category data; based on the similarity between the data, calculating the first complexity of each group of category data, wherein the complexity of the category data and the similarity between the category data are negatively correlated; extracting multiple training sample data according to tasks, generating multiple groups of task data; based on the similarity between the data, calculating the second complexity of each group of task data, wherein the complexity of the task data and the similarity between the task data are negatively correlated; determining the amount of sample data required for each task according to the first complexity, the second complexity and the number of category data corresponding to the first complexity; determining the ratio of the amount of sample data between the multiple tasks according to the amount of sample data of the multiple tasks.

[0011] According to the method for determining the ratio of training sample data provided by the present invention, the acquired training sample data is first classified according to categories. The categories can also be understood as different scenes of the training sample data. Taking the training ship recognition model as an example, the categories can be divided according to the sea scene, the river scene, the shore scene, etc., so that there are multiple training sample data under each group of category data. Then, according to the similarity between the multiple training sample data in each group of category data, the first complexity of each group of category data is calculated, and the number of the first complexity is the same as the number of divided category groups. At the same time, the complexity of the category data is negatively correlated with the similarity between the category data, that is, the higher the similarity between the multiple training sample data in each scene, the lower its complexity. Then, according to the actual task situation, the corresponding task features are extracted from the multiple training sample data. The task can refer to a detection task, a semantic segmentation task, or an online map identification task, etc. According to the similarity between the multiple task features in each group of tasks, the second complexity of each group of task data is calculated, and the number of the second complexity is the same as the number of divided task groups. At the same time, the complexity of task data and the similarity between task data are negatively correlated, that is, the higher the similarity between multiple task features under each group of task data, the lower its complexity. In this way, the sample data volume of each task can be determined by using the first complexity of different categories of data, the second complexity of different task data, and the number of category data corresponding to the calculation of the first complexity, so as to determine the data volume ratio between the tasks. For example, when determining the sample data volume required for the detection task in the offshore scene, it is necessary to calculate the first complexity of the training sample data in the offshore scene, the second complexity of the detection task in all scenes, and the number of samples when calculating the first complexity of the training sample data in the offshore scene, so as to determine the sample data volume of the training sample ratio. The method for determining the training sample data ratio provided in the present application can make the training sample data more biased towards complex tasks by reasonably automatically generating the ratio between the sample data volumes of the training data, while avoiding the model over-fitting and problems caused by repeated sampling when the number of certain categories is too small or there are a large number of similar pictures, thereby effectively improving the recognition accuracy of the training model, while saving a lot of costs for manually adjusting the parameters of the training model and improving the efficiency of model training.

[0012] In addition, the method for determining the training sample data ratio provided by the present invention may also have the following additional technical features:

[0013] In some technical solutions, optionally, based on the similarity between the data, the step of calculating the first complexity of each group of category data includes: converting all data in each group of category data into a first Hamming code; determining the first Hamming distance between any two data in each group of category data according to the first Hamming code; and calculating the first complexity of each group of category data according to multiple first Hamming distances in each group of category data.

[0014] In this technical solution, the Hamming code is a linear adjustment code, and the training sample data can be converted into the Hamming code by the metric learning method. In this application, all the data in each group of category data are converted into the corresponding first Hamming code. The first Hamming code is actually the Hamming code, but it is converted after the data is classified according to the category. There are multiple training sample data in each group of category data. After all the training sample data are converted into their respective Hamming codes, the Hamming distance between any two Hamming codes in each group of category data is determined. The Hamming distance represents the number of 1s in the two Hamming codes AXorB. A and B are two different Hamming codes, and Xor (exclusive OR) is a bitwise exclusive OR. Assuming that there are N training sample data in a group of category data, a (N-1, N-1) Hamming distance matrix is ​​generated, and the mean of all values ​​of the Hamming distance matrix is ​​calculated as the first complexity of the group of category data. This application uses the similarity between the training sample data as the criterion for the complexity of the category data. The higher the similarity, the lower the data complexity, and vice versa. The Hamming distance is used to measure the similarity between any two training sample data in the same category to improve the accuracy of judging the complexity between category data.

[0015] Of course, there are many ways to calculate the similarity between data in each group of category data, which are not listed here one by one.

[0016] In some technical solutions, optionally, the step of extracting multiple training sample data according to tasks to generate multiple groups of task data includes: extracting data corresponding to the tasks from multiple training sample data according to the tasks to generate multiple groups of task data to be processed; and performing unified processing on each group of task data to be processed to generate multiple groups of task data.

[0017] In this technical solution, considering that the task feature formats extracted from different training sample data are not unified, which affects the accuracy of the subsequent calculation of the second complexity of the task data, after the data is extracted according to the task, the data is formatted in a unified manner to ensure the accuracy of the subsequent calculation of the second complexity.

[0018] In some technical schemes, optionally, based on the similarity between the data, the step of calculating the second complexity of each group of task data includes: converting all data in each group of task data into a second Hamming code; determining the second Hamming distance between any two data in each group of task data according to the second Hamming code; and calculating the second complexity of each group of task data according to multiple second Hamming distances in each group of task data.

[0019] In this technical solution, all data in each set of task data are converted into corresponding second Hamming codes. The second Hamming code is actually a Hamming code, but it is converted based on data extracted and classified according to the task. There are multiple task features in each set of task data. After all task features are converted into their respective Hamming codes, the Hamming distance between any two Hamming codes in each set of task data is determined to calculate the second complexity of each set of task data. This application uses the Hamming distance to measure the similarity between any two task features under the same task to improve the accuracy of judging the complexity between task data.

[0020] Of course, there are many ways to calculate the similarity between data in each set of task data, which are not listed here one by one.

[0021] In some technical solutions, optionally, the training sample data includes image data and point cloud data.

[0022] In this technical solution, the image data may include ship image data, specifically image data taken when the ship is traveling, image data taken when the ship is moored at the shore, and image data taken when the ship is operating. Point cloud data may include laser radar point cloud data, specifically laser radar point cloud data scanned by the ship during travel. Laser radar point cloud data is data obtained by scanning by laser radar. Laser radar is an active remote sensing device that uses laser as the emission light source and photoelectric detection technology. Laser radar has the characteristics of high precision, high resolution and real-time performance, thereby ensuring the accuracy of training sample data.

[0023] In some technical solutions, optionally, the first complexity is a i , the second complexity is b j , the number of category data corresponding to the first complexity is d i , the sample data volume is h, where: a i 、b j d i and h satisfy the following relationship:

[0024] h=a i ×b j ×w×d i +C;

[0025] Among them, w is a constant, C is a constant, i is the category data type, and j is the task data type.

[0026] In this technical solution, the amount of sample data required for a task in a certain scenario can be determined by calculating the first complexity, the second complexity, and the number of category data corresponding to the first complexity. For example, the first complexity a1 is the complexity of the offshore scenario, the first complexity a2 is the complexity of the shore scenario, the second complexity b1 is the complexity of the detection task, the second complexity b2 is the complexity of the semantic segmentation task, d1 is the number of data in the offshore scenario, and d2 is the number of data in the shore scenario. Therefore, it can be determined that the amount of sample data required for the detection task in the offshore scenario is a1×b1×w×d1+C, the amount of sample data for the detection task in the shore scenario is a2×b1×w×d2+C, and the amount of sample data for the semantic segmentation task in the shore scenario is a2×b2×w×d2+C.

[0027] In some technical solutions, optionally, the method for determining the ratio of training sample data also includes: obtaining a first training model, obtaining training data for each task according to the ratio of sample data amounts between multiple tasks, inputting the training data for each task into the first training model for training, and generating a second training model.

[0028] In this technical solution, after determining the ratio of sample data amounts among multiple tasks, the training data for each task is obtained according to the ratio and input into the corresponding model for training, thereby obtaining a training model with better recognition accuracy. The model can also ensure that the training sample data is more biased towards complex tasks, while avoiding model over-fitting and problems caused by repeated sampling when the number of certain categories is too small or there are a large number of similar images.

[0029] The second aspect of the present invention provides a device for determining the ratio of training sample data, comprising: an acquisition unit for acquiring multiple training sample data; a processing unit for classifying the multiple training sample data according to data categories, generating multiple groups of category data, and obtaining the number of each group of category data; a calculation unit for calculating the first complexity of each group of category data based on the similarity between the data, wherein the complexity of the category data and the similarity between the category data are negatively correlated; an extraction unit for extracting the multiple training sample data according to the task to generate multiple groups of task data; the calculation unit is also used to calculate the second complexity of each group of task data based on the similarity between the data, wherein the complexity of the task data and the similarity between the task data are negatively correlated; a determination unit for determining the amount of sample data required for each task based on the first complexity, the second complexity and the number of category data corresponding to the calculation of the first complexity; and determining the ratio of the amount of sample data between the multiple tasks based on the amount of sample data of the multiple tasks.

[0030] According to the apparatus for determining the ratio of training sample data provided by the present invention, the acquired training sample data is first classified according to categories, and the categories can also be understood as different scenes of the training sample data. Taking the training ship recognition model as an example, the categories can be divided according to the sea scene, the river scene, the shore scene, etc., so that there are multiple training sample data under each group of category data. Then, according to the similarity between the multiple training sample data in each group of category data, the first complexity of each group of category data is calculated, and the number of the first complexity is the same as the number of divided category groups. At the same time, the complexity of the category data is negatively correlated with the similarity between the category data, that is, the higher the similarity between the multiple training sample data in each scene, the lower its complexity. Then, according to the actual task situation, the corresponding task features are extracted from the multiple training sample data. The task can refer to the detection task, the semantic segmentation task or the online map identification task, etc. According to the similarity between the multiple task features in each group of tasks, the second complexity of each group of task data is calculated, and the number of the second complexity is the same as the number of divided task groups. At the same time, the complexity of task data and the similarity between task data are negatively correlated, that is, the higher the similarity between multiple task features under each group of task data, the lower its complexity. In this way, the sample data volume of each task can be determined by using the first complexity of different categories of data, the second complexity of different task data, and the number of category data corresponding to the calculation of the first complexity, thereby determining the data volume ratio between the tasks. The device for determining the training sample data ratio provided in the present application can make the training sample data more biased towards complex tasks by automatically generating a reasonable ratio between the sample data volumes of training data, while avoiding the model over-fitting and problems caused by repeated sampling when the number of certain categories is too small or there are a large number of similar pictures, thereby effectively improving the recognition accuracy of the training model, while saving a lot of costs for manually adjusting the training model parameters and improving the model training efficiency.

[0031] The technical solution of the third aspect of the present invention provides an electronic device, comprising: a memory storing programs and / or instructions; a processor executing the programs and / or instructions; wherein, when the processor executes the programs and / or instructions, it implements the steps of the method for determining the training sample data ratio in any technical solution of the first aspect.

[0032] According to the electronic device provided by the present invention, since it can implement the steps of the method for determining the training sample data ratio in any technical solution of the first aspect, the electronic device also has all the beneficial effects of the method for determining the training sample data ratio in any technical solution of the first aspect, which will not be repeated here.

[0033] The technical solution of the fourth aspect of the present invention provides a storage medium on which programs and / or instructions are stored. When the programs and / or instructions are executed, the steps of the method for determining the training sample data ratio in any technical solution of the first aspect are implemented.

[0034] Additional aspects and advantages of the present invention will become apparent in the following description or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The above and / or additional aspects and advantages of the embodiments of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0036] Figure 1 One of the flow charts of the method for determining the training sample data ratio provided by an embodiment of the present invention is shown;

[0037] Figure 2 The second flowchart of the method for determining the training sample data ratio provided by the embodiment of the present invention is shown;

[0038] Figure 3 The third flowchart of the method for determining the training sample data ratio provided by the embodiment of the present invention is shown;

[0039] Figure 4 A schematic diagram showing a first complexity of calculating category data provided by an embodiment of the present invention is shown;

[0040] Figure 5 A schematic diagram showing a second complexity of computing task data provided by an embodiment of the present invention;

[0041] Figure 6 A block diagram of a device for determining a training sample data ratio provided by an embodiment of the present invention is shown;

[0042] Figure 7 A block diagram of an electronic device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0043] In order to more clearly understand the above aspects, features and advantages of the embodiments of the present invention, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0044] In the following description, many specific details are set forth to facilitate a full understanding of the embodiments according to the present invention. However, the embodiments according to the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the embodiments according to the present invention is not limited to the specific embodiments disclosed below.

[0045] like Figure 1 As shown, an embodiment of the present invention provides a method for determining a training sample data ratio, comprising the following steps:

[0046] S102: Acquire multiple training sample data;

[0047] S104: Classifying the plurality of training sample data according to data categories, generating a plurality of groups of category data, and obtaining the quantity of each group of category data;

[0048] S106: Based on the similarity between the data, calculate and obtain the first complexity of each group of category data;

[0049] S108: extracting a plurality of training sample data according to the tasks to generate a plurality of sets of task data;

[0050] S110: Based on the similarity between the data, calculate the second complexity of each group of task data;

[0051] S112: Determine the amount of sample data required for each task according to the first complexity, the second complexity, and the amount of category data corresponding to the first complexity;

[0052] S114: Determine a ratio of sample data volumes among the multiple tasks according to the sample data volumes of the multiple tasks.

[0053] According to the method for determining the ratio of training sample data provided by the present invention, the acquired training sample data is first classified according to categories. The categories can also be understood as different scenes of the training sample data. Taking the training ship recognition model as an example, the categories can be divided according to the sea scene, the river scene, the shore scene, etc., so that there are multiple training sample data under each group of category data. Then, according to the similarity between the multiple training sample data in each group of category data, the first complexity of each group of category data is calculated, and the number of the first complexity is the same as the number of divided category groups. At the same time, the complexity of the category data is negatively correlated with the similarity between the category data, that is, the higher the similarity between the multiple training sample data in each scene, the lower its complexity. Then, according to the actual task situation, the corresponding task features are extracted from the multiple training sample data. The task can refer to a detection task, a semantic segmentation task, or an online map identification task, etc. According to the similarity between the multiple task features in each group of tasks, the second complexity of each group of task data is calculated, and the number of the second complexity is the same as the number of divided task groups. At the same time, the complexity of task data and the similarity between task data are negatively correlated, that is, the higher the similarity between multiple task features under each group of task data, the lower its complexity. In this way, the sample data volume of each task can be determined by using the first complexity of different categories of data, the second complexity of different task data, and the number of category data corresponding to the calculation of the first complexity, so as to determine the data volume ratio between the tasks. For example, when determining the sample data volume required for the detection task in the offshore scene, it is necessary to calculate the first complexity of the training sample data in the offshore scene, the second complexity of the detection task in all scenes, and the number of samples when calculating the first complexity of the training sample data in the offshore scene, so as to determine the sample data volume of the training sample ratio. The method for determining the training sample data ratio provided in the present application can make the training sample data more biased towards complex tasks by reasonably automatically generating the ratio between the sample data volumes of the training data, while avoiding the model over-fitting and problems caused by repeated sampling when the number of certain categories is too small or there are a large number of similar pictures, thereby effectively improving the recognition accuracy of the training model, while saving a lot of costs for manually adjusting the parameters of the training model and improving the efficiency of model training.

[0054] In one embodiment of the present invention, Figure 2 As shown, step S106, based on the similarity between the data, calculate the first complexity of each group of category data, including:

[0055] S202: converting all data in each group of category data into a first Hamming code;

[0056] S204: Determine a first Hamming distance between any two data in each group of category data according to the first Hamming code;

[0057] S206: Calculate the first complexity of each group of category data according to multiple first Hamming distances in each group of category data.

[0058] In this embodiment, the Hamming code is a linear adjustment code, and the training sample data can be converted into the Hamming code by the metric learning method. In the present application, all data in each group of category data are converted into the corresponding first Hamming code. The first Hamming code is actually the Hamming code, but it is converted after the data is classified according to the category. There are multiple training sample data in each group of category data. After all the training sample data are converted into their respective Hamming codes, the Hamming distance between any two Hamming codes in each group of category data is determined. The Hamming distance represents the number of 1s in the two Hamming codes AXorB. A and B are two different Hamming codes, and Xor (exclusive OR) is a bitwise exclusive OR. Assuming that there are N training sample data in a group of category data, a (N-1, N-1) Hamming distance matrix is ​​generated, and the mean of all values ​​of the Hamming distance matrix is ​​calculated as the first complexity of the group of category data. In the present application, the similarity between the training sample data is used as the criterion for judging the complexity of the category data. The higher the similarity, the lower the data complexity, and vice versa. The Hamming distance is used to measure the similarity between any two training sample data in the same category to improve the accuracy of judging the complexity between category data.

[0059] Of course, there are many ways to calculate the similarity between data in each group of category data, which are not listed here one by one.

[0060] In some embodiments, optionally, the step of extracting multiple training sample data according to tasks to generate multiple groups of task data includes: extracting data corresponding to the tasks from multiple training sample data according to the tasks to generate multiple groups of task data to be processed; and performing unified processing on each group of task data to be processed to generate multiple groups of task data.

[0061] In this embodiment, considering that the task feature formats extracted from different training sample data are not uniform, which affects the accuracy of the subsequent calculation of the second complexity of the task data, after the data is extracted according to the task, the data is formatted in a unified manner to ensure the accuracy of the subsequent calculation of the second complexity.

[0062] In one embodiment of the present invention, Figure 3 As shown, step S110, based on the similarity between the data, calculates the second complexity of each group of task data, including:

[0063] S302: converting all data in each group of task data into a second Hamming code;

[0064] S304: Determine the second Hamming distance between any two data in each group of task data according to the second Hamming code;

[0065] S306: Calculate the second complexity of each group of task data according to multiple second Hamming distances in each group of task data.

[0066] In this embodiment, all data in each group of task data are converted into corresponding second Hamming codes. The second Hamming code is actually a Hamming code, but the data is extracted and classified according to the task. There are multiple task features in each group of task data. After all task features are converted into their respective Hamming codes, the Hamming distance between any two Hamming codes in each group of task data is determined to calculate the second complexity of each group of task data. The present application uses the Hamming distance to measure the similarity between any two task features under the same task to improve the accuracy of judging the complexity between task data.

[0067] Of course, there are many ways to calculate the similarity between data in each set of task data, which are not listed here one by one.

[0068] In some embodiments, optionally, the training sample data includes image data and point cloud data.

[0069] In this embodiment, the image data may include ship image data, specifically image data taken when the ship is traveling, image data taken when the ship is moored at the shore, and image data taken when the ship is operating. The point cloud data may include laser radar point cloud data, which is data obtained by scanning by laser radar. Laser radar is an active remote sensing device that uses laser as the emission light source and photoelectric detection technology. Laser radar has the characteristics of high precision, high resolution and real-time performance, thereby ensuring the accuracy of training sample data.

[0070] In some implementations, optionally, the first complexity is a i , the second complexity is b j , the number of category data corresponding to the first complexity is d i , the sample data volume is h, where: a i 、b j d i and h satisfy the following relationship:

[0071] h=a i ×b j ×w×d i +C;

[0072] Among them, w is a constant, C is a constant, i is the category data type, and j is the task data type.

[0073] In this embodiment, the amount of sample data required for a task in a certain scenario can be determined by calculating the first complexity, the second complexity, and the number of category data corresponding to the first complexity. For example, the first complexity a1 is the complexity of the offshore scenario, the first complexity a2 is the complexity of the shore scenario, the second complexity b1 is the complexity of the detection task, the second complexity b2 is the complexity of the semantic segmentation task, d1 is the number of data in the offshore scenario, and d2 is the number of data in the shore scenario. It can be determined that the amount of sample data required for the detection task in the offshore scenario is a1×b1×w×d1+C, the amount of sample data for the detection task in the shore scenario is a2×b1×w×d2+C, and the amount of sample data for the semantic segmentation task in the shore scenario is a2×b2×w×d2+C.

[0074] In some embodiments, optionally, the method for determining the ratio of training sample data also includes: obtaining a first training model, obtaining training data for each task according to the ratio of sample data amounts between multiple tasks, inputting the training data for each task into the first training model for training, and generating a second training model.

[0075] In this embodiment, after determining the ratio of sample data amounts among multiple tasks, the training data for each task is obtained according to the ratio and input into the corresponding model for training, thereby obtaining a training model with better recognition accuracy. The model can also ensure that the training sample data is more biased towards complex tasks, while avoiding model over-fitting and problems caused by repeated sampling when the number of certain categories is too small or there are a large number of similar pictures.

[0076] Another embodiment of the present invention provides a method for determining a training sample data ratio, comprising the following steps:

[0077] 1. Data input:

[0078] 1) Receive the laser radar point cloud data scanned by the ship while it is moving.

[0079] 2) Receive image data captured while the ship is moving.

[0080] 2. Calculate the complexity of different types of data: The similarity of sampled images is used as the criterion for judging data complexity. The higher the similarity, the lower the data complexity, and vice versa. There are many methods for calculating image similarity. This solution takes metric learning as an example.

[0081] A) Each image or lidar (laserradar) point cloud is converted into a 01 Hamming code through a metric learning method.

[0082] B) The similarity between two images is measured by the Hamming distance. Assume that the Hamming code of the first image is A and the Hamming code of the second image is B. The Hamming distance is the number of 1s in A Xor B, where Xor is bitwise exclusive OR.

[0083] C) Match all the images pairwise and calculate the Hamming distance. Assuming there are N images, a (N-1, N-1) Hamming distance matrix is ​​generated.

[0084] D) Find the mean of all values ​​of the Hamming distance matrix as the data complexity of this category, denoted as a i .

[0085] like Figure 4 As shown, the image data is extracted by category, and the corresponding final features are obtained, the features are converted into corresponding Hamming codes, and the Hamming distances of all data in each category of data are calculated pairwise, and the average is calculated to obtain the complexity of the category of data.

[0086] 3. Calculate the data complexity of different tasks: Extract the task-related features, such as the detection task, extract the features of the corresponding position in the box, resize (different sizes) to size (fixed size), and then use the method in step 2 to calculate the task data complexity b i .

[0087] like Figure 5 As shown in the figure, images of the same task are extracted from different images through tasks, and then all images in the task are resized to a uniform size. After the resize is unified, task features are extracted to obtain multiple final features, such as Figure 5 The final feature 1, final feature 2, etc. in the task are converted into corresponding Hamming codes respectively, and then the Hamming distance is calculated pairwise, and the average of all Hamming distances is used to obtain the complexity of the task data.

[0088] 4. Calculate the data sampling ratio for different task training. Taking binary classification as an example, the data sampling ratio is (a1×b1×w×d1+C): (a2×b2×w×d2+C). Among them, w and C are custom parameters and can be constants. i is the category data complexity of the corresponding category, b i is the task data complexity of the corresponding category, d i For example, a1 is the category data complexity in the offshore scene, a2 is the category data complexity in the shore scene, b1 is the task data complexity of the detection task, b2 is the task data complexity of the semantic segmentation task, d1 is the number of data in the offshore scene, and d2 is the number of data in the shore scene. The above formula can be used to obtain the ratio of the number of samples between the detection task in the offshore scene and the semantic segmentation task in the shore scene.

[0089] The multi-category and multi-task training method based on deep learning has been widely used in various fields. When screening data, the sampling ratio of each category and task data will have a great impact on the final model result. For example, if a certain category or task data accounts for too large a proportion, the accuracy of other tasks and categories will be too low. In traditional methods, some solve this problem by ensuring that the number of training data for each category and task is the same when sampling data, and some solve this problem by adding multiple weights to the final loss (loss function) part and repeatedly adjusting the parameters. In the first method, when the number of certain categories is too small or there are a large number of similar pictures, repeated sampling will cause the model to be over-fitted. Moreover, different problems have different difficulties, which will also lead to low accuracy for complex tasks. The second method will waste a lot of manpower, and the parameters are easy to fit the data.

[0090] The present invention relates to a method for determining the ratio of training sample data, which can make data more focused on complex tasks without the need for additional manpower, while avoiding model overfitting and problems caused by repeated sampling when the number of certain categories is too small or there are a large number of similar pictures.

[0091] The present invention has the following beneficial effects:

[0092] 1. Reasonable proportion of automatically generated training data.

[0093] 2. Save a lot of manual parameter adjustment costs.

[0094] 3. Improve model accuracy.

[0095] like Figure 6 As shown, the second aspect of the present invention provides a device 1 for determining the ratio of training sample data, including: an acquisition unit 10, used to acquire multiple training sample data; a processing unit 12, used to classify the multiple training sample data according to data categories, generate multiple groups of category data, and obtain the number of each group of category data; a calculation unit 14, used to calculate the first complexity of each group of category data based on the similarity between the data, wherein the complexity of the category data and the similarity between the category data are negatively correlated; an extraction unit 16, used to extract the multiple training sample data according to the task, generate multiple groups of task data; the calculation unit 14 is also used to calculate the second complexity of each group of task data based on the similarity between the data, wherein the complexity of the task data and the similarity between the task data are negatively correlated; a determination unit 18, used to determine the amount of sample data required for each task according to the first complexity, the second complexity and the number of category data corresponding to the calculation of the first complexity; and determine the ratio of the sample data amounts between the multiple tasks according to the sample data amounts of the multiple tasks.

[0096] The device 1 for determining the ratio of training sample data provided by the present invention also includes a generation unit, which is used to obtain a first training model, obtain training data for each task according to the ratio of sample data amounts between multiple tasks, input the training data for each task into the first training model for training, and generate a second training model.

[0097] According to the apparatus 1 for determining the ratio of training sample data provided by the present invention, a plurality of training sample data are obtained by using the acquisition unit 10, and the acquired training sample data are first classified by the processing unit 12 according to categories. Categories can also be understood as different scenes of training sample data. Taking the training of ship recognition model as an example, the categories can be divided according to sea scenes, river scenes, shore scenes, etc., so that there are multiple training sample data under each group of category data. Then, the calculation unit 14 is used to calculate the first complexity of each group of category data according to the similarity between the multiple training sample data in each group of category data, and the number of the first complexity is the same as the number of divided category groups. At the same time, the complexity of the category data is negatively correlated with the similarity between the category data, that is, the higher the similarity between the multiple training sample data in each scene, the lower its complexity. Then, according to the actual task situation, the extraction unit 16 is used to extract the corresponding task features from the multiple training sample data. The task can refer to a detection task, a semantic segmentation task, or an online map identification task, etc. According to the similarity between the multiple task features in each group of tasks, the second complexity of each group of task data is calculated, and the number of the second complexity is the same as the number of divided task groups. At the same time, the complexity of task data and the similarity between task data are negatively correlated, that is, the higher the similarity between multiple task features under each group of task data, the lower its complexity. In this way, the determination unit 18 can be used to determine the sample data volume of each task for the first complexity of different category data, the second complexity of different task data, and the number of category data corresponding to the calculation of the first complexity, thereby determining the data volume ratio between the tasks. The training sample data ratio determination device 1 provided in the present application can make the training sample data more biased towards complex tasks by reasonably automatically generating the ratio between the sample data volumes of training data, while avoiding the model over-fitting and problems caused by repeated sampling when the number of certain categories is too small or there are a large number of similar pictures, thereby effectively improving the recognition accuracy of the training model, while saving a lot of costs for manually adjusting the training model parameters and improving the model training efficiency.

[0098] The apparatus 1 for determining the training sample data ratio provided in the embodiment of the present application can implement each process implemented in the above method embodiment, and will not be described again here to avoid repetition.

[0099] like Figure 7As shown, the technical solution of the third aspect of the present invention provides an electronic device 300, including: a memory 302, storing programs and / or instructions; a processor 304 executing the programs and / or instructions; wherein, when the processor 304 executes the programs and / or instructions, it implements the steps of the method for determining the ratio of training sample data in any technical solution of the first aspect, and can achieve the same technical effect, so it will not be repeated here to avoid repetition.

[0100] The technical solution of the fourth aspect of the present invention provides a storage medium on which programs and / or instructions are stored. When the programs and / or instructions are executed, the steps of the method for determining the training sample data ratio in any technical solution of the first aspect are implemented.

[0101] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned training sample data ratio determination method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0102] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0103] In the embodiments according to the present invention, the terms "first", "second", and "third" are only used for description and cannot be understood as indicating or implying relative importance; the term "plurality" refers to two or more, unless otherwise clearly defined. The terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense. For example, "connected" can be a fixed connection, a detachable connection, or an integral connection; "connected" can be a direct connection or an indirect connection through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms in the embodiments according to the present invention can be understood according to specific circumstances.

[0104] In addition, although each operation is described in a specific order, this should be understood as requiring such operation to be performed in a specific order or in a sequential order, or requiring that all illustrated operations should be performed to obtain desired results. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present invention. Some features described in the context of a separate embodiment can also be implemented in a single implementation in combination. On the contrary, the various features described in the context of a single implementation can also be implemented in multiple implementations individually or in any suitable sub-combination mode.

[0105] Although the subject matter has been described in the language of specific structural features and / or method logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

[0106] The above are only preferred embodiments according to the embodiments of the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, various modifications and changes may be made according to the embodiments of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of the present invention shall be included in the protection scope of the embodiments of the present invention.

Claims

1. A method for determining the ratio of training sample data, characterized in that: include: Acquire a plurality of the training sample data; Classifying the plurality of training sample data according to data categories to generate a plurality of groups of category data, and obtaining the quantity of each group of category data; Based on the similarity between the data, a first complexity of each group of the category data is calculated, wherein the complexity of the category data is negatively correlated with the similarity between the category data; Extracting a plurality of the training sample data according to the tasks to generate a plurality of groups of task data; Based on the similarity between the data, a second complexity of each group of the task data is calculated, wherein the complexity of the task data is negatively correlated with the similarity between the task data; Determining the amount of sample data required for each of the tasks according to the first complexity, the second complexity, and the amount of the category data corresponding to the calculation of the first complexity; Determining a ratio of the sample data amounts among the multiple tasks according to the sample data amounts of the multiple tasks; The training sample data includes image data and point cloud data; The image data includes ship image data, and the ship image data includes image data taken when the ship is traveling, image data taken when the ship is moored at the shore, and image data taken when the ship is operating; The point cloud data includes laser radar point cloud data, and the laser radar point cloud data is data obtained by scanning by a laser radar.

2. The method for determining the training sample data ratio according to claim 1, characterized in that: The step of calculating the first complexity of each group of the category data based on the similarity between the data includes: Convert all data in each group of the category data into a first Hamming code; Determine, according to the first Hamming code, a first Hamming distance between any two data in each group of the category data; The first complexity of each group of the category data is calculated according to a plurality of the first Hamming distances in each group of the category data.

3. The method for determining the training sample data ratio according to claim 1, characterized in that: The step of extracting a plurality of the training sample data according to the tasks to generate a plurality of sets of task data comprises: Extracting data corresponding to the task from the plurality of training sample data according to the task, and generating a plurality of groups of task data to be processed; Each group of the task data to be processed is processed in a unified manner to generate multiple groups of the task data.

4. The method for determining the training sample data ratio according to claim 3, characterized in that: The step of calculating the second complexity of each group of task data based on the similarity between the data includes: Convert all data in each group of task data into a second Hamming code; Determine a second Hamming distance between any two data in each group of the task data according to the second Hamming code; The second complexity of each group of task data is calculated according to a plurality of the second Hamming distances in each group of task data.

5. The method for determining the training sample data ratio according to claim 1, characterized in that: The first complexity is a i The second complexity is b j , the number of category data corresponding to the first complexity is d i , the sample data volume is h, where: a i , b j ,d i and h satisfy the following relationship: h=a i ×b j ×w×d i +C; Among them, w is a constant, C is a constant, i is the category data type, and j is the task data type.

6. The method for determining the training sample data ratio according to any one of claims 1 to 5, characterized in that: Also includes: Obtain a first training model; The training data of each task is obtained according to the ratio of the sample data amounts among the multiple tasks, and the training data of each task is input into the first training model for training to generate a second training model.

7. A device for determining the ratio of training sample data, characterized in that: include: An acquisition unit, used for acquiring a plurality of the training sample data; A processing unit, used to classify the plurality of training sample data according to data categories, generate a plurality of groups of category data, and obtain the quantity of each group of category data; A calculation unit, configured to calculate a first complexity of each group of the category data based on the similarity between the data, wherein the complexity of the category data is negatively correlated with the similarity between the category data; An extraction unit, used for extracting the plurality of training sample data according to tasks to generate a plurality of sets of task data; The calculation unit is further used to calculate the second complexity of each group of the task data based on the similarity between the data, wherein the complexity of the task data is negatively correlated with the similarity between the task data; a determining unit, configured to determine the amount of sample data required for each of the tasks according to the first complexity, the second complexity, and the amount of the category data corresponding to the calculation of the first complexity; and to determine the ratio of the amount of sample data among the multiple tasks according to the amount of sample data of the multiple tasks; The training sample data includes image data and point cloud data; The image data includes ship image data, and the ship image data includes image data taken when the ship is traveling, image data taken when the ship is moored at the shore, and image data taken when the ship is operating; The point cloud data includes laser radar point cloud data, and the laser radar point cloud data is data obtained by scanning by a laser radar.

8. An electronic device, characterized in that: include: A memory storing programs and / or instructions; A processor, executing the program and / or the instructions; Wherein, when the processor executes the program and / or the instruction, it implements the steps of the method for determining the training sample data ratio as described in any one of claims 1 to 6.

9. A storage medium, characterized in that: include: The storage medium stores a program and / or an instruction, and when the program and / or the instruction are executed, the steps of the method for determining the training sample data ratio as described in any one of claims 1 to 6 are implemented.

Images