Effective deep learning model training test method and device

Abstract

The invention provides an effective deep learning model training test method and device. The method comprises the following steps: performing de-duplication processing on an original image data set D0, and deleting similar images to obtain a data set D1; performing data enhancement processing on the data set D1 to obtain a data set D2 with increased data volume; and randomly dividing the data set D2 into a plurality of subsets, taking each subset as a test set, taking a set of other subsets as a training set, and carrying out circular cross training and testing on the deep learning model until the trained model meets a precision requirement. According to the method, the data quality can be improved by performing de-duplication processing on the original data set and deleting repeated or similar images; data enhancement processing is carried out on the data set after de-duplication, diversified image samples are generated, and the generalization ability and robustness of the trained model can be improved; by adopting a cross training test method, the model training precision can be improved.

Description

Technical Field

[0001] The present invention belongs to the field of model training and testing, and specifically relates to an effective deep learning model training and testing method and device. Background Art

[0002] Currently, deep learning models have been widely used, and a large number of annotated data sets are required during model training. However, among the massive labels in the data set, noisy labels are often generated due to incorrect annotations. In practical research, the performance of the model is often significantly affected by the quality and quantity of data. Especially in classification tasks, the diversity and richness of training data are directly related to the generalization ability and classification accuracy of the model. Therefore, it is of great significance to quickly establish image classification methods with low annotation costs. Especially in classification tasks, the diversity and accuracy of training data are directly related to the generalization ability and classification accuracy of the model. However, many datasets available in practical applications often have problems such as insufficient samples, unbalanced categories, or noisy data, which may lead to poor performance of the model on new data.

[0003] In view of this, the present invention proposes an effective deep learning model training and testing method and device. Summary of the invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides an effective deep learning model training and testing method and device, which improves the training accuracy and generalization ability of the model through image deduplication, generative data enhancement and cross-training testing of the model.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions.

[0006] In a first aspect, the present invention provides an effective deep learning model training and testing method, comprising the following steps:

[0007] The original image dataset D0 is deduplicated, similar images are deleted, and the dataset D1 is obtained;

[0008] Perform data enhancement processing on the data set D1 to obtain a data set D2 with increased data volume;

[0009] The data set D2 is randomly divided into multiple subsets, each subset is used as a test set, and the collection of the remaining subsets is used as a training set. The deep learning model is cross-trained and tested repeatedly until the trained model meets the accuracy requirements.

[0010] Furthermore, the method for obtaining the data set D1 includes:

[0011] Reduce the image in the dataset D0 to M rows × N columns of pixels and convert it into a grayscale image;

[0012] If the pixel value of the i-th row and j-th column is greater than the pixel value of the i-th row and j+1-th column, it is recorded as a ij =1; otherwise, it is recorded as a ij = 0, and we get the M-row × (N-1)-column matrix A = {a ij} M×(N-1) ; Wherein, i=1,2,...,M, j=1,2,...,N-1;

[0013] The number of elements a at the same position in the matrix A corresponding to any two images in the statistical data set D0 ij If S is less than a set threshold, the two images are similar images, and the two images are combined into a similar image pair;

[0014] Based on similar image pairs, images in data set D0 are deleted to obtain data set D1.

[0015] Furthermore, the method for deleting images in the data set D0 based on similar image pairs includes: deleting any one image in each similar image pair from the data set D0.

[0016] Furthermore, the method for deleting images in the data set D0 based on similar image pairs includes:

[0017] Count the number of times each image in a similar image pair appears in all similar image pairs;

[0018] sorting each image in descending order of said number of times;

[0019] The similar image pairs including the image ranked at the front are deleted in sequence from all similar image pairs, and the image is deleted from the data set D0 at the same time, until all similar image pairs are deleted.

[0020] Furthermore, the data set D1 is enhanced using a generative adversarial network, including:

[0021] Input the data in dataset D1 into the discriminant model D, and input the random noise into the generative model G;

[0022] The generative model G outputs the generated data and inputs the generated data into the discriminative model D;

[0023] The discriminant model D outputs the probability that the generated data is the data in the data set D1, and feeds the probability back to the generative model G;

[0024] The generation model G adjusts the model parameters based on the feedback data to improve the probability output by the discriminant model D;

[0025] Through repeated iterations, generate data that meets the requirements for the output accuracy of the generation model G;

[0026] Continuously supplement the generated data into the dataset D1 to obtain the dataset D2.

[0027] Furthermore, the method for training and testing a deep learning model based on the dataset D2 includes:

[0028] S1. Randomly divide the dataset D2 into K subsets;

[0029] S2. Use the k-th subset as the test set, and the union of the remaining subsets as the training set. Use the training set to train the deep learning model, and use the test set to test the trained model; where 1 ≤ k ≤ K;

[0030] S3. Update k to MOD(k + 1, K), go to step S2, and loop through steps S2 and S3 until the classification accuracy of the deep learning model reaches the set threshold; where MOD(k + 1, K) represents the remainder of (k + 1) divided by K, and when the remainder is 0, its value is K.

[0031] Furthermore, the method for training and testing a deep learning model based on the dataset D2 includes:

[0032] S1. Randomly divide the dataset D2 into K subsets;

[0033] S2. Use the k-th subset as the test set, and the union of the remaining subsets as the training set. Use the training set to train the deep learning model, use the test set to test the trained model, and record the training and testing times m; where 1 ≤ k ≤ K;

[0034] S3. If m < K, update k to MOD(k + 1, K) and go to step S2; otherwise, clear m and go to step S1; loop through steps S2 and S3 until the classification accuracy of the deep learning model reaches the set threshold; where MOD(k + 1, K) represents the remainder of (k + 1) divided by K, and when the remainder is 0, its value is K.

[0035] Furthermore, the method for training and testing a deep learning model based on the dataset D2 includes:

[0036] S1. Randomly divide the dataset D2 into K subsets;

[0037] S2. Use the k-th subset as the test set, and the union of the remaining subsets as the training set. Use the training set to train the deep learning model, use the test set to test the trained model, and record the negative sample data where the model classification is incorrect in the test results and the training and testing times m; where 1 ≤ k ≤ K;

[0038] S3. If m < K, update k to MOD(k + 1, K), and go to step S2; otherwise, go to step S4. Here, MOD(k + 1, K) represents the remainder of (k + 1) divided by K, and when the remainder is 0, its value is K.

[0039] S4. Rebuild the training set containing all negative sample data and the test set containing all negative sample data.

[0040] S5. Use the training set to train the deep learning model, use the test set to test the trained model, and add the new negative sample data that appears in the test results to the training set. Repeat step S5 until the classification accuracy of the deep learning model reaches the set threshold.

[0041] Furthermore, K = 5.

[0042] In a second aspect, the present invention provides an effective deep learning model training and testing device, including:

[0043] A data deduplication module for deduplicating the original image dataset D0, deleting similar images, and obtaining the dataset D1.

[0044] A data augmentation module for performing data augmentation on the dataset D1 to obtain the dataset D2 with an increased amount of data.

[0045] A training and testing module for randomly dividing the dataset D2 into multiple subsets, using each subset as the test set and the union of the remaining subsets as the training set to perform cyclic cross-training and testing on the deep learning model until the trained model meets the accuracy requirements.

[0046] Compared with the prior art, the present invention has the following beneficial effects.

[0047] The present invention realizes effective training and testing of the deep learning model by deduplicating the original image dataset D0, deleting similar images to obtain the dataset D1, performing data augmentation on the dataset D1 to obtain the dataset D2 with an increased amount of data, randomly dividing the dataset D2 into multiple subsets, using each subset as the test set and the union of the remaining subsets as the training set to perform cyclic cross-training and testing on the deep learning model until the trained model meets the accuracy requirements. By deduplicating the original dataset and deleting duplicate or similar images, the present invention can improve the data quality; by performing data augmentation on the deduplicated dataset to generate diverse image samples, the generalization ability and robustness of the trained model can be enhanced; by adopting the cross-training and testing method, the model training accuracy can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1The present invention provides a flowchart of an effective deep learning model training and testing method according to an embodiment of the present invention.

[0049] Figure 2 Schematic diagram of the structure of the generated adversarial model.

[0050] Figure 3 Schematic diagram of the allocation of training and test sets for cross-training and testing.

[0051] Figure 4 A block diagram of the composition of an effective deep learning model training and testing device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present invention clearer and more understandable, the present invention is further described below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0053] Figure 1 The flowchart of an effective deep learning model training and testing method according to an embodiment of the present invention includes the following steps:

[0054] Step 101, perform deduplication processing on the original image data set D0, delete similar images, and obtain a data set D1;

[0055] Step 102, performing data enhancement processing on the data set D1 to obtain a data set D2 with an increased data volume;

[0056] Step 103, randomly divide the data set D2 into multiple subsets, use each subset as a test set, and use the collection of the remaining subsets as a training set, and perform cyclic cross-training and testing on the deep learning model until the trained model meets the accuracy requirements.

[0057] In this embodiment, step 101 is mainly used to perform deduplication processing on the original data set D0. The data set of this embodiment is image data, which can be an RGB image or a medical image. Since similar or identical images often exist in the original image data set, if these similar or identical images are not deleted, it will not only affect the speed of model training and testing, but also affect the effect of model training and testing. Therefore, it is very necessary to perform deduplication processing on image data. There are many methods for deduplication of image data. This embodiment does not limit the specific deduplication method. The following embodiments will give a specific deduplication method.

[0058] In this embodiment, step 102 is mainly used to perform data enhancement processing on the data set D1 obtained after deduplication. Model training generally requires a massive amount of training data, and the amount of data in the original data set initially collected may not meet the training requirements; in addition, the original data (even after deduplication) generally has problems such as few styles and uneven distribution of data types (for example, the data set samples show a long-tail distribution characteristic). To this end, by performing data enhancement processing on the data set D1, a data set D2 with a significantly increased data volume is obtained, and the sample data after enhancement processing is more diversified than the original data, which can improve the generalization ability and robustness of the model. This embodiment does not limit the specific data enhancement method, and the following embodiments will provide a specific data enhancement method.

[0059] In this embodiment, step 103 is mainly used to train and test the model. The training and testing method of this embodiment is: randomly divide the data set D2 into multiple subsets (generally equal), and use each subset as a test set, and the collection of the remaining subsets as a training set, and perform cyclic cross-training and testing on the deep learning model until the trained model meets the accuracy requirements. Since the initial division of the data set D2 is random, and the training set and test set of each round are different (or not completely the same), this allows the model to learn more diverse data categories, thereby obtaining better learning and classification effects.

[0060] As an optional embodiment, a method for obtaining the data set D1 includes:

[0061] Reduce the image in the dataset D0 to M rows × N columns of pixels and convert it into a grayscale image;

[0062] If the pixel value of the i-th row and j-th column is greater than the pixel value of the i-th row and j+1-th column, it is recorded as a ij =1; otherwise, it is recorded as a ij = 0, and we get the M-row × (N-1)-column matrix A = {a ij} M×(N-1) ; Wherein, i=1,2,...,M, j=1,2,...,N-1;

[0063] The number of elements a at the same position in the matrix A corresponding to any two images in the statistical data set D0 ij If S is less than a set threshold, the two images are similar images, and the two images are combined into a similar image pair;

[0064] Based on similar image pairs, images in data set D0 are deleted to obtain data set D1.

[0065] This embodiment provides a technical solution for obtaining a data set D1 by deduplication of images. In order to reduce the amount of data processing, the image in the original data set D0 is first reduced to a fixed size of M rows × N columns of pixels (8 × 9), and the RGB image is converted into a grayscale image. Then, each pixel value outside the Nth column is compared with the pixel value adjacent to its right. If the former is greater than the latter, it is recorded as 1; otherwise, it is recorded as 0. After such processing, a matrix A with M rows and (N-1) columns can be obtained for each image. Obviously, the elements of matrix A are only 0 and 1. Next, the number S of unequal elements in the same position in matrix A of any two images is compared. Obviously, the larger S is, the greater the difference between the two images. Therefore, a threshold can be set. If S is less than the threshold, the two images are considered similar, and the two images are combined into a similar image pair. Finally, based on the similar image pairs, the images in the data set D0 are deleted so that the data set D0 no longer contains similar images, thereby obtaining the data set D1.

[0066] As an optional embodiment, the method for deleting images in the data set D0 based on similar image pairs includes: deleting any one image in each similar image pair from the data set D0.

[0067] This embodiment provides a technical solution for deleting images in D0. The deletion solution of this embodiment is the easiest and simplest solution, because as long as any one image in each similar image pair is deleted from D0, it can be ensured that there are no similar images in D0.

[0068] As an optional embodiment, a method for deleting images in the data set D0 based on similar image pairs includes:

[0069] Count the number of times each image in a similar image pair appears in all similar image pairs;

[0070] sort each image in descending order of said number of times;

[0071] The similar image pairs including the image ranked at the front are deleted in sequence from all similar image pairs, and the image is deleted from the data set D0 at the same time, until all similar image pairs are deleted.

[0072] This embodiment provides another technical solution for deleting images in D0. The simplest solution provided in the previous embodiment has the following problem: the number of deleted images may not be the least, which reduces the amount of data in the data set D1. For example, if A, B and B, C are a pair of similar image pairs respectively, according to the solution of the previous embodiment, the deleted images may be two images A and C, but in fact, only one image B can be deleted to delete the similar images. To this end, this embodiment provides an improved solution of the previous embodiment: first count the number of times each image in the similar image pair appears in all similar image pairs, for example, in the above similar image pairs, A and C appear only once, and B appears twice; then sort each image in order from high to low according to the number of times, and based on the principle of deleting the image with the most appearances first, delete the image from D0 and the similar image pair containing the image from the similar image pair in accordance with the sorting, until all similar image pairs are deleted. For example, delete B and the similar image pair containing B first. Since there is no similar image pair after deleting B, the deletion ends, and the remaining A and C are not similar images. In this way, the number of deleted images can be guaranteed to be minimized.

[0073] As an optional embodiment, data enhancement processing is performed on the data set D1 using a generative adversarial network, including:

[0074] Input the data in dataset D1 into the discriminant model D, and input the random noise into the generative model G;

[0075] The generative model G outputs the generated data and inputs the generated data into the discriminative model D;

[0076] The discriminant model D outputs the probability that the generated data is the data in the data set D1, and feeds the probability back to the generative model G;

[0077] The generation model G adjusts the model parameters based on the feedback data to improve the probability output by the discriminant model D;

[0078] Through repeated iterations, the generated model G is made to output generated data with accuracy that meets the requirements;

[0079] The generated data is continuously added to the data set D1 to obtain the data set D2.

[0080] This embodiment provides a technical solution for data enhancement processing of the data set D1. This embodiment uses a generative adversarial network to perform data enhancement processing on the data set D1. The generative adversarial network is an unsupervised deep learning model, and its network structure is as follows: Figure 2As shown in the figure, it is mainly composed of two neural networks, the generative model G and the discriminative model D. The generative model G receives a random noise and generates image data based on the noise. The discriminative model D receives the data in the data set D1, i.e., the real image, and the generated image data output by the generative model G, and determines whether the generated image is "real". The output probability that the generated data may be the data in the data set D1 can also be understood as the similarity between the generated data and the real image, and the output probability value is fed back to the generative model G, so that the generated data output by the generative model G is closer to the real image. During the training process, the goal of the generative model G is to output a generated image that is as close to the real image as possible to deceive the discriminative model D; and the goal of the discriminative model D is to separate the generated image output by the generative model G from the real image as much as possible. Finally, the generated data that meets the requirements output by the generative model G is added to the data set D1 to obtain the data set D2.

[0081] As an optional embodiment, a method for training and testing a deep learning model based on data set D2 includes:

[0082] S1, randomly divide the data set D2 into K subsets;

[0083] S2, use the kth subset as the test set, and the collection of the remaining subsets as the training set, use the training set to train the deep learning model, and use the test set to test the trained model; where 1≤k≤K;

[0084] S3. Update k to MOD(k+1,K), go to step S2, and execute steps S2 and S3 in a loop until the classification accuracy of the deep learning model reaches the set threshold; wherein MOD(k+1,K) represents the remainder of k+1 divided by K, and when the remainder is 0, its value is K.

[0085] This embodiment provides a technical solution for training and testing a deep learning model. This embodiment consists of steps S1 to S3. Step S1 randomly divides the data set D2 into K (>2) subsets, and the number of data in each subset is equal or approximately equal. Step S2 first determines the training set and the test set: select any subset from the K subsets, such as selecting the kth (1≤k≤K) subset as the test set, and the remaining subsets are combined as the training set; then the deep learning model is trained using the training set, and the trained model is tested using the test set. Step S3 updates k to k+1. Since k+1 may be greater than K, when k+1>K, k is updated to the remainder of k+1 divided by K, and then go to step S2. Steps S2 to S3 are executed in a loop until the classification accuracy of the deep learning model reaches the set threshold. The subsets used in the test set during the loop are the kth, k+1th, ..., K-1th, Kth, 1st, 2nd, ... in sequence.

[0086] As an alternative embodiment, a method for training and testing a deep learning model based on dataset D2 includes:

[0087] S1. Randomly divide dataset D2 into K subsets;

[0088] S2. Use the k-th subset as the test set, and use the union of the remaining subsets as the training set. Train the deep learning model using the training set, test the trained model using the test set, and record the number of training and testing times m; where 1 ≤ k ≤ K;

[0089] S3. If m < K, update k to MOD(k + 1, K), and go to step S2; otherwise, clear m and go to step S1; Loop through steps S2 and S3 until the classification accuracy of the deep learning model reaches the set threshold; where MOD(k + 1, K) represents the remainder of (k + 1) divided by K, and when the remainder is 0, its value is K.

[0090] This embodiment provides another technical solution for training and testing a deep learning model. The difference between this embodiment and the previous one is that: after the test set has used all K subsets, instead of reusing these K subsets, it returns to step S1, randomly groups dataset D2 again, divides it into K new subsets, and selects any one subset from the K new subsets as the test set. The advantage of this processing is that the training set and the test set are constantly changing, which is beneficial to improving the model training accuracy.

[0091] As an alternative embodiment, a method for training and testing a deep learning model based on dataset D2 includes:

[0092] S1. Randomly divide dataset D2 into K subsets;

[0093] S2. Use the k-th subset as the test set, and use the union of the remaining subsets as the training set. Train the deep learning model using the training set, test the trained model using the test set, and record the negative sample data where the model classification is incorrect in the test result and the number of training and testing times m; where 1 ≤ k ≤ K;

[0094] S3. If m < K, update k to MOD(k + 1, K), and go to step S2; otherwise, go to step S4; where MOD(k + 1, K) represents the remainder of (k + 1) divided by K, and when the remainder is 0, its value is K;

[0095] S4. Rebuild a training set containing all negative sample data and a test set containing all negative sample data;

[0096] S5. Train the deep learning model using the training set, test the trained model using the test set, and add new negative sample data appearing in the test results to the training set; repeat step S5 until the classification accuracy of the deep learning model reaches the set threshold.

[0097] This embodiment provides a third technical solution for training and testing a deep learning model. This embodiment is an improved solution of the previous embodiment. Unlike the previous embodiment, this embodiment records the negative sample data in the test results where the model is misclassified after each test; when all K subsets of the test set are used, the training set and the test set are reconstructed, and the characteristics are that the training set and the test set both include all the negative sample data obtained previously, that is, the training set and the test set are composed of all negative samples and some positive samples. Of course, the training set and the test set use different positive samples, and the number of positive samples in the training set is significantly greater than the number of positive samples in the test set. Then the deep learning model is trained using the training set, the trained model is tested using the test set, and the new negative sample data appearing in the test results is added to the training set, and then new training and testing are started. By repeating the above process continuously, the number of negative samples will be rapidly reduced, and the model's ability to correctly classify negative samples will be improved, thereby effectively improving the classification accuracy of the model.

[0098] As an optional embodiment, K=5.

[0099] This embodiment limits the number K of subsets into which the data set D2 is divided. In this embodiment, K=5, and the method for training and testing the model is as follows: Figure 3 As shown, the data set D2 is divided into 5 subsets, and the test sets for each training test are the 5th, 4th, 3rd, 2nd, and 1st subsets respectively. It is worth noting that this embodiment only provides a preferred implementation method, and does not deny or exclude other feasible implementation methods, such as K taking a value other than 5.

[0100] Figure 4 The present invention is a schematic diagram of an effective deep learning model training and testing device according to an embodiment of the present invention, wherein the device comprises:

[0101] The data deduplication module 11 is used to perform deduplication processing on the original image data set D0, delete similar images, and obtain the data set D1;

[0102] A data enhancement module 12 is used to perform data enhancement processing on the data set D1 to obtain a data set D2 with an increased data volume;

[0103] The training and testing module 13 is used to randomly divide the data set D2 into multiple subsets, use each subset as a test set, and use the collection of the remaining subsets as a training set, and perform cyclic cross-training and testing on the deep learning model until the trained model meets the accuracy requirements.

[0104] The device of this embodiment can be used to perform Figure 1 The technical solution of the method embodiment shown has similar implementation principles and technical effects, which will not be repeated here.

[0105] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An effective deep learning model training and testing method, characterized in that: The following steps are involved: The original image dataset D0 is deduplicated, similar images are deleted, and the dataset D1 is obtained; Perform data enhancement processing on the data set D1 to obtain a data set D2 with increased data volume; The data set D2 is randomly divided into multiple subsets, each subset is used as a test set, and the collection of the remaining subsets is used as a training set. The deep learning model is cross-trained and tested repeatedly until the trained model meets the accuracy requirements.

2. The effective deep learning model training and testing method according to claim 1, characterized in that: The method of obtaining the data set D1 includes: Reduce the image in the dataset D0 to M rows × N columns of pixels and convert it into a grayscale image; If the pixel value of the i-th row and j-th column is greater than the pixel value of the i-th row and j+1-th column, it is recorded as a ij =1; otherwise, it is recorded as a ij = 0, and we get the M-row × (N-1)-column matrix A = {a ij } M×(N-1) ; Wherein, i=1,2,...,M, j=1,2,...,N-1; The number of elements a at the same position in the matrix A corresponding to any two images in the statistical data set D0 ij If S is less than a set threshold, the two images are similar images, and the two images are combined into a similar image pair; Based on similar image pairs, images in data set D0 are deleted to obtain data set D1.

3. The effective deep learning model training and testing method according to claim 2, characterized in that: The method for deleting images in the data set D0 based on similar image pairs includes: deleting any one image in each similar image pair from the data set D0.

4. The effective deep learning model training and testing method according to claim 2, characterized in that: The method for deleting images in the data set D0 based on similar image pairs includes: Count the number of times each image in a similar image pair appears in all similar image pairs; sorting each image in descending order of said number of times; The similar image pairs including the image ranked at the front are deleted in sequence from all similar image pairs, and the image is deleted from the data set D0 at the same time, until all similar image pairs are deleted.

5. The effective deep learning model training and testing method according to claim 1, characterized in that: The data set D1 is enhanced using a generative adversarial network, including: Input the data in dataset D1 into the discriminant model D, and input the random noise into the generative model G; The generative model G outputs the generated data and inputs the generated data into the discriminative model D; The discriminant model D outputs the probability that the generated data is the data in the data set D1, and feeds the probability back to the generative model G; The generation model G adjusts the model parameters based on the feedback data to improve the probability output by the discriminant model D; Through repeated iterations, the generated model G is made to output generated data with accuracy that meets the requirements; The generated data is continuously added to the data set D1 to obtain the data set D2.

6. The effective deep learning model training and testing method according to claim 1, characterized in that: The method for training and testing the deep learning model based on the data set D2 includes: S1, randomly divide the data set D2 into K subsets; S2. Use the kth subset as the test set, and the collection of the remaining subsets as the training set. Use the training set to train the deep learning model, use the test set to test the trained model, and record the number of training and testing times m; where 1≤k≤K; S3. Update k to MOD(k+1,K), and go to step S2; loop through steps S2 and S3 until the classification accuracy of the deep learning model reaches the set threshold; wherein MOD(k+1,K) represents the remainder of k+1 divided by K, and when the remainder is 0, its value is K.

7. The effective deep learning model training and testing method according to claim 1, characterized in that: The method for training and testing the deep learning model based on the data set D2 includes: S1, randomly divide the data set D2 into K subsets; S2. Use the kth subset as the test set, and the collection of the remaining subsets as the training set. Use the training set to train the deep learning model, use the test set to test the trained model, and record the number of training and testing times m; where 1≤k≤K; S3. If m < K, update k to MOD(k + 1, K), and go to step S2; otherwise, clear m and go to step S1; loop and execute steps S2 and S3 until the classification accuracy of the deep learning model reaches the set threshold; where MOD(k + 1, K) represents the remainder of (k + 1) divided by K, and its value is K when the remainder is 0.

8. The effective deep learning model training and testing method according to claim 1, characterized in that: The method for training and testing a deep learning model based on dataset D2 includes: S1. Randomly divide dataset D2 into K subsets. S2. Use the k-th subset as the test set, and use the union of the remaining subsets as the training set. Use the training set to train the deep learning model, use the test set to test the trained model, and record the negative sample data where the model classification is incorrect in the test results and the training and testing times m; where 1 ≤ k ≤ K. S3. If m < K, update k to MOD(k + 1, K), and go to step S2; otherwise, go to step S4; where MOD(k + 1, K) represents the remainder of (k + 1) divided by K, and its value is K when the remainder is 0. S4. Rebuild a training set containing all negative sample data and a test set containing all negative sample data. S5. Use the training set to train the deep learning model, use the test set to test the trained model, and add the new negative sample data that appears in the test results to the training set; repeat step S5 until the classification accuracy of the deep learning model reaches the set threshold.

9. The effective deep learning model training and testing method according to claim 8, characterized in that: K＝5。 10. An effective deep learning model training and testing device, characterized in that: including: A data deduplication module for deduplicating the original image dataset D0, deleting similar images, and obtaining dataset D1. A data augmentation module for performing data augmentation on dataset D1 to obtain dataset D2 with an increased amount of data. A training and testing module for randomly dividing dataset D2 into multiple subsets, using each subset as the test set and the union of the remaining subsets as the training set, and performing cyclic cross-training and testing on the deep learning model until the trained model meets the accuracy requirements.

Images