A reinforcement learning-based method and system for assessing the quality of augmented data

Abstract

This invention belongs to the field of ship image processing technology, and provides a method and system for evaluating the quality of augmented data based on reinforcement learning. The method includes: dividing the dataset to be augmented into a training set and a test set; augmenting the training set using relevant augmentation methods; inputting each ship image data, its corresponding loss value, and accuracy into an established data evaluation model to obtain an evaluation value for each ship image data; completing the reinforcement learning process of the data evaluation model according to a specified number of iterations to obtain an optimized data evaluation model and an evaluation probability distribution map; and calculating the KL divergence between each ship image data in the dataset to be augmented and the corresponding original ship image data in the dataset to be augmented to determine the quality of each ship image data. This invention effectively preserves the scoring results while improving the stability of the output results, ensuring the usability and effectiveness of the data evaluation results for augmentation work.

Description

Technical Field

[0001] This invention relates to the field of ship image processing technology, and in particular to a method and system for evaluating the quality of augmented data based on reinforcement learning. Background Technology

[0002] In recent years, with the development of Artificial Intelligence (AI) technology, various fields are realizing intelligent applications. Data has become a particularly important part of AI technology, and data quality directly affects the final inference success rate of AI models. However, in specific specialized fields, data remains scarce due to the difficulty and high cost of data collection. Data augmentation is one of the most effective solutions to the problem of lack of datasets in specialized fields, and many data augmentation methods are currently available to meet normal model training needs. However, the effectiveness of these data augmentation methods, i.e., the quality of augmented data, is usually verified manually or experimentally. Manual verification methods mostly rely on domain experts to screen new data one by one, which cannot quantify the true effect of the augmentation method and consumes a lot of time. Many existing studies that use data augmentation to solve the problem of lack of samples mostly demonstrate the effect of the selected data augmentation methods on training through experimental design. These experiments mainly reflect the role of data augmentation from the perspective of results, and the evaluation is relatively lagging and the judgment period is long. For example, the quality of augmented data obtained by methods such as AutoAug and RandAug is mostly judged by the final inference results of the model. This means that evaluating data quality after training and inference are complete, such as with NAS-based data augmentation strategies, often involves enormous computational costs, resulting in long evaluation cycles and a lack of real-time performance. Furthermore, there is currently no quantitative verification method to determine whether complex augmentation methods can achieve higher quality than augmented data obtained solely through position transformation and pruning. Addressing this issue could improve the efficiency of selecting data augmentation methods.

[0003] Reinforcement learning is an interactive learning method that designs interactive methods to provide reward feedback, ultimately iterating to maximize gains. Data Valuation Using Reinforcement Learning (DVRL) is a reinforcement learning dataset evaluation and analysis model proposed by Google. It achieves the optimal data score through dynamic adjustments of policy changes and value feedback, determining the quantitative value of each training sample for training a predictive model. This model can be used to build dataset quality assessment tools, effectively screening noisy data within the dataset.

[0004] Research and analysis of existing technologies have revealed the following problems:

[0005] In existing methods for evaluating the quality of augmented or augmented data, researchers often need to conduct multiple experiments to evaluate the application effect of the augmented dataset. During this process, multiple rounds of model training are required, which consumes a lot of computing power, has high trial and error costs, long cycles, and lacks real-time performance. The evaluation process is not coupled with the training process. Existing image quality evaluation methods based on deep feature extraction and evaluation methods for general data quality indicators (balance, normality, bias, discrimination, etc.) do not involve combining the evaluation process with the training process. If they are directly applied to augmented data, they cannot reflect the value of augmented data in the training task.

[0006] Therefore, it is necessary to provide a new reinforcement learning-based method for assessing the quality of augmented data in order to address the aforementioned problems. Summary of the Invention

[0007] This invention aims to provide a reinforcement learning-based method and system for evaluating the quality of augmented data. This addresses the technical problems of existing image quality evaluation methods and evaluations of general data quality indicators (balance, normality, bias, discrimination, etc.) that do not simultaneously involve the evaluation and training processes. In other words, the evaluation and training processes are not coupled, lacking coordination and resulting in poor evaluation method integrity, high trial-and-error costs, long cycles, and lack of real-time performance. The invention addresses these issues by providing the following technical solutions.

[0008] The first aspect of this invention proposes a reinforcement learning-based method for evaluating the quality of augmented data, comprising: dividing the dataset to be augmented into a training set and a test set; performing augmentation processing on the training set using a relevant augmentation method to obtain an augmented dataset; allocating a portion of the augmented dataset as a validation set; the dataset to be augmented including ship image data labeled with ship category tags; establishing a data evaluation reinforcement learning model, the data evaluation reinforcement learning model including a data prediction model and a data evaluation model; and optimizing the data prediction model and the data evaluation model by performing the following steps: Step S201: Initializing the data prediction model; inputting a preset batch of ship image data into the data prediction model, recording the loss value generated by each ship image data, and so on. The accuracy of the data prediction model for each ship image data is verified using a validation set, and the data prediction model is optimized according to the optimization objective. Step S202: Each ship image data, its corresponding loss value, and accuracy are input into the data evaluation model to obtain the evaluation value of each ship image data. Step S203: The data prediction model is initialized again, and the reinforcement learning process of the data evaluation model is completed according to the specified number of iterations to obtain the optimized data evaluation model and the evaluation probability distribution map. Based on the obtained evaluation probability distribution map, the KL divergence between each ship image data in the dataset to be augmented and the corresponding original ship image data in the dataset to be augmented is calculated to determine the quality of each ship image data in the augmented dataset.

[0009] According to a preferred embodiment, the data prediction model is a ResNet model, using... This represents the optimized data prediction model, where the optimization objective of the data prediction model is to optimize the weighted loss function. Minimize to obtain the optimized data prediction model; the data evaluation model is a BP neural network model, using... Represents the i-th training sample (x) i ,y i The evaluation value of );

[0010] The optimization objective of the data prediction model is characterized by the following expression:

[0011] ;

[0012] in, This refers to the optimized data prediction model, where the optimization objective is to minimize the loss function value. Compared with sample evaluation value The product of x is minimized. i The input to the data prediction model includes the i-th ship image data, the loss value and accuracy corresponding to the i-th ship image data, y. iα represents the ship category label value corresponding to the i-th ship image data; α represents the learning rate of the data prediction model; β represents the learning rate of the data evaluation model; θ and φ represent the weight parameters of each layer of the data prediction model and the data evaluation model participating in the forward propagation calculation, respectively.

[0013] According to a preferred embodiment, the following expression is used to represent the update process of weights and parameters θ and φ in each iteration:

[0014] ,

[0015] ,

[0016] Where α represents the learning rate of the data prediction model; β represents the learning rate of the data evaluation model; B p This refers to the number of data points contained in a batch of data during the training process. p is a subscript marker, indicating that the batch of data is used for the prediction model. This indicates that the training will be conducted in one batch. express One piece of data in the data, where B s This represents a batch of data during the verification process, where s represents the correct or incorrect verification result. This refers to the validation result of the m-th data in the current summation loop, where m is a positive integer, representing 1, 2, ..., n; L represents the number of parameters in the model; This represents the image data of the training set; This represents the data labeling of the training set, k indicates that the summation operation has been performed to the kth round; v indicates that this data is used to validate the model; h is the sample evaluation value. , x i The input to the data prediction model includes the i-th ship image data, the loss value and accuracy corresponding to the i-th ship image data, y. i This represents the ship category label value corresponding to the i-th ship image data;

[0017] m represents the m-th round of the summation operation; L represents the number of dimensions of the input data to the prediction model in one iteration.

[0018] θ and φ represent the weight parameters of each layer of the data prediction model and the data evaluation model, respectively, for the forward propagation calculation.

[0019] According to a preferred embodiment, the KL divergence between each ship image data in the augmented dataset and the corresponding original ship image data in the dataset to be augmented is calculated using the following expression:

[0020] ;

[0021] in, Let P(x) represent the KL divergence between the i-th ship image data and the corresponding original ship image data in the dataset to be augmented, where i is a positive integer, specifically 1, 2, ..., N; P(x) is the quality probability distribution of the augmented dataset, and Q(x) is the quality probability distribution of the dataset to be augmented, i.e., the original dataset.

[0022] According to a preferred embodiment, if the model accuracy of the data prediction model trained using the augmented dataset is greater than a specified value, then the augmented dataset is determined to be effective in restoring the quality of the original ship image data. The specified value is determined by the model accuracy of the data prediction model obtained by training the data prediction model using the training dataset.

[0023] According to a preferred embodiment, based on the ship image classification application scenario, a specified hyperparameter and a specified number of iterations are selected, wherein the specified hyperparameter is in the range of 50 to 400 times, and the specified number of iterations is in the range of 50 to 200 times.

[0024] According to a preferred embodiment, the step of re-initializing the data prediction model and completing the reinforcement learning process of the data evaluation model according to a specified number of iterations to obtain an optimized data evaluation model includes: when re-initializing the data prediction model, randomly generating the network weight parameters of each layer in the data prediction model, and repeatedly executing the iteration process according to a specified number of iterations until the specified number of iterations is completed, thereby completing the reinforcement learning process of the data prediction model.

[0025] According to a preferred embodiment, the single ship image data (x i ,y i The image data of this single ship (x) i ,y i The corresponding loss value The accuracy μ is input into the data evaluation model to obtain the evaluation value h(x) of the single ship image data. i ,y i The evaluation value h(x) of the single ship image data i ,y i () is a double-precision floating-point number between 0 and 1.

[0026] The second aspect of this invention proposes a ship image augmentation data quality evaluation system, employing the augmentation data quality evaluation method described in the first aspect of this invention, comprising: a data processing module, which divides the dataset to be augmented into a training set and a test set, performs augmentation processing on the training set using relevant augmentation methods to obtain an augmented dataset, and allocates a portion of the augmented dataset as a validation set, wherein the dataset to be augmented includes ship image data labeled with ship category tags; and a model building module, which establishes a data evaluation reinforcement learning model, wherein the data evaluation reinforcement learning model includes a data prediction model and a data evaluation model, and optimizes the data prediction model and the data evaluation model by: initializing the data prediction model; inputting a preset batch of ship image data into the data prediction model, and recording each The loss value generated from each ship image data is used to verify the accuracy of the data prediction model for each ship image data using a validation set. The data prediction model is then optimized according to the optimization objective. Each ship image data, its corresponding loss value, and accuracy are input into the data evaluation model to obtain an evaluation value for each ship image data. The data prediction model is then initialized again, and the reinforcement learning process of the data evaluation model is completed according to a specified number of iterations to obtain the optimized data evaluation model and an evaluation probability distribution map. The calculation module calculates the KL divergence between each ship image data in the dataset to be augmented and the corresponding original ship image data in the dataset to be augmented, based on the obtained evaluation probability distribution map, to determine the quality of each ship image data in the augmented dataset.

[0027] A third aspect of the present invention provides an electronic device comprising: one or more processors; a storage device for storing one or more programs; wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in the first aspect of the present invention.

[0028] A fourth aspect of the present invention provides a computer-readable medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the method described in the first aspect of the present invention.

[0029] The present invention has the following advantages:

[0030] Compared with existing technologies, this invention can dynamically adjust the learning rate based on the predicted score during model training iterations. It can effectively retain the predicted score results while improving the stability of the output results, thereby ensuring the usability and effectiveness of the data evaluation results for augmentation work. This further improves the efficiency of data-related workers in the field. It can effectively couple the evaluation process with the training process, thereby avoiding the problem of poor data quality integrity caused by the lack of coordination between the evaluation process and the training process. By achieving the optimization goal, it optimizes the data prediction model and the data evaluation model at the same time.

[0031] Furthermore, by designing a data value evaluation model based on a reward mechanism, and by exploring the loss function values ​​generated by different data during the training of the data prediction model and their impact on the prediction results, the value of the obtained data can be accurately calculated.

[0032] In addition, the value of the augmented dataset is determined by calculating the ranking of the value of each data point in the obtained dataset and the distribution of the augmented data in the overall ranking. Attached Figure Description

[0033] Figure 1 This is a flowchart illustrating an example of the augmented data quality assessment method based on reinforcement learning according to the present invention.

[0034] Figure 2 This is a schematic diagram of an example of a ship image in an application example of the reinforcement learning-based augmented data quality assessment method of the present invention;

[0035] Figure 3 It shows Figure 1 A flowchart illustrating an example of optimizing the data prediction model and the data evaluation model in an augmented data quality assessment method;

[0036] Figure 4 This is a schematic diagram of an example of augmented data quality assessment based on reinforcement learning according to the present invention;

[0037] Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention;

[0038] Figure 6 This is a schematic diagram of a computer-readable medium embodiment according to the present invention. Detailed Implementation

[0039] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0040] In view of the above problems, this invention proposes a reinforcement learning-based method for evaluating the quality of augmented data. This method targets ship image datasets and their augmented portions, utilizes a reinforcement learning reward mechanism to obtain the corresponding tasks and pre-trained models, predicts the score ranking of each data point in the dataset by analyzing parameter changes during training, and evaluates the quality of the augmented dataset by analyzing the distribution of the augmented data in the ranking. This method can effectively improve the effectiveness of augmented data quality evaluation.

[0041] It's important to note that reinforcement learning is an interactive learning method that provides reward feedback through interactive mechanisms, ultimately iterating to maximize gains. Data Valuation Using Reinforcement Learning (DVRL) is a reinforcement learning dataset evaluation and analysis model. It achieves the optimal data score through dynamic adjustments of policy changes and value feedback, determining the quantitative value of each training sample for training a prediction model (i.e., a data prediction model). For example, it can also be used to build dataset quality assessment tools based on this data prediction model, effectively screening noisy data within the dataset.

[0042] Example 1:

[0043] The following reference Figure 1 , Figure 2 and Figure 3 The present invention will be described in detail below.

[0044] Figure 1 This is a flowchart illustrating an example of the reinforcement learning-based augmented data quality assessment method of the present invention. Figure 2 This is a schematic diagram of a ship image, which is an application example of the reinforcement learning-based augmented data quality assessment method of the present invention.

[0045] like Figure 1 As shown, in step S101, the dataset to be augmented is divided into a training set and a test set. The training set is augmented using a relevant augmentation method to obtain the augmented dataset. A portion of the data is then divided from the augmented dataset as a validation set. The dataset to be augmented includes ship image data labeled with ship category tags.

[0046] Specifically, collecting, for example Figure 2 The data to be augmented, such as ship-related images, is divided into a training set and a test set. Then, the training set is augmented using relevant augmentation methods (such as generative adversarial networks in the image domain) to obtain augmented data similar to the data to be augmented (e.g., generating ship image data with ship category labels) to build a training dataset. A portion of the augmented training set is then divided into a validation set.

[0047] It should be noted that in other embodiments, bidirectional constrained generative adversarial networks or other algorithms can be used for augmentation processing. The above are merely illustrative examples and should not be construed as limiting the present invention.

[0048] Next, in step S102, a data evaluation reinforcement learning model is established, which includes a data prediction model and a data evaluation model.

[0049] For example, the data prediction model in a ship identification task scenario is a ResNet model, using... This represents the optimized data prediction model. The data evaluation model is a BP neural network model.

[0050] Specifically, the data prediction model is, for example, a multi-classification model or a ship image classification model.

[0051] In one specific implementation, the data prediction model is a ship image classification model, for example, used to distinguish ship categories such as transport ships, passenger ships, fishing vessels, police vessels, small and medium-sized vessels (e.g., civilian vessels), large vessels, and aircraft carriers. The training data includes ship images and ship classification labels (e.g., ship images labeled with the aforementioned ship category labels, or images with ship locations marked).

[0052] Preferably, for ship image classification applications, the hyperparameters Batchsize (the amount of data in a batch), for example, denoted by m, and Epoch (the number of iterations), for example, denoted by k, are set. The selection of hyperparameters m and k can be adaptively varied according to the experimental requirements of the user's actual application scenario. Optionally, a specified hyperparameter m and a specified number of iterations k can be selected according to the ship image classification application scenario, wherein hyperparameter m is between 50 and 400 iterations, and hyperparameter k is between 50 and 200 iterations.

[0053] To avoid the problem of the evaluation process not being coupled with the training process, and the problem of poor data quality and integrity due to the lack of coordination between the evaluation process and the training process, this invention optimizes the data evaluation model while simultaneously optimizing the data prediction model.

[0054] Specifically, the optimization objective (i.e., the optimization objective of the data prediction model) is to make the weighted loss function... Minimize. Optimize the data prediction model according to the optimization objective to obtain the optimized data prediction model.

[0055] The optimization objective of the data prediction model is represented by the following expression:

[0056] ,

[0057] in, This refers to the optimized data prediction model, where the optimization objective is to minimize the loss function value. Compared with sample evaluation value The product of x is minimized. i y represents the input to the data prediction model, including the i-th ship image data, the corresponding loss value and accuracy, and y'. iα represents the ship category label value corresponding to the i-th ship image data; α represents the learning rate of the data prediction model; β represents the learning rate of the data evaluation model; θ and φ represent the weight parameters of each layer of the data prediction model and the data evaluation model participating in the forward propagation calculation, respectively.

[0058] Figure 3 It shows Figure 1 A flowchart illustrating an example of optimizing the data prediction model and the data evaluation model in an augmented data quality assessment method.

[0059] like Figure 3 As shown, the data prediction model and data evaluation model are optimized by performing the following steps:

[0060] Step S201: Initialize the data prediction model.

[0061] Specifically, the weight parameters of each layer in the data prediction model (e.g., the ResNet model) are randomly generated to assist subsequent steps in verifying the gain effect of the data on the model weight updates. A predetermined batch of ship image data (e.g., a specified number from the training dataset) is input into the data prediction model according to the dataset order. After [n / m] backpropagation iterations, the loss function value generated for each ship image data is recorded. The accuracy μ of the data prediction model for each ship image data was then verified using a validation set.

[0062] Step S202: Input each ship image data, the corresponding loss value and accuracy into the data evaluation model to obtain the evaluation value of each ship image data.

[0063] Specifically, each piece of ship image data, its corresponding loss value, and accuracy (i.e., the loss function value corresponding to each piece of ship image data obtained in step S201 above) are used. And accuracy μ, loss function value The value of μ is between 0 and 1; the value of μ is between 0 and 1. These are used as input features and input into the data evaluation model to obtain the evaluation value of each ship image data.

[0064] It should be noted that the data evaluation model outputs an evaluation score for each ship image data after several updates and iterations. The data evaluation model consists of a multilayer perceptron, with the input being a ship image data, the output features of the last layer of the data prediction model, and the accuracy μ of the prediction model. The output is the score of that ship image data, i.e., the evaluation score.

[0065] For example, single ship image data (x i ,y i The image data of this single ship (x)i ,y i The corresponding loss value The accuracy μ is input into the data evaluation model to obtain the evaluation value h(x) of the single ship image data. i ,y i The evaluation value h(x) of the single ship image data i ,y i ) is an n-dimensional array generated iteratively by the data evaluation model, or a double-precision floating-point number between 0 and 1.

[0066] In other words, before the training process of the data prediction model is complete, the current accuracy of the model on the validation set and the loss value generated by the input ship image data during model training are also used as inputs to the data evaluation model. A higher success rate and a lower loss function value mean higher data quality. A simple reinforcement learning classifier is used to classify the quality of the augmented data, and the output is, for example, a floating-point number between 0 and 1, representing the quality score of the augmented data. The closer to 1, the higher the quality, and the closer to 0, the lower the quality.

[0067] Step S203: Reinitialize the data prediction model, complete the reinforcement learning process of the data evaluation model according to the specified number of iterations, and obtain the optimized data evaluation model.

[0068] When the data prediction model is initialized again, the weight parameters of each layer of the network in the data prediction model are randomly generated. Steps S201 and S202 (i.e., the iteration process is repeated) are executed repeatedly according to the specified number of iterations until the specified number of iterations is completed, thus completing the reinforcement learning process of the data prediction model.

[0069] Specifically, the following expression characterizes the update process of weights and parameters θ and φ in each iteration:

[0070] ,

[0071] ,

[0072] Where α represents the learning rate of the data prediction model; β represents the learning rate of the data evaluation model; B p This refers to the number of data points contained in a batch during training. p is a subscript indicating that the batch of data is used for the prediction model. This indicates that the training will be conducted in one batch. express One piece of data in the data, where B s This represents a batch of data during the verification process, where s indicates a correct or incorrect verification result (e.g., 0 for correct and 1 for incorrect). This refers to the validation result of the m-th data in the current summation loop, where m is a positive integer, representing 1, 2, ..., n; L represents the number of parameters in the model; This represents the image data of the training set; This represents the data labeling of the training set, k indicates that the summation operation has been performed to the kth round; v indicates that this data is used to validate the model; h is the sample evaluation value. , x i y represents the input to the data prediction model, including the i-th ship image data, the corresponding loss value and accuracy, and y'. i This represents the ship category label value corresponding to the i-th ship image data;

[0073] m represents the m-th round of the summation operation; L represents the number of dimensions of the input data to the prediction model in one iteration.

[0074] θ and φ represent the weight parameters of each layer of the data prediction model and the data evaluation model, respectively, for the forward propagation calculation.

[0075] It's important to note that the data evaluation model utilizes a simple reinforcement learning classifier. In this model, the input includes not only the image itself and its label, but also the model's current accuracy on the validation set before the training process is complete, and the loss value generated by the images during training. A higher success rate and a lower loss function value indicate higher data quality.

[0076] Next, in step S103, the KL divergence between each ship image data in the dataset to be augmented and the corresponding original ship image data in the dataset to be augmented is calculated to determine the quality of each ship image data in the augmented dataset.

[0077] For the data prediction model and the data evaluation model, after completing the above reinforcement learning model iteration process, the evaluation score h(x) is then used as the basis for further evaluation. i ,y i To obtain the probability distribution of the evaluation values, for example, by using a tool similar to Distribution Fitter to obtain the evaluation score h(x) i ,y i The evaluation probability distribution plot of the augmented data is used to calculate the KL divergence between the augmented data and the original data. Specifically, the following expression is used to calculate the KL divergence between each ship image data in the augmented dataset (i.e., augmented data) and the corresponding original ship image data in the dataset to be augmented (i.e., original data), in order to determine the degree of fit between the augmented data and the original data in terms of training value:

[0078] ;

[0079] in, Let P(x) represent the KL divergence between the i-th ship image data and the corresponding original ship image data in the dataset to be augmented, where i is a positive integer, specifically 1, 2, ..., N; P(x) is the evaluation probability distribution of the augmented dataset, and Q(x) is the evaluation probability distribution of the dataset to be augmented, i.e., the original dataset.

[0080] It should be noted that the KL divergence of ship image data is used to characterize the fit. The smaller the calculated KL divergence, the closer the ship image data corresponding to the augmented data is to the original data, indicating that the augmented data better reproduces the original data in terms of quality, i.e., the higher the data quality of the augmented data. Conversely, the larger the calculated KL divergence, the less closely the ship image data corresponding to the augmented data is to the original data, i.e., the lower the data quality of the augmented data.

[0081] Optionally, a simple reinforcement learning classifier can be used to classify the quality of the obtained ship image data. The output is a floating-point number between 0 and 1, representing the quality score of the ship image data; the closer to 1, the higher the quality. The ship image data is, for example, a 640*512 resolution RGB three-channel color image.

[0082] It should be noted that the above description is merely an optional example and should not be construed as limiting the present invention. Furthermore, the above figures are only illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0083] Compared with existing technologies, this invention can dynamically adjust the learning rate based on the predicted score during model training iterations. It can effectively retain the predicted score results while improving the stability of the output results, thereby ensuring the usability and effectiveness of the data evaluation results for augmentation work. This further improves the efficiency of data-related workers in the field. It can effectively couple the evaluation process with the training process, thereby avoiding the problem of poor data quality integrity caused by the lack of coordination between the evaluation process and the training process. By achieving the optimization goal, it optimizes the data prediction model and the data evaluation model at the same time.

[0084] Furthermore, by designing a data value evaluation model based on a reward mechanism, and by exploring the loss function values ​​generated by different data during the training of the data prediction model and their impact on the prediction results, the value of the obtained data can be accurately calculated.

[0085] In addition, the value of the augmented dataset is determined by calculating the ranking of the value of each data point in the obtained dataset and the distribution of the augmented data in the overall ranking.

[0086] Example 2:

[0087] The following are system embodiments of the present invention, which can be used to execute the system embodiments of the present invention. For details not disclosed in the system embodiments of the present invention, please refer to the system embodiments of the present invention.

[0088] Figure 4 This is a schematic diagram of an example of a ship image augmentation data quality evaluation system according to the present invention.

[0089] like Figure 4 As shown, the second aspect of this disclosure provides a ship image augmentation data quality evaluation system 400, which includes a data processing module 410, a model building module 420, and a calculation module 430.

[0090] Specifically, the data processing module 410 divides the dataset to be augmented into a training set and a test set, uses relevant augmentation methods to augment the training set to obtain the augmented dataset, and divides a portion of the data from the augmented dataset as a validation set. The dataset to be augmented includes ship image data labeled with ship category tags.

[0091] The model building module 420 is used to build a data evaluation reinforcement learning model, which includes a data prediction model and a data evaluation model, and to optimize the data prediction model and the data evaluation model.

[0092] Specifically, a pre-set batch of ship image data is used as input to the data prediction model. The loss value generated by each ship image data is recorded. The accuracy of the data prediction model for each ship image data is verified using a validation set. The data prediction model is then optimized according to the optimization objective.

[0093] The model building module 420 also includes an initialization module, a processing module, and a re-initialization module.

[0094] In one specific implementation, the initialization module is used to initialize the data prediction model. The processing module is used to input each ship image data, its corresponding loss value, and accuracy into the data evaluation model to obtain an evaluation value for each ship image data. The re-initialization module is used to re-initialize the data prediction model, completing the reinforcement learning process of the data evaluation model according to a specified number of iterations, to obtain an optimized data evaluation model.

[0095] Next, the calculation module 430 is used to calculate the KL divergence between each ship image data in the dataset to be augmented and the corresponding original ship image data in the dataset to be augmented, so as to determine the quality of each ship image data in the augmented dataset.

[0096] In one optional implementation, a specified hyperparameter and a specified number of iterations are selected according to the ship image classification application scenario, wherein the specified hyperparameter is in the range of 50 to 400 times, and the specified number of iterations is in the range of 50 to 200 times.

[0097] The data prediction model is a ResNet model, using... This indicates that the optimization objective of the data prediction model is to optimize the weighted loss function. Minimize to obtain the optimized data prediction model.

[0098] The data evaluation model is a BP neural network model, using... Represents the i-th training sample (x) i ,y i The evaluation value of ).

[0099] The data prediction model is a ResNet model, using... This represents the optimized data prediction model, the optimization objective of which is to make the weighted loss function... Minimize to obtain the optimized data prediction model;

[0100] The data evaluation model is a BP neural network model, using... Represents the i-th training sample (x) i ,y i The evaluation value of );

[0101] The optimization objective of the data prediction model is characterized by the following expression:

[0102] ;

[0103] in, This refers to the optimized data prediction model, where the optimization objective is to minimize the loss function value. Compared with sample evaluation value The product of x is minimized. i The input to the data prediction model includes the i-th ship image data, the loss value and accuracy corresponding to the i-th ship image data, y. i α represents the ship category label value corresponding to the i-th ship image data; α represents the learning rate of the data prediction model; β represents the learning rate of the data evaluation model; θ and φ represent the weight parameters of each layer of the data prediction model and the data evaluation model participating in the forward propagation calculation, respectively.

[0104] The following expression represents the update process of weights and parameters θ and φ in each iteration:

[0105] ,

[0106] ,

[0107] Where α represents the learning rate of the data prediction model; β represents the learning rate of the data evaluation model; B p This refers to the number of data points contained in a batch during training. p is a subscript indicating that the batch of data is used for the prediction model. This indicates that the training will be conducted in one batch. express One piece of data in the data, where B s This represents a batch of data during the verification process, where s indicates a correct or incorrect verification result (e.g., 0 for correct and 1 for incorrect). This refers to the validation result of the m-th data in the current summation loop, where m is a positive integer, representing 1, 2, ..., n; L represents the number of parameters in the model; This represents the image data of the training set; This represents the data labeling of the training set, k indicates that the summation operation has been performed to the kth round; v indicates that this data is used to validate the model; h is the sample evaluation value. , x i y represents the input to the data prediction model, including the i-th ship image data, the corresponding loss value and accuracy, and y'. i This represents the ship category label value corresponding to the i-th ship image data;

[0108] m represents the m-th round of the summation operation; L represents the number of dimensions of the input data to the prediction model in one iteration.

[0109] θ and φ represent the weight parameters of each layer of the data prediction model and the data evaluation model, respectively, for the forward propagation calculation.

[0110] The KL divergence between each ship image in the augmented dataset and the corresponding original ship image in the dataset to be augmented is calculated using the following expression:

[0111] ;

[0112] in, Let P(x) represent the KL divergence between the i-th ship image data and the corresponding original ship image data in the dataset to be augmented, where i is a positive integer, specifically 1, 2, ..., N; P(x) is the evaluation probability distribution of the augmented dataset, and Q(x) is the evaluation probability distribution of the dataset to be augmented, i.e., the original dataset.

[0113] If the model accuracy of the data prediction model trained using the augmented dataset is greater than a specified value, then the augmented dataset is deemed to be effective in restoring the quality of the original ship image data. The specified value is determined by the model accuracy of the data prediction model obtained by training the data prediction model using the training dataset.

[0114] The data prediction model is re-initialized, and the reinforcement learning process of the data evaluation model is completed according to the specified number of iterations to obtain the optimized data evaluation model.

[0115] When the data prediction model is initialized again, the weight parameters of each layer of the network in the data prediction model are randomly generated. The steps of initializing the data prediction model and calculating the evaluation value of each ship image data are repeated according to the specified number of iterations until the specified number of iterations is completed, thus completing the reinforcement learning process of the data prediction model.

[0116] Single ship image data (x i ,y i The image data of this single ship (x) i ,y i The corresponding loss value The accuracy μ is input into the data evaluation model to obtain the evaluation value h(x) of the single ship image data. i ,y i The evaluation value h(x) of the single ship image data i ,y i ) is an n-dimensional array generated iteratively by the data evaluation model, or a double-precision floating-point number between 0 and 1.

[0117] Example 3:

[0118] Figure 5 This is a schematic diagram of an embodiment of an electronic device according to the present invention.

[0119] like Figure 5 As shown, the electronic device is embodied in the form of a general-purpose computing device. There can be one or more processors working collaboratively. This invention also does not preclude distributed processing, meaning that processors can be distributed across different physical devices. The electronic device of this invention is not limited to a single entity, but can also be the sum of multiple physical devices.

[0120] The memory stores a computer-executable program, typically machine-readable code. The computer-readable program can be executed by the processor to enable the electronic device to perform the method of the present invention, or at least some steps of the method.

[0121] The memory includes volatile memory, such as random access memory (RAM) and / or cache memory, and may also be non-volatile memory, such as read-only memory (ROM).

[0122] Optionally, in this embodiment, the electronic device further includes an I / O interface for exchanging data with external devices. The I / O interface can represent one or more of several bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0123] It should be understood that Figure 5 The illustrated electronic device is merely one example of the present invention, and the electronic device of the present invention may also include elements or components not shown in the above examples. For example, some electronic devices also include display units such as displays, and some electronic devices also include human-computer interaction elements such as buttons and keyboards. Any electronic device capable of executing a computer-readable program in memory to implement the method of the present invention or at least some steps of the method can be considered as an electronic device covered by the present invention.

[0124] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software, or by combining software with necessary hardware. Therefore, as... Figure 6 As shown, the technical solution according to the embodiments of the present invention can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, mobile hard drive, etc.) or on a network, and includes several commands to cause a computing device (such as a personal computer, server, or network device, etc.) to execute the above-described method according to the embodiments of the present invention.

[0125] The software product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0126] The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting programs for use by or in connection with a command execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0127] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0128] The aforementioned computer-readable medium carries one or more programs (e.g., computer-executable programs) that, when executed by a device, cause the computer-readable medium to implement the methods of this disclosure.

[0129] Those skilled in the art will understand that the above modules can be distributed in the device as described in the embodiments, or they can be modified accordingly and placed in one or more devices that are unique to this embodiment. The modules in the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0130] Through the description of the above embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions of the embodiments of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, portable hard drive, etc.) or on a network, including several commands to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of the present invention.

[0131] Exemplary embodiments of the present invention have been specifically shown and described above. It should be understood that the present invention is not limited to the detailed structures, arrangements, or implementations described herein; rather, the present invention is intended to cover various modifications and equivalent arrangements contained within the spirit and scope of the appended claims.

Claims

1. A reinforcement learning-based augmented data quality evaluation method, characterized by, Includes the following steps: The dataset to be augmented is divided into a training set and a test set. The training set is augmented using an augmentation method to obtain the augmented dataset. A portion of the data in the augmented dataset is then used as a validation set. The dataset to be augmented includes ship image data labeled with ship category tags. A data evaluation reinforcement learning model is established, comprising a data prediction model and a data evaluation model. The data prediction model and the data evaluation model are then optimized by performing the following steps: Initialize the data prediction model; input the data prediction model with a pre-set batch of ship image data training set, record the loss value generated by each ship image data, use the validation set to verify the accuracy of the data prediction model for each ship image data, and optimize the data prediction model according to the optimization objective; Each ship image data, along with its corresponding loss value and accuracy, is input into the data evaluation model to obtain the evaluation value for each ship image data. The data prediction model is re-initialized, and the reinforcement learning process of the data evaluation model is completed according to the specified number of iterations to obtain the optimized data evaluation model. Calculate the KL divergence between each ship image data in the to-be-augmented data set and the corresponding original ship image data in the to-be-augmented data set to determine the quality of each ship image data in the augmented data set; the data prediction model is a Resnet model, and the data prediction model is trained by using The optimized data prediction model is represented by f, and the optimization target of the data prediction model is to minimize the weighted loss function to obtain the optimized data prediction model. The data evaluation model is a BP neural network model, using... Indicates the first training samples Evaluation value; The optimization objective of the data prediction model is characterized by the following expression: in, This refers to the optimized data prediction model, where the optimization objective is to minimize the loss function value. Compared with sample evaluation value The product is the smallest. This represents the input to the data prediction model, including the first... Ship image data, the first The loss value and accuracy corresponding to the ship image data. Indicates the first The ship category label value corresponding to the ship image data; θ and Φ represent the weight parameters of each layer of the data prediction model and data evaluation model participating in the forward propagation calculation, respectively.

2. The reinforcement learning-based augmented data quality assessment method according to claim 1, characterized in that, The following expression represents the update process of weights and parameters θ and Φ in each iteration: Where α represents the learning rate of the data prediction model; β represents the learning rate of the data evaluation model; This refers to the number of data points contained in a batch of data during the training process. p is a subscript marker, indicating that the batch of data is used for the prediction model. This indicates that the training will be conducted in one batch. express One piece of data in which This represents a batch of data during the verification process, where s represents the correct or incorrect verification result. This refers to the verification result of the data obtained in the m-th round of the current summation loop, where m is a positive integer, and m is 1, 2, ..., n; This represents the image data of the training set; This represents the data labeling of the training set, where k indicates that the summation operation has been performed up to the kth round; v indicates that this data is used to validate the model; and h is the sample evaluation value. L represents the number of dimensions of the input data for the prediction model.

3. The reinforcement learning-based augmented data quality assessment method according to claim 1, characterized in that, Use the following expression to calculate the KL divergence between each ship image in the augmented dataset and the corresponding original ship image in the dataset to be augmented: in, Let represent the KL divergence between the i-th ship image data and the corresponding original ship image data in the dataset to be augmented, where i is a positive integer, specifically 1, 2, ..., N; To augment the quality probability distribution of the dataset, The dataset to be augmented is the quality probability distribution of the original dataset.

4. The reinforcement learning-based augmented data quality assessment method according to claim 1, characterized in that, If the model accuracy of the data prediction model trained using the augmented dataset is greater than a specified value, then the augmented dataset is deemed to be effective in restoring the quality of the original ship image data. The specified value is determined by the model accuracy of the data prediction model obtained by training the data prediction model using the training dataset.

5. The reinforcement learning-based augmented data quality assessment method according to claim 1, characterized in that, Based on the application scenario of ship image classification, a specified hyperparameter and a specified number of iterations are selected. The specified hyperparameter is in the range of 50 to 400 iterations, and the specified number of iterations is in the range of 50 to 200 iterations.

6. The reinforcement learning-based augmented data quality assessment method according to claim 1 or 5, characterized in that, The process of re-initializing the data prediction model and completing the reinforcement learning process of the data evaluation model according to a specified number of iterations to obtain the optimized data evaluation model includes: When the data prediction model is initialized again, the weight parameters of each layer of the network in the data prediction model are randomly generated, and the iteration process is repeated according to the specified number of iterations until the specified number of iterations is completed, thus completing the reinforcement learning process of the data prediction model.

7. The reinforcement learning-based augmented data quality assessment method according to claim 6, characterized in that, Single ship image data The image data of this single ship The corresponding loss value λ and accuracy μ are input together into the data evaluation model to obtain the evaluation value of the single ship image data. The evaluation value of the single ship image data It is a double-precision floating-point number between 0 and 1.

8. A reinforcement learning-based augmented data quality assessment system, characterized in that, include: The data processing module divides the dataset to be augmented into a training set and a test set, uses an augmentation method to augment the training set to obtain an augmented dataset, and divides a portion of the data from the augmented dataset as a validation set. The dataset to be augmented includes ship image data labeled with ship category tags. The model building module establishes a data evaluation reinforcement learning model, which includes a data prediction model and a data evaluation model. Optimizing the data prediction model and the data evaluation model specifically includes: Initialize the data prediction model; input a pre-set batch of ship image data into the data prediction model, record the loss value generated by each ship image data, use the validation set to verify the accuracy of the data prediction model for each ship image data, and optimize the data prediction model according to the optimization objective; Each ship image data, along with its corresponding loss value and accuracy, is input into the data evaluation model to obtain the evaluation value for each ship image data. The data prediction model is re-initialized, and the reinforcement learning process of the data evaluation model is completed according to the specified number of iterations to obtain the optimized data evaluation model and the evaluation probability distribution map. The calculation module calculates the KL divergence between each ship image data in the dataset to be augmented and the corresponding original ship image data in the dataset to be augmented, based on the obtained evaluation probability distribution map, so as to determine the quality of each ship image data in the augmented dataset. The data prediction model is a ResNet model, using... This indicates that the optimization objective of the data prediction model is to optimize the weighted loss function. Minimize to obtain the optimized data prediction model; The data evaluation model is a BP neural network model, using... Represents the i-th training sample Evaluation value; The optimization objective of the data prediction model is represented by the following expression: in, This refers to the optimized data prediction model, where the optimization objective is to minimize the loss function value. Compared with sample evaluation value The product is minimized; θ and Φ represent the weight parameters of each layer of the data prediction model and data evaluation model participating in the forward propagation calculation, respectively.

Images