Training data augmentation method, device, equipment and storage medium

Abstract

The present application provides a training data augmentation method, device, equipment and storage medium, which relates to the field of artificial intelligence technology and can be applied to various scenarios such as cloud technology, artificial intelligence, smart transportation, and assisted driving. The method includes obtaining training sample data and the current iteration number of neural network model training; selecting a preset number of candidate strategy sets corresponding to the training sample data from multiple augmentation strategy sets; obtaining the total number of strategies and the corresponding lower limit values ​​of strategies for each preset number of candidate strategy sets; determining the number of reference strategies corresponding to each candidate strategy set according to the total number of strategies, the lower limit value of strategies, the current iteration number and the preset iteration number; respectively selecting target augmentation strategies corresponding to the number of reference strategies from each candidate strategy set; augmenting the training sample data based on the selected target augmentation strategy to obtain augmented sample data. The present application can significantly improve the efficiency of data augmentation.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a training data augmentation method, apparatus, device and storage medium. Background Art

[0002] Data augmentation techniques, such as geometric transformation (random horizontal flip, rotation, cropping, etc.), color space enhancement (color jitter, etc.), etc., are widely used in training neural network models. They can increase the number of training data sets and enrich the diversity of data to alleviate the overfitting phenomenon of neural network models to data sets, so that the trained models have stronger generalization capabilities. For example, the pioneering work of automatic data augmentation, AutoAugment (AA), set up 25 data augmentation methods, used a recurrent neural network (RNN) controller and a reinforcement learning algorithm for policy search to optimize the selection process of the augmentation method, that is, predict which sub-strategies to select in each step of the RNN, and use the mean average precision (mAP) of the deep neural network model on the validation set as feedback, achieving excellent performance in tasks such as data classification and detection. However, the search space of AA is as high as dozens of orders of magnitude, and the search process consumes a lot of resources and time. It takes about 5,000 GPU hours to search once, and the cost of policy search model training and policy selection is extremely high. To improve the efficiency of search, related technologies have proposed methods such as Population-Based Augmentation (PBA) and Fast AutoAugment (Fast AA). Although the search speed is optimized compared to the AA method, it is still very slow. For example, PBA still takes 5 GPU hours to perform a search on the reduced CIFAR-10 dataset, resulting in low data augmentation efficiency, high resource usage and training cost of model training.

[0003] Therefore, it is necessary to provide an improved data augmentation solution to improve the efficiency of data augmentation and optimize the efficiency of model training. Summary of the invention

[0004] The present application provides a training data augmentation method, apparatus, device and storage medium, which can effectively improve data augmentation efficiency and reduce resource usage and cost.

[0005] On the one hand, the present application provides a method for training data augmentation, the method comprising:

[0006] Get the training sample data and the current iteration number of the neural network model training;

[0007] Selecting a preset number of candidate strategy sets corresponding to the training sample data from a plurality of augmentation strategy sets, wherein the augmentation strategy sets are obtained by classifying a plurality of augmentation strategies based on strategy attributes;

[0008] Obtaining the total number of strategies and the corresponding lower limit values ​​of strategies of the preset number of candidate strategy sets;

[0009] Determine the number of reference strategies corresponding to each candidate strategy set according to the total number of strategies, the strategy lower limit, the current number of iterations and the preset number of iterations;

[0010] Selecting target augmentation strategies corresponding to the number of reference strategies from each of the candidate strategy sets;

[0011] The training sample data is augmented based on the selected target augmentation strategy to obtain augmented sample data.

[0012] Another aspect provides a training data augmentation device, the device comprising:

[0013] Iteration number acquisition module: used to obtain the current iteration number of training sample data and neural network model training;

[0014] A strategy set selection module: used to select a preset number of candidate strategy sets corresponding to the training sample data from multiple augmented strategy sets, wherein the augmented strategy set is obtained by classifying multiple augmented strategies based on strategy attributes;

[0015] A strategy set parameter acquisition module: used to obtain the total number of strategies and the corresponding lower limit values ​​of strategies of the preset number of candidate strategy sets;

[0016] A strategy quantity determination module is used to determine the number of reference strategies corresponding to each candidate strategy set according to the total number of strategies, the strategy lower limit, the current iteration number and the preset iteration number;

[0017] An augmentation strategy selection module: used to select target augmentation strategies corresponding to the number of reference strategies from each of the candidate strategy sets;

[0018] Augmentation processing module: used to perform augmentation processing on the training sample data based on the selected target augmentation strategy to obtain augmented sample data.

[0019] On the other hand, a training data augmentation device is provided, the device comprising a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the training data augmentation method as described above.

[0020] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction or at least one program is stored in the storage medium, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the training data augmentation method as described above.

[0021] On the other hand, a training data augmentation terminal is provided, the terminal includes a processor and a memory, the device includes a processor and a memory, the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the training data augmentation method as described above.

[0022] On the other hand, a server is provided, the server includes a processor and a memory, the device includes a processor and a memory, the memory stores at least one instruction or at least one program, the at least one instruction or the at least one program is loaded and executed by the processor to implement the training data augmentation method as described above.

[0023] On the other hand, a computer program product or a computer program is provided, which includes computer instructions, and when the computer instructions are executed by a processor, the training data augmentation method as described above is implemented.

[0024] The training data augmentation method, apparatus, device, storage medium, terminal, server and computer program product provided in this application have the following technical effects:

[0025] The technical solution of the present application first obtains the training sample data and the current iteration number of the neural network model training, and selects a preset number of candidate strategy sets corresponding to the training sample data from multiple augmentation strategy sets, wherein the augmentation strategy set is obtained by pre-classifying multiple augmentation strategies based on strategy attributes; then obtains the total number of strategies and the corresponding lower limit values ​​of strategies of the preset number of candidate strategy sets, and determines the number of reference strategies corresponding to each candidate strategy set according to the total number of strategies, the lower limit value of strategies, the current number of iterations and the preset number of iterations, and then selects the target augmentation strategy corresponding to the number of reference strategies from each candidate strategy set, so as to perform augmentation processing on the training sample data based on the selected target augmentation strategy, and obtain augmented sample data. In the input data preprocessing process of neural network training, the scheme groups the augmentation strategies, and uses an adaptive method to adaptively adjust the number of augmentation strategies used, so as to realize adaptive augmentation of data, and ensures excellent augmentation effect and model training effect without searching for augmentation strategies, while saving search time and significantly reducing augmentation cost and training resource occupation. In addition, this solution only processes the training sample data and does not involve model modification. It has a wide range of applications and is not limited to data set categories. For example, it can be applied to almost all image detection models, and the method has strong generalization capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0027] Figure 1 is a schematic diagram of an application environment provided by an embodiment of the present application;

[0028] Figure 2 It is a flowchart of a training data augmentation method provided in an embodiment of the present application;

[0029] Figure 3 It is a visualization effect diagram of various augmentation strategies in Table 1 provided in the embodiments of the present application;

[0030] Figure 4 It is a flowchart of another training data augmentation method provided in an embodiment of the present application;

[0031] Figure 5 It is a flowchart of another training data augmentation method provided in an embodiment of the present application;

[0032] Figure 6 It is a structural schematic diagram of a training data augmentation device provided in an embodiment of the present application;

[0033] Figure 7 It is a hardware structure block diagram of an electronic device for a training data augmentation method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0035] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, mechatronics and other technologies. Artificial intelligence software technologies mainly include computer vision technology, speech processing technology, natural language processing technology, as well as machine learning / deep learning, autonomous driving, smart transportation and other major directions.

[0036] Among them, machine learning (ML) is a multi-disciplinary interdisciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0037] Computer Vision (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to machine vision such as using cameras and computers to replace human eyes to identify and measure targets, and further perform graphic processing so that computer processing becomes an image that is more suitable for human eye observation or transmission to instrument detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and map construction, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0038] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, drones, robots, smart medical care, smart customer service, Internet of Vehicles, automatic driving, smart transportation, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role. This application mainly involves technologies such as training data augmentation, semantic understanding, machine learning and deep learning, which are specifically explained through the following embodiments.

[0039] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0040] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or server comprising a series of steps or submodules is not necessarily limited to those steps or submodules clearly listed, but may include other steps or submodules that are not clearly listed or inherent to these processes, methods, products or devices.

[0041] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0042] Adaptation: refers to the process of automatically adjusting the processing method, processing sequence, processing parameters, boundary conditions or constraints according to the data characteristics of the processed data during the processing and analysis process, so that they are adapted to the statistical distribution characteristics and structural characteristics of the processed data to achieve the best processing effect.

[0043] Data Augmentation: refers to the process of increasing the value of basic data by deriving information from internal or external data resources.

[0044] Mean Average Precision (mAP): An indicator for measuring recognition accuracy in object detection. In multiple category object detection, a curve can be drawn for each category based on recall and precision. AP is the area under the curve, and mAP means averaging the AP of each category.

[0045] See also Figure 1 , Figure 1 is a schematic diagram of an application environment provided by an embodiment of the present application, such as Figure 1As shown, the application environment may include at least a server 01 and a terminal 02. In actual applications, the server 01 and the terminal 02 may be directly or indirectly connected via wired or wireless communication to achieve interaction between the terminal 02 and the server 01, and this application does not limit this.

[0046] In the embodiment of the present application, server 01 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Specifically, the server may include physical devices, which may specifically include network communication submodules, processors, and memories, etc., and may also include software running in physical devices, which may specifically include applications, etc.

[0047] Specifically, cloud technology refers to a hosting technology that unifies hardware, software, network and other resources in a wide area network or local area network to achieve data computing, storage, processing and sharing. It distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space and information services as needed. The network that provides resources is called "cloud". Among them, artificial intelligence cloud service is generally also called AIaaS (AI as a Service, Chinese for "AI as a Service"). This is a mainstream service mode of artificial intelligence platform. Specifically, AIaaS platform will split several common AI services and provide independent or packaged services in the cloud. This service model is similar to opening an AI theme mall: all developers can access one or more artificial intelligence services provided by the platform through API interfaces. Some senior developers can also use the AI ​​framework and AI infrastructure provided by the platform to deploy and operate their own cloud artificial intelligence services.

[0048] In the embodiment of the present application, the terminal 02 may include physical devices such as smart phones, desktop computers, tablet computers, laptop computers, digital assistants, augmented reality (AR) / virtual reality (VR) devices, intelligent voice interaction devices, smart home appliances, smart wearable devices, and vehicle-mounted terminal devices, and may also include software running in physical devices, such as applications. Specifically, the terminal 02 may send a training data augmentation request or a model training request to the server 01 to trigger the server 01 to perform corresponding operations. The server 01 may be used to receive a training data augmentation request or a model training request sent by the terminal, trigger data augmentation processing in response to a training data augmentation request, or perform training of a neural network model in response to a model training request, and perform data augmentation during the training process.

[0049] Furthermore, it is understandable that Figure 1 What is shown is only an application environment of a training data augmentation method, and the application environment may include more or fewer nodes, and this application does not make any limitation here.

[0050] The application environment involved in the embodiment of the present application, or the server of the local server 02 or the cloud server 03 in the application environment, and the terminal 01, can be a distributed system formed by connecting a client and multiple nodes (any form of computing devices in the access network, such as servers and user terminals) through network communication. The distributed system can be a blockchain system, which can provide the above-mentioned data augmentation services, model training services, data storage services, etc.

[0051] The following is an introduction to a training data augmentation method of the present application based on the above application environment, which is applied to the server or terminal. The embodiments of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, etc. The technical solution of the present application can be used for various types of model training data augmentation, such as data augmentation of almost all types of image detection models. In the data preprocessing stage of model training, such as the image preprocessing stage, the data preprocessing using the technical solution of the present application can improve the evaluation indicators of the model, such as the mAP indicator of the image detection model. Please refer to Figure 2 , Figure 2It is a flowchart of a training data augmentation method provided in an embodiment of the present application. This specification provides method operation steps such as the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or server product is executed, it can be executed in the order shown in the embodiment or the accompanying drawings or in parallel (for example, in a parallel processor or multi-threaded processing environment). Specifically, Figure 2 As shown, the method may include:

[0052] S201: Obtain training sample data and the current iteration number of neural network model training.

[0053] Among them, the current iteration number refers to the number of iteration trainings that need to be performed during the training of the neural network model. For example, if the first iteration training is required, the current iteration number is 1, and if the second iteration training is required, the current iteration number is 2. During the training of the neural network model, data augmentation is performed on the training sample set before each iteration, or data augmentation is performed on the training sample set at intervals, such as performing data augmentation once every two iterations. The training sample set includes multiple training sample data, and the training sample data may include but is not limited to sample images and sample texts. In scenarios such as semi-supervised and supervised learning, the training sample set also includes labels corresponding to the training sample data. For example, in the field of image detection, the training sample data is a sample image, and the label is an image detection frame, which is used to frame the target detection object in the sample image.

[0054] S203: Selecting a preset number of candidate strategy sets corresponding to the training sample data from the multiple augmented strategy sets.

[0055] Specifically, a preset number of candidate strategy sets can be randomly selected from multiple augmentation strategy sets, that is, the candidate strategy sets are selected from each augmentation strategy set with equal probability. Among them, the augmentation strategy set is obtained by classifying multiple augmentation strategies based on strategy attributes, and the preset number is greater than or equal to 1, which can be set according to task requirements. The strategy attribute represents the augmentation characteristics of the augmentation strategy, and may include but is not limited to parameter attributes that represent whether the augmentation parameters are automatically generated, exclusive attributes that represent whether they can be used in combination with other augmentation strategies, or augmentation effect attributes that represent the strength of the augmentation effect. Grouping the augmentation strategies to be selected based on the strategy attributes and realizing the classified selection of strategies can reduce the selection cost in each strategy selection process, thereby reducing resource usage.

[0056] Different augmentation strategies are used in different data types or different task requirements. For example, for sample image data, augmentation strategies may include but are not limited to rotation transformation, flip transformation, scaling transformation, translation transformation, scale transformation, region cropping, adding noise, segmented affine, random masking, boundary detection, contrast transformation, color jitter, random mixing and composite superposition, etc. For sample text data, augmentation strategies may include but are not limited to word replacement, back translation, adding text noise and text segment deletion, etc. In some cases, the same augmentation strategy may have multiple strategy attributes at the same time. Based on the priority of each strategy attribute, the augmentation strategy is classified into the augmentation strategy set corresponding to the strategy attribute with the highest priority.

[0057] In an optional implementation, taking the augmentation strategy in the following Table 1 as an example, the strategy attributes used for classification include parameter attributes, exclusive attributes, and augmentation effect attributes. Table 1 shows the augmentation strategy name, augmentation method description, strategy set identifier, and augmentation parameter threshold. The augmentation parameter threshold includes the parameter upper limit value and the parameter lower limit value. The strategy set identifier represents the augmentation strategy set to which the augmentation strategy belongs. Each augmentation strategy set corresponds to a strategy attribute; the strategy attribute of augmentation strategy set A is automatic generation of augmentation parameters, that is, its augmentation parameters are automatically determined by the augmentation strategy based on the input sample image, without the need to set and adjust. The strategy attribute of augmentation strategy set B is non-automatic generation of augmentation parameters, that is, the augmentation parameters need to be set before the execution of its augmentation strategy. The strategy attribute of augmentation strategy set C is to operate only on the detection box (bbox). The strategy attribute of augmentation strategy set D is a strong augmentation method, which belongs to the augmentation effect attribute. Among them, the priority of operating only on the detection box (bbox) and the strong augmentation method is higher than that of automatically generating augmentation parameters and non-automatically generating augmentation parameters.

[0058] In some tasks, for augmentation strategy set C, the bbox situation is more complicated, with multiple bboxes occluding and nesting each other, which is not suitable for combination with other augmentation methods in the set. For augmentation strategy set D, the augmentation strategies are all strong augmentation, which is not suitable for combination with other augmentation methods in the set, that is, operations on the detection box (bbox) and strong augmentation methods are also exclusive attributes. Or, in some cases, there are augmentation strategies in the augmentation strategy set that are not suitable for combination with any other augmentation strategies.

[0059] Table 1

[0060]

[0061]

[0062] Accordingly, please refer to Figure 3 , Figure 3 The figure shows the visualization effects of various augmentation strategies in Table 1.

[0063] S205: Obtain the total number of strategies and the corresponding lower limit values ​​of strategies of a preset number of candidate strategy sets.

[0064] The total number of strategies refers to the number of augmented strategies included in the candidate strategy set, such as the total number of strategies of augmented strategy set A is 4, and the strategy lower limit value indicates the minimum number of target augmented strategies to be selected from the corresponding candidate strategy set, such as 1. It can be understood that the strategy lower limits corresponding to different augmented strategy sets can be the same or different, such as all 1, or the strategy lower limit value of the augmented strategy set with exclusive attributes is 1, and the strategy lower limit value of the augmented strategy set with non-exclusive attributes is greater than 1, etc. In some cases, a strategy upper limit value can also be set to indicate the maximum number of target augmented strategies to be selected from the corresponding candidate strategy set, and the strategy upper limit value is less than or equal to the corresponding total number of strategies.

[0065] S207: Determine the number of reference strategies corresponding to each candidate strategy set according to the total number of strategies, the strategy lower limit, the current iteration number and the preset iteration number.

[0066] The preset number of iterations may refer to the total number of iterations in the neural network model training process. After completing the iterative training of the preset number of iterations, the model training process ends. Alternatively, when judging whether the model converges based on the model loss, the preset number of iterations may also be the preset upper limit of the number of iterations, that is, before the current number of iterations reaches the preset number of iterations, the model loss has met the model convergence condition. The number of reference strategies is the number of augmentation strategies to be selected from the corresponding candidate strategy set. For example, if the number of reference strategies in augmentation strategy set A is 2, then 2 augmentation strategies are selected from it.

[0067] When the policy attributes of the candidate policy set are not exclusive, the policy selection for the candidate policy set adopts an adaptive calculation method, and the number of reference policies decreases as the current iteration number increases. For example, for the above-mentioned augmented policy set A and augmented policy set B, an adaptive calculation method is used to determine the number of reference policies. In this way, as the model training process progresses, the number of augmented policies can be adaptively adjusted, and then the data volume and data difference of the training sample set can be adjusted, while avoiding overfitting, accelerating the model convergence speed and ensuring training efficiency. Correspondingly, please refer to Figure 4 , S207 may include the following steps.

[0068] S301: For each candidate strategy set, determine an adaptive factor corresponding to the current iteration according to the ratio between the current iteration number and the preset iteration number.

[0069] Specifically, the adaptive factor is an adaptive value, which changes dynamically with the current number of iterations during the training of the neural network model, thereby making the number of reference strategies calculated based on the adaptive factor also change adaptively. The adaptive factor has a value range greater than zero and less than or equal to 1, and the adaptive factor decreases with the increase of the number of training iterations, and the number of reference strategies decreases with the increase of the adaptive factor.

[0070] In an optional implementation, the calculation formula of the adaptive factor is as follows, wherein Factor is the adaptive factor, cur_epoch is the current iteration number, and total_epoch is the preset iteration number.

[0071] Factor=cur_epoch / total_epoch

[0072] In some embodiments, when judging whether the model has converged based on the model loss, there is a situation where the model loss has satisfied the model convergence condition before the current iteration number reaches the preset iteration number, and during the training process of the neural network model, the model loss keeps changing, and then the adaptive factor is determined in combination with the change in the current iteration number and the change in the model loss. Accordingly, before determining the adaptive factor, the method also includes S401: obtaining the model loss of the first iteration training of the neural network model and the model loss of the previous iteration training. The model loss of the first iteration training is usually large, and as the number of iterations increases, the overall change of the model loss tends to gradually decrease. Of course, an abnormal situation of increasing may also occur. After obtaining the model loss of the first iteration training and the model loss of the previous iteration training, S301 can specifically include the following steps.

[0073] S3011: Determine the coefficient of variation based on the model loss of the previous iteration training and the model loss of the first iteration training.

[0074] S3012: Performing a mutation process on the ratio between the current iteration number and the preset iteration number based on the coefficient of variation to obtain an adaptive factor.

[0075] Among them, the value range of the coefficient of variation is greater than zero and less than or equal to 1. The coefficient of variation decreases as the model loss of the previous iterative training decreases, thereby playing a role in improving the adaptive factor, and increases as the model loss increases, thereby playing a role in calling back the growth of the adaptive factor. In an optional embodiment, the coefficient of variation k is the ratio of the model loss of the previous iterative training to the model loss of the first iterative training, as shown in the following formula, wherein cur_loss is the model loss of the previous iterative training, init_loss is the model loss of the first iterative training, and when the training data set is augmented in the first iterative training, k is set to 1.

[0076] k = cur_loss / init_loss

[0077] Correspondingly, the calculation formula of the adaptive factor is as follows.

[0078] Factor=(cur_epoch / total_epoch)^(cur__loss / init_loss)

[0079] In this way, the adaptive factor is adaptively changed in combination with the number of iterations and the change of model loss, and the diversity of the adaptive change of the number of reference strategies is increased. When the model loss decreases, the adaptive factor value can be improved in coordination with the current number of iterations, thereby reducing the number of reference strategies and improving the model training efficiency. On the contrary, when the model loss increases abnormally, the current number of iterations is still increasing, thereby playing a role in improving the adaptive factor. Due to the increase in model loss, the coefficient of variation of the current iteration also increases compared to the coefficient of variation of the previous iteration, which can call back the growth of the adaptive factor to a certain extent, reduce the growth rate of the adaptive factor, and then reduce the reduction rate of the reference strategy amount, thereby ensuring data augmentation to improve the model training effect.

[0080] S303: Perform an adaptive operation to increase the number of strategies according to the adaptive factor, the total number of strategies and the lower limit of the strategies, and obtain the number of reference strategies corresponding to each candidate strategy set.

[0081] Specifically, the larger the adaptive factor is, the smaller the number of reference strategies is. The adaptive operation here may include but is not limited to a cosine adaptive method, a linear adaptive method or a sine adaptive method.

[0082] In some embodiments, the cosine adaptive method is used to perform adaptive calculations on the number of reference strategies. Specifically, the total number of strategies and the lower limit of the strategies are subtracted to obtain the difference in the number of strategies; the preset cosine descent adaptive method is called to adaptively calculate the lower limit of the strategies, the difference in the number of strategies and the adaptive factor to obtain the number of reference strategies corresponding to each candidate strategy set. In an optional implementation, the adaptive calculation formula for the number of reference strategies is shown below. Wherein, N is the number of reference strategies, Cos is the cosine function, and pi is ∏; end represents the minimum value of the adaptive range, and for the number of reference strategies, it represents the lower limit of the strategy, such as 1; Start represents the maximum value of the adaptive range, and for the number of reference strategies, Start is the total number of strategies in the current candidate strategy set. Specifically, the number of reference strategies is an integer. It can be understood that if the calculated N is not an integer, it can be rounded up, rounded down, or rounded off to obtain an integer result.

[0083] N=end+0.5*(start-end)*(cos(pi*factor)+1)

[0084] S209: Select target augmentation strategies corresponding to the number of reference strategies from each candidate strategy set.

[0085] Specifically, after determining the corresponding number of reference strategies for each candidate strategy set, select the target augmentation strategies of the reference strategy number from the candidate strategy set. For example, if the candidate strategy set determined is A and the number of reference strategies is 2, then select 2 augmentation strategies from the candidate strategy set. Specifically, the augmentation strategy can be selected by random equal probability extraction.

[0086] In some cases, when the policy attribute of the candidate policy set is exclusive, the augmentation strategy therein may not be able to be combined with other augmentation strategies in the set, or cannot be combined with any other augmentation strategy. Accordingly, for the candidate policy set with exclusive attributes, the number of reference strategies is not calculated in an adaptive manner. Before S205, the method also includes: obtaining the policy attribute of each candidate policy set, and then determining whether the policy attribute is an exclusive attribute. If the policy attribute is an exclusive attribute, select a candidate augmentation strategy from the corresponding candidate policy set as the corresponding target augmentation strategy. When the preset number is 1, that is, only one candidate policy set is selected, directly select a target augmentation strategy from it, and then execute step S211.

[0087] When the preset number is greater than 1 and the exclusive attribute is that it cannot be combined with any other augmented strategy, an augmented strategy is selected from the candidate strategy set, and the number of reference strategies of other candidate strategy sets is assumed to be 0. When the preset number is greater than 1 and the exclusive attribute is that it cannot be combined with other augmented strategies in the set, an augmented strategy is selected from the candidate strategy set, and augmented strategies corresponding to the number of reference strategies are selected from other candidate strategy sets, so that the selected augmented strategy is used as the target augmented strategy; specifically, after selecting each target augmented strategy, each augmented strategy can be combined in pairs to determine whether it can be combined. If there is a strategy that cannot be combined, one is selected and abandoned. For example, if both strategies are exclusive strategies, one is randomly selected and abandoned; or, if one is a non-exclusive strategy and the other is an exclusive strategy, when the exclusive strategy cannot be combined with any other augmented strategy, one is randomly selected and abandoned. When the exclusive strategy cannot be combined with other augmented strategies in the set but can be combined with the non-exclusive strategy, both strategies can be retained at the same time.

[0088] S211: Perform augmentation processing on the training sample data based on the selected target augmentation strategy to obtain augmented sample data.

[0089] Specifically, the training sample data is augmented by selecting at least one target augmentation strategy to obtain augmented data, such as Figure 3 In the case where there are multiple target augmentation strategies, the augmentation process can be performed based on a random order, or sequentially according to the selection order, or can also be performed based on the priority of each target augmentation strategy in descending order of priority.

[0090] Specifically, when the policy attributes of the target augmentation strategy are automatically generated parameter attributes, such as the aforementioned augmentation strategy set A, data augmentation processing can be performed directly. When the policy attributes of the target augmentation strategy are non-automatically generated parameter attributes, S211 includes the following steps.

[0091] S601: Obtain an augmentation parameter threshold, a discretization number of parameters, and a lower limit value of the discretization number of the target augmentation strategy.

[0092] S603: Determine a target augmentation parameter corresponding to the target augmentation strategy according to the adaptive factor, the number of discretized parameters, the lower limit of the number of discrete parameters, and the augmentation parameter threshold.

[0093] Specifically, the augmentation parameter threshold includes an upper parameter limit and a lower parameter limit, such as the parameter values ​​of various augmentation strategies shown in Table 1 above; the number of parameter discretization copies is the number of copies for discretization processing of the augmentation parameter range of the augmentation strategy, such as 10 copies. Taking Solarize as an example, its parameter range of 0-256 is discretized into 10 copies; the lower limit of the number of discrete copies is the minimum discrete value that can be taken for discrete processing, such as 1.

[0094] For some examples, please refer to Figure 5 , S603 may include the following steps.

[0095] S6031: Perform adaptive calculation of augmentation strength according to the adaptive factor, the number of discretized parameters and the lower limit of the number of discrete parameters to obtain a reference augmentation strength corresponding to the target augmentation strategy.

[0096] S6032: Based on the augmentation parameter threshold and the number of parameter discretizations, intensity mapping is performed on the reference augmentation intensity to obtain a target augmentation parameter corresponding to the target augmentation strategy.

[0097] Specifically, the adaptive factor applied to the reference augmentation quantity calculation and the adaptive factor used for the target augmentation parameter calculation can be the same or different, and the adaptive calculation formulas of the reference augmentation strength and the reference augmentation quantity can be the same or different. Similarly, as the current number of iterations increases, or the model loss decreases, the adaptive factor increases, the reference augmentation strength becomes smaller, and then the target augmentation parameter obtained by the adaptive calculation becomes smaller, and the augmentation degree using the corresponding augmentation strategy becomes lower. The adaptive calculation here can include but is not limited to the cosine adaptive method, the linear adaptive method or the sine adaptive method, etc.

[0098] In an optional embodiment, the reference augmentation intensity is adaptively calculated using a cosine adaptive method, and the adaptive calculation formula used is similar to the adaptive calculation formula of the aforementioned reference augmentation quantity, with the difference that, M is the reference augmentation intensity, End represents the minimum value of the adaptive range, representing the lower limit of the discrete number, such as 1; Start represents the maximum value of the adaptive range, representing the number of discrete parameters.

[0099] M=end+0.5*(start-end)*(cos(pi*factor)+1)

[0100] Specifically, the reference augmentation strength represents the discretization value corresponding to the current target augmentation strategy. For example, the number of discretization portions of the parameter is 10, and accordingly, the lower limit of the number of discrete portions can be 1. The adaptive strength range is 1-10. If the calculated reference augmentation strength is 9, it means that 9 out of 10 portions are taken to calculate the target augmentation parameters.

[0101] Further, the augmented parameter range of the target augmented strategy is discretized according to the augmented parameter threshold and the number of parameter discretizations to obtain the range value corresponding to the unit parameter range, and then the range value corresponding to the unit parameter range is multiplied by the reference augmented strength to obtain the target augmented parameter corresponding to the target augmented strategy. The discretization operation here refers to discretizing the augmented parameter range ([parameter upper limit value, parameter lower limit value]) corresponding to the augmented parameter threshold (parameter upper limit value and parameter lower limit value) based on a preset discretization method to obtain each discretized range value, that is, the range value corresponding to the unit parameter range. For example, the discretization method is discrete uniform distribution, the lower limit value of the discrete number of portions can be 1, and the adaptive strength range is 1-10, then the augmented parameter range is divided into 10 portions, and the range value of each portion, that is, the unit parameter range, is (parameter upper limit value-parameter lower limit value) / 10; taking rotate_with_bboxes as an example, its parameter range of 0-30 is discretized into 10 portions, and the range value corresponding to the unit parameter range is 3. Furthermore, if the calculated reference augmentation intensity is 9, the target augmentation parameter is 27. Accordingly, the intensity mapping formula is as follows.

[0102] L = M*(parameter upper limit - parameter lower limit) / number of parameter discretizations

[0103] S605: Execute the target augmentation strategy on the training sample data based on the target augmentation parameters to obtain augmented sample data.

[0104] Specifically, for each target augmentation strategy, the input in the augmentation process is training sample data, including samples and corresponding labels, such as sample images and corresponding detection frames, and target augmentation parameters; the output is augmented sample data, including augmented samples and augmented labels, such as augmented images and augmented detection frames.

[0105] In this way, the augmentation strength is adaptively adjusted as the model training process progresses, which improves the model training efficiency, reduces resource usage and training costs while ensuring the augmentation effect.

[0106] In one embodiment, taking an image detection model as an example, the sample training data is a sample image and a detection frame. During the training process of the neural network model, the augmentation strategies to be selected are divided into P augmentation strategy sets, as shown in Table 1 above, where P is 4. Augmentation strategy set A uses a cosine adaptive method to calculate the reference augmentation quantity, and the augmentation parameters are automatically generated when the strategy is executed. Augmentation strategy set B uses a cosine adaptive method to calculate the reference augmentation quantity and reference augmentation strength. Augmentation strategy sets C and D use a cosine adaptive method to calculate the reference augmentation strength, and the reference strategy quantity is a preset value of 1. Then, for the training sample data in the training sample set, one candidate strategy set is selected from the P augmentation strategy sets with equal probability. When the candidate strategy set does not have an exclusive attribute, based on The aforementioned step S207 determines the number of reference strategies, and randomly selects the reference number of target augmentation strategies from the candidate strategy set with equal probability. When the candidate strategy set has a preset number of strategies, randomly selects the preset number of target augmentation strategies from the candidate strategy set with equal probability, and determines the reference augmentation strength in the manner of step S6031; for the target augmentation strategy that automatically generates augmentation parameters, directly input the training sample data; for the target augmentation strategy that does not automatically generate augmentation parameters, when performing augmentation processing, in addition to inputting the training sample data, it is also necessary to input the reference augmentation strength, execute the target augmentation strategy in the order of extraction, augment the training sample data, and obtain the augmented image and detection box for the current iterative training of the neural network model.

[0107] As mentioned above, the AutoAugment (AA) method defines 25 data augmentation strategies, using an RNN controller and a reinforcement learning algorithm. Each step of the RNN is used to predict which sub-strategies to select. The RNN strategy prediction model needs to be trained, and a strategy search is required before augmentation. The search space during the search is composed of: each data augmentation strategy set consists of K sub-strategy sets, each sub-strategy set contains N data augmentation strategies, and each data augmentation strategy contains two hyperparameters, which are the probability P of using the data augmentation strategy and the strength M of the data augmentation strategy. If P and M are discretized into 6 parts, then during the training process, a sub-strategy set is extracted each time, and the training images are augmented according to the augmentation strategies in the sub-strategy set. If K=5 and N=2 are set, the total search space size is: (25*6*6) (5*2) =3.5*10 29 . Accordingly, for 5k data extracted from the COCO dataset as the training set, the AA method was used to train on 400 TPUs for 8 hours, that is, 3200 TPU hours. Although AA has significantly improved the model detection effect, it consumes a lot of resources. In related work, the Randaugment (RA) method was used to set 14 augmentation strategies, the number of augmentation strategies is N, the augmentation strength is M, and the strength of all selected data augmentation strategies is set to a fixed strength M value, thereby reducing a lot of search space and compressing the search space to 10 2 level, and used the grid search method as a simplified search algorithm. However, the RA method still takes 5 GPU hours to search on the reduced CIFAR-10 dataset. In other words, the search speed is closely related to the search space. The larger the search space, the longer the search time tends to be. Please refer to Table 2 below, which shows the search space of different augmentation methods, as well as the number of GPU hours consumed for CIFAR-10 / 100 and the number of GPU hours consumed for ImageNet. It can be seen that the resource consumption and time cost of the relevant augmentation methods are difficult to meet the actual task requirements.

[0108] Table 2

[0109] Augmentation Methods AA FastAA PBA RA Search Space <![CDATA[10 32 ]]> <![CDATA[10 32 ]]> <![CDATA[10 61 ]]> <![CDATA[10 2 ]]> CIFAR-10 / 100 5000 3.5 5 33 ImageNet 15000 450 - 4750

[0110] The technical solution of the present application adopts a group adaptive method, and uses an adaptive method to calculate the number of reference strategies and the reference augmentation strength, such as using the adaptive calculation method of cosine descent, and then adaptively adjusts the number of augmentation strategies and the augmentation parameter strength during the model training process. There is no need to perform a network search process. Under the premise of ensuring the augmentation effect and model training effect, it avoids consuming a large amount of computing resources and significantly reduces the time cost.

[0111] Table 3

[0112] Augmentation strategy Backbone Baseline(mAP) Model mAP G4 RetinaNet (target detection algorithm) R50 37.4 38.7(+1.3) AA RetinaNet R50 36.9 38.4(+1.5) G4 FasterRCNN R50 38.4 39.6(+1.2)

[0113] Please refer to Table 3, which shows the mAP results of the models obtained by the AA method and the augmentation method of the present application, where G4 represents the data augmentation method of the present application. It can be seen from the data in the table that, compared with AA, although the baseline of G4 has been improved from 36.9 to 37.4, the model mAP has still been significantly improved after adding the augmentation strategy. In other words, by using the method of the present application for data augmentation, the mAP optimization effects of the obtained models RetinaNet R50 and FasterRCNN R50 are similar, but a lot of search time is saved.

[0114] The present application embodiment also provides a training data augmentation device 700, such as Figure 6 As shown, Figure 6 A schematic structural diagram of a training data augmentation device provided in an embodiment of the present application is shown, and the device may include the following modules.

[0115] Iteration number acquisition module 10: used to obtain the current iteration number of training sample data and neural network model training.

[0116] The strategy set selection module 20 is used to select a preset number of candidate strategy sets corresponding to the training sample data from multiple augmented strategy sets, wherein the augmented strategy set is obtained by classifying multiple augmented strategies based on strategy attributes.

[0117] The strategy set parameter acquisition module 30 is used to obtain the total number of strategies and the corresponding lower limit values ​​of strategies of a preset number of candidate strategy sets.

[0118] The strategy quantity determination module 40 is used to determine the number of reference strategies corresponding to each candidate strategy set according to the total number of strategies, the strategy lower limit, the current iteration number and the preset iteration number.

[0119] The augmentation strategy selection module 50 is used to select target augmentation strategies corresponding to the number of reference strategies from each candidate strategy set.

[0120] The augmentation processing module 60 is used to perform augmentation processing on the training sample data based on the selected target augmentation strategy to obtain augmented sample data.

[0121] In some embodiments, the policy quantity determination module 40 may include the following sub-modules.

[0122] Adaptive factor determination submodule: used to determine the adaptive factor corresponding to the current iteration for each candidate strategy set according to the ratio between the current iteration number and the preset iteration number. The value range of the adaptive factor is greater than zero and less than or equal to 1, and the adaptive factor decreases with the increase of the number of training iterations.

[0123] Strategy quantity calculation submodule: used to perform adaptive calculations on the number of augmented strategies based on the adaptive factor, the total number of strategies and the lower limit of strategies, and obtain the number of reference strategies corresponding to each candidate strategy set.

[0124] In some embodiments, the strategy quantity calculation submodule may include the following units.

[0125] Strategy quantity difference calculation unit: used to subtract the total number of strategies from the strategy lower limit value to obtain the strategy quantity difference.

[0126] Strategy quantity adaptive calculation unit: used to call the preset cosine descent adaptive method, perform adaptive calculations on the strategy lower limit value, strategy quantity difference and adaptive factor, and obtain the reference strategy quantity corresponding to each candidate strategy set.

[0127] In some embodiments, the augmentation processing module 60 may include the following sub-modules.

[0128] Strategy parameter acquisition submodule: used to obtain the augmentation parameter threshold, parameter discretization number and discrete number lower limit of the target augmentation strategy.

[0129] Augmentation parameter determination submodule: used to determine the target augmentation parameter corresponding to the target augmentation strategy according to the adaptive factor, the number of discretized parameters, the lower limit of the discrete number and the augmentation parameter threshold.

[0130] Augmentation submodule: used to execute the target augmentation strategy on the training sample data based on the target augmentation parameters to obtain augmented sample data.

[0131] In some embodiments, the augmentation parameter determination submodule may include the following units.

[0132] Augmentation strength adaptive calculation unit: used to perform adaptive calculation of augmentation strength according to the adaptive factor, the number of discretized parameters and the lower limit of the number of discrete parameters, and obtain the reference augmentation strength corresponding to the target augmentation strategy.

[0133] Mapping processing unit: used to perform intensity mapping processing on the reference augmentation intensity based on the augmentation parameter threshold and the number of parameter discretizations to obtain the target augmentation parameter corresponding to the target augmentation strategy.

[0134] In some embodiments, the mapping processing unit can be specifically used to: discretize the augmentation parameter range of the target augmentation strategy according to the augmentation parameter threshold and the number of parameter discretizations to obtain a range value corresponding to the unit parameter range; and multiply the range value corresponding to the unit parameter range by the reference augmentation intensity to obtain a target augmentation parameter corresponding to the target augmentation strategy.

[0135] In some embodiments, the device may further include a model loss acquisition module: used to obtain the model loss of the first iterative training of the neural network model and the model loss of the previous iterative training.

[0136] Accordingly, the adaptive factor determination submodule may include the following units.

[0137] Coefficient of variation determination unit: used to determine the coefficient of variation according to the model loss of the previous iterative training and the model loss of the first iterative training, wherein the value range of the coefficient of variation is greater than zero and less than or equal to 1, and the coefficient of variation decreases as the model loss of the previous iterative training decreases.

[0138] Mutation processing unit: used to perform mutation processing on the ratio between the current iteration number and the preset iteration number based on the coefficient of variation to obtain an adaptive factor.

[0139] In some embodiments, the device may also include the following policy attribute acquisition module: used to obtain the policy attributes of each candidate policy set before obtaining the total number of policies and the corresponding policy lower limit values ​​of a preset number of candidate policy sets; and used to select a candidate augmentation strategy from the corresponding candidate policy set as the corresponding target augmentation strategy if the policy attribute is an exclusive attribute.

[0140] It should be noted that the above device embodiment and method embodiment are based on the same implementation method.

[0141] An embodiment of the present application provides a training data augmentation device, which includes a processor and a memory, in which at least one instruction or at least one program is stored, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the training data augmentation method provided in the above method embodiment.

[0142] The memory can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required for functions, etc.; the data storage area can store data created according to the use of the device, etc. In addition, the memory can include a high-speed random access memory and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory can also include a memory controller to provide the processor with access to the memory.

[0143] The method embodiments provided in the embodiments of the present application can be executed in electronic devices such as mobile terminals, computer terminals, servers or similar computing devices. Figure 7is a hardware structure block diagram of an electronic device for a training data augmentation method provided in an embodiment of the present application. Figure 7 As shown, the electronic device 900 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 910 (the processor 910 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 930 for storing data, and one or more storage media 920 (such as one or more mass storage devices) for storing application programs 923 or data 922. Among them, the memory 930 and the storage medium 920 can be short-term storage or permanent storage. The program stored in the storage medium 920 may include one or more modules, each of which may include a series of instruction operations in the electronic device. Furthermore, the central processing unit 910 can be configured to communicate with the storage medium 920 and execute a series of instruction operations in the storage medium 920 on the electronic device 900. The electronic device 900 may also include one or more power supplies 960, one or more wired or wireless network interfaces 950, one or more input and output interfaces 940, and / or, one or more operating systems 921, such as Windows Server TM , Mac OS X TM , Unix TM , LinuxTM, FreeBSDTM, etc.

[0144] The input / output interface 940 may be used to receive or send data via a network. The specific example of the network may include a wireless network provided by a communication provider of the electronic device 900. In one example, the input / output interface 940 includes a network adapter (Network Interface Controller, NIC), which may be connected to other network devices via a base station so as to communicate with the Internet. In one example, the input / output interface 940 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0145] It can be understood by those skilled in the art that Figure 7 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 7 More or fewer components as shown, or with Figure 7 Different configurations shown.

[0146] An embodiment of the present application also provides a computer-readable storage medium, which can be set in an electronic device to store at least one instruction or at least one program related to a training data augmentation method in a method embodiment. The at least one instruction or the at least one program is loaded and executed by the processor to implement the training data augmentation method provided in the above method embodiment.

[0147] Optionally, in this embodiment, the storage medium may be located in at least one of the multiple network servers of the computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0148] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above-mentioned various optional implementations.

[0149] It can be seen from the embodiments of the training data augmentation method, apparatus, equipment, server, terminal, storage medium and program product provided by the present application that the technical solution of the present application first obtains the training sample data and the current iteration number of the neural network model training, and selects a preset number of candidate strategy sets corresponding to the training sample data from multiple augmentation strategy sets, wherein the augmentation strategy set is obtained by pre-classifying multiple augmentation strategies based on strategy attributes; then the total number of strategies and the corresponding lower limit value of the strategy of each preset number of candidate strategy sets are obtained, and the number of reference strategies corresponding to each candidate strategy set is determined according to the total number of strategies, the lower limit value of the strategy, the current iteration number and the preset iteration number, and then the target augmentation strategy corresponding to the reference strategy number is selected from each candidate strategy set, so as to perform augmentation processing on the training sample data based on the selected target augmentation strategy to obtain augmented sample data. In the process of preprocessing the input data of neural network training, this scheme groups the augmentation strategies and uses an adaptive method to adaptively adjust the number of augmentation strategies used to achieve adaptive augmentation of data. Without searching for augmentation strategies, it ensures excellent augmentation effects and model training effects, while saving search time and significantly reducing training resource usage and augmentation costs. In addition, this scheme only processes training sample data and does not involve model modification. It has a wide range of applications and is not limited to data set categories. For example, it can be applied to almost all image detection models, and the method has strong generalization capabilities.

[0150] It should be noted that the above-mentioned sequence of the embodiments of the present application is for description only and does not represent the advantages and disadvantages of the embodiments. The above-mentioned specific embodiments of the present application are described. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0151] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0152] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by instructing the relevant hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0153] The above are only preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A training data augmentation method, characterized in that: The method comprises: Obtaining training sample data and a current iteration number of neural network model training, wherein the training sample data may include a sample image or a sample text; Selecting a preset number of candidate strategy sets corresponding to the training sample data from a plurality of augmentation strategy sets, wherein the augmentation strategy sets are obtained by classifying a plurality of augmentation strategies based on strategy attributes; Obtaining the total number of strategies and the corresponding lower limit values ​​of strategies of the preset number of candidate strategy sets; Determine the number of reference strategies corresponding to each candidate strategy set according to the total number of strategies, the strategy lower limit, the current number of iterations and the preset number of iterations; Selecting target augmentation strategies corresponding to the number of reference strategies from each of the candidate strategy sets; The training sample data is augmented based on the selected target augmentation strategy to obtain augmented sample data.

2. The method according to claim 1, characterized in that The determining, according to the total number of strategies, the strategy lower limit, the current number of iterations, and the preset number of iterations, of the number of reference strategies corresponding to each candidate strategy set comprises: For each candidate strategy set, an adaptive factor corresponding to the current iteration is determined according to the ratio between the current iteration number and the preset iteration number, wherein the adaptive factor has a value range of greater than zero and less than or equal to 1, and the adaptive factor decreases as the number of training iterations increases; An adaptive operation of increasing the number of strategies is performed according to the adaptive factor, the total number of strategies and the strategy lower limit value to obtain the number of reference strategies corresponding to each of the candidate strategy sets.

3. The method according to claim 2, characterized in that The adaptive calculation of the number of augmented strategies is performed according to the adaptive factor, the total number of strategies and the lower limit of the strategies to obtain the number of reference strategies corresponding to each of the candidate strategy sets, including: Subtracting the total number of strategies from the lower limit of the strategies to obtain a strategy quantity difference; The preset cosine descent adaptive method is called to perform adaptive calculations on the strategy lower limit value, the strategy quantity difference and the adaptive factor to obtain the reference strategy quantity corresponding to each of the candidate strategy sets.

4. The method according to claim 2, characterized in that: The augmentation process of the training sample data based on the selected target augmentation strategy to obtain the augmented sample data includes: Obtaining an augmentation parameter threshold, a discretization number of parameters, and a lower limit value of the discretization number of the target augmentation strategy; Determine the target augmentation parameter corresponding to the target augmentation strategy according to the adaptive factor, the discretization number of the parameter, the lower limit of the discrete number of shares and the augmentation parameter threshold; The target augmentation strategy is executed on the training sample data based on the target augmentation parameter to obtain the augmented sample data.

5. The method according to claim 4, characterized in that Determining the target augmentation parameter corresponding to the target augmentation strategy according to the adaptive factor, the discretization number of parameters, the lower limit of the discrete number of parameters and the augmentation parameter threshold comprises: Performing an adaptive calculation of augmentation strength according to the adaptive factor, the discretization number of parameters and the lower limit of the discrete number, to obtain a reference augmentation strength corresponding to the target augmentation strategy; Based on the augmentation parameter threshold and the number of parameter discretizations, intensity mapping processing is performed on the reference augmentation intensity to obtain a target augmentation parameter corresponding to the target augmentation strategy.

6. The method according to claim 5, characterized in that The performing intensity mapping processing on the reference augmentation intensity based on the augmentation parameter threshold and the parameter discretization number to obtain the target augmentation parameter corresponding to the target augmentation strategy includes: Performing discretization operation on the augmentation parameter range of the target augmentation strategy according to the augmentation parameter threshold and the parameter discretization number to obtain a range value corresponding to the unit parameter range; The range value corresponding to the unit parameter range is multiplied by the reference augmentation strength to obtain a target augmentation parameter corresponding to the target augmentation strategy.

7. The method according to any one of claims 2 to 6, characterized in that: The method further comprises: Obtaining a model loss of the first iteration training of the neural network model and a model loss of the previous iteration training; Determining the adaptive factor corresponding to the current iteration according to the ratio between the current iteration number and the preset iteration number includes: Determine a coefficient of variation according to the model loss of the previous iterative training and the model loss of the first iterative training, wherein the value range of the coefficient of variation is greater than zero and less than or equal to 1, and the coefficient of variation decreases as the model loss of the previous iterative training decreases; The adaptive factor is obtained by performing a variation process on the ratio between the current iteration number and the preset iteration number based on the variation coefficient.

8. The method according to any one of claims 1 to 6, characterized in that Before obtaining the total number of strategies and the corresponding lower limit values ​​of strategies of the preset number of candidate strategy sets, the method further includes: Obtaining the policy attributes of each of the candidate policy sets; If the strategy attribute is an exclusive attribute, a candidate augmentation strategy is selected from the corresponding candidate strategy set as the corresponding target augmentation strategy.

9. A training data augmentation device, characterized in that: The device comprises: Iteration number acquisition module: used to obtain training sample data and the current iteration number of neural network model training, wherein the training sample data may include sample images or sample texts; A strategy set selection module: used to select a preset number of candidate strategy sets corresponding to the training sample data from multiple augmented strategy sets, wherein the augmented strategy set is obtained by classifying multiple augmented strategies based on strategy attributes; A strategy set parameter acquisition module: used to obtain the total number of strategies and the corresponding lower limit values ​​of strategies of the preset number of candidate strategy sets; A strategy quantity determination module is used to determine the number of reference strategies corresponding to each candidate strategy set according to the total number of strategies, the strategy lower limit, the current iteration number and the preset iteration number; An augmentation strategy selection module: used to select target augmentation strategies corresponding to the number of reference strategies from each of the candidate strategy sets; Augmentation processing module: used to perform augmentation processing on the training sample data based on the selected target augmentation strategy to obtain augmented sample data.

10. The device according to claim 9, characterized in that The strategy quantity determination module includes: Adaptive factor determination submodule: for each candidate strategy set, determining the adaptive factor corresponding to the current iteration according to the ratio between the current iteration number and the preset iteration number, wherein the adaptive factor has a value range greater than zero and less than or equal to 1, and the adaptive factor decreases as the number of training iterations increases; The strategy quantity calculation submodule is used to perform adaptive calculation on the number of augmented strategies according to the adaptive factor, the total number of strategies and the strategy lower limit value, so as to obtain the number of reference strategies corresponding to each of the candidate strategy sets.

11. The device according to claim 10, characterized in that The strategy quantity calculation submodule includes: A strategy quantity difference calculation unit is used to perform a subtraction process on the total number of strategies and the strategy lower limit value to obtain a strategy quantity difference; Strategy quantity adaptive calculation unit: used to call the preset cosine descent adaptive method, perform adaptive calculation on the strategy lower limit value, the strategy quantity difference and the adaptive factor, and obtain the reference strategy quantity corresponding to each of the candidate strategy sets.

12. The device according to claim 10, characterized in that The augmentation processing module comprises: Strategy parameter acquisition submodule: used to obtain the augmentation parameter threshold, parameter discretization number and discrete number lower limit of the target augmentation strategy; An augmentation parameter determination submodule is used to determine the target augmentation parameter corresponding to the target augmentation strategy according to the adaptive factor, the discretization number of the parameter, the lower limit of the discrete number of portions and the augmentation parameter threshold; Augmentation submodule: used for executing the target augmentation strategy on the training sample data based on the target augmentation parameters to obtain the augmented sample data.

13. The device according to claim 12, characterized in that The augmented parameter determination submodule includes: An augmentation strength adaptive calculation unit is used to perform an augmentation strength adaptive calculation according to the adaptive factor, the discretization number of parameters and the lower limit of the discrete number, so as to obtain a reference augmentation strength corresponding to the target augmentation strategy; A mapping processing unit is used to perform intensity mapping processing on the reference augmentation intensity based on the augmentation parameter threshold and the number of parameter discretizations to obtain a target augmentation parameter corresponding to the target augmentation strategy.

14. The device according to claim 13, characterized in that The mapping processing unit is specifically used for: Performing discretization operation on the augmentation parameter range of the target augmentation strategy according to the augmentation parameter threshold and the parameter discretization number to obtain a range value corresponding to the unit parameter range; The range value corresponding to the unit parameter range is multiplied by the reference augmentation strength to obtain a target augmentation parameter corresponding to the target augmentation strategy.

15. The device according to any one of claims 10 to 14, characterized in that The device also includes a model loss acquisition module: used to acquire the model loss of the first iteration training of the neural network model and the model loss of the previous iteration training; The adaptive factor determination submodule includes: A coefficient of variation determination unit: used to determine the coefficient of variation according to the model loss of the previous iterative training and the model loss of the first iterative training, wherein the value range of the coefficient of variation is greater than zero and less than or equal to 1, and the coefficient of variation decreases as the model loss of the previous iterative training decreases; A variation processing unit is used to perform variation processing on the ratio between the current iteration number and the preset iteration number based on the variation coefficient to obtain the adaptive factor.

16. The device according to any one of claims 9 to 14, characterized in that The device also includes: A strategy attribute acquisition module: used for acquiring the strategy attributes of each of the preset number of strategy sets to be selected before acquiring the total number of strategies and the corresponding lower limit values ​​of strategies of each of the preset number of strategy sets to be selected; If the strategy attribute is an exclusive attribute, a candidate augmentation strategy is selected from the corresponding candidate strategy set as the corresponding target augmentation strategy.

17. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the training data augmentation method according to any one of claims 1 to 8.

18. A computer device, comprising a processor and a memory, characterized in that: The memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the training data augmentation method according to any one of claims 1 to 8.

19. A computer program product, characterized in that The computer program product comprises computer instructions, and when the computer instructions are executed by a processor, the training data augmentation method according to any one of claims 1 to 8 is implemented.

Images