Image augmentation method

Abstract

The invention discloses an image augmentation method. The image augmentation method provided by the embodiment of the invention comprises the steps of determining a plurality of augmentation strategies of a training data set; respectively defining a parameter range corresponding to each augmentation strategy so as to determine a search space of the plurality of augmentation strategies; performing iterative optimization on the plurality of augmentation strategies in the search space based on a predetermined optimization algorithm, and determining an optimal strategy combination; performing image augmentation on the training data set based on the optimal strategy combination to obtain a first augmented data set; and performing model training on the industrial detection model based on the first augmented data set. Thus, the flexibility and generalization are high, the augmentation effect of the determined optimal strategy combination on the training data set is good and stable, the training effect of the industrial detection model can be improved by performing model training based on the first augmentation data set, and therefore the method can adapt to different types of industrial detection scenes and segmentation tasks.

Description

Technical Field

[0001] This application relates to the field of image augmentation technology, and in particular to an image augmentation method. Background Technology

[0002] Benefiting from the rapid development of deep learning, industrial inspection has made significant breakthroughs in recent years, achieving high detection accuracy in numerous complex scenarios. However, industrial inspection models rely heavily on the quantity and quality of training images, and acquiring a large number of industrial defect images with precise segmentation annotations is often costly and time-consuming. To address this issue, data augmentation techniques have been widely used to expand the training set. However, current augmentation methods do not take into account the characteristics of specific training sets, resulting in poor augmentation effects and consequently, poor training performance of industrial inspection models. Summary of the Invention

[0003] This application provides an image augmentation method, an image augmentation apparatus, an image augmentation system, and a computer-readable storage medium to solve at least one of the aforementioned technical problems.

[0004] The image augmentation method of this application includes:

[0005] Determine multiple augmentation strategies for the training dataset;

[0006] Define the parameter range for each of the augmentation strategies to determine the search space for the multiple augmentation strategies;

[0007] Based on a predetermined optimization algorithm, the multiple augmentation strategies are iteratively optimized in the search space to determine the optimal strategy combination.

[0008] Based on the optimal strategy combination, the training dataset is augmented to obtain the first augmented dataset;

[0009] The industrial detection model is trained based on the first augmented dataset.

[0010] In some implementations, the predetermined optimization algorithm includes any one of the Bayesian optimizer algorithm, reinforcement learning algorithm, and evolutionary algorithm.

[0011] In some implementations, the step of iteratively optimizing multiple augmentation strategies in the search space based on a predetermined optimization algorithm to determine the optimal strategy combination includes:

[0012] The optimal strategy combination is determined by iteratively optimizing multiple augmentation strategies in the search space using the Bayesian optimizer algorithm.

[0013] In some implementations, the Bayesian optimizer algorithm iteratively optimizes multiple augmentation strategies in the search space to determine the optimal strategy combination, including:

[0014] Initialize the Bayesian optimizer and determine its surrogate model and acquisition function;

[0015] Based on the proxy model, the acquisition function is used to select a composite transformation strategy corresponding to multiple augmentation strategies in the search space;

[0016] Determine the first evaluation result of the composite transformation strategy;

[0017] The proxy model is updated based on the first evaluation result, and the step of selecting a composite transformation strategy corresponding to multiple augmentation strategies in the search space based on the proxy model and using the acquisition function is returned.

[0018] When the preset conditions are met, the iterative optimization stops, and the composite transformation strategy corresponding to the best first evaluation result is taken as the optimal strategy combination.

[0019] In some implementations, initializing the Bayesian optimizer and determining the surrogate model and acquisition function of the Bayesian optimizer includes:

[0020] A Gaussian process regression model is selected as the surrogate model for the Bayesian optimizer, and the kernel function of the surrogate model is determined.

[0021] The desired improvement function is selected as the acquisition function of the Bayesian optimizer;

[0022] Determine the prior distribution of the surrogate model.

[0023] In some implementations, the step of selecting a composite transformation strategy corresponding to multiple augmentation strategies in the search space based on the proxy model and using the acquisition function includes:

[0024] Based on the surrogate model, the acquisition function is used to select from multiple augmentation strategies in the search space to obtain a strategy sequence to be evaluated. Each position in the strategy sequence to be evaluated is any one of the multiple augmentation strategies, which include any combination of geometric transformation strategies, color transformation strategies, noise addition strategies, and image filtering strategies.

[0025] In the sequence of policies to be evaluated, a probability value is assigned to each augmentation policy to obtain a composite transformation policy.

[0026] In some implementations, determining the first evaluation result of the composite transformation strategy includes:

[0027] The training dataset is augmented based on the composite transformation strategy to obtain a second augmented dataset.

[0028] The first preset model is trained based on the second augmented dataset;

[0029] The performance of the first preset model is evaluated based on the first validation dataset to obtain the first evaluation result.

[0030] In some implementations, the first preset model is a lightweight segmentation model.

[0031] In some implementations, stopping iterative optimization when a preset condition is met includes:

[0032] The iterative optimization shall be stopped when the number of iterations reaches the preset number of iterations; or when the change between the two most recent first evaluation results is less than the preset threshold; or when the computing resources reach the preset computing resources; or when the computing time reaches the preset time.

[0033] In some embodiments, the image augmentation method further includes:

[0034] The training dataset is augmented using the optimal strategy combination to obtain a third augmented dataset.

[0035] The second preset model is trained based on the third augmented dataset;

[0036] The second preset model is evaluated based on the second validation dataset to obtain the second evaluation result;

[0037] The optimal strategy combination is fine-tuned based on the second evaluation result.

[0038] The image augmentation device according to the embodiments of this application includes:

[0039] The determination module is used to determine multiple augmentation strategies for the training dataset;

[0040] A definition module is used to define the parameter range corresponding to each of the augmentation strategies, so as to determine the search space of the multiple augmentation strategies;

[0041] The optimization module is used to iteratively optimize multiple augmentation strategies in the search space based on a predetermined optimization algorithm to determine the optimal strategy combination;

[0042] An augmentation module is used to augment images in the training dataset based on the optimal strategy combination to obtain a first augmented dataset.

[0043] The training module trains the industrial detection model based on the first augmented dataset.

[0044] The image augmentation system according to the embodiments of this application includes one or more processors and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the image augmentation method of any of the above embodiments.

[0045] The computer-readable storage medium of the present application embodiments stores a computer program that, when executed by a processor, implements the image augmentation method of any of the above embodiments.

[0046] In the image augmentation method, image augmentation apparatus, image augmentation system, and computer-readable storage medium of this application, multiple augmentation strategies are iteratively optimized based on a predetermined optimization algorithm within the search space corresponding to multiple augmentation strategies in the training dataset to obtain the optimal strategy combination corresponding to the training dataset. Then, image augmentation is performed on the training dataset based on the optimal strategy combination to obtain a first augmented dataset for training an industrial detection model. This approach offers strong flexibility and generalization; the determined optimal strategy combination provides good and stable augmentation effects on the training dataset; and training the model based on the first augmented dataset improves the training effect of the industrial detection model, thereby adapting to different types of industrial detection scenarios and segmentation tasks.

[0047] Additional aspects and advantages of embodiments of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of embodiments of this application. Attached Figure Description

[0048] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, wherein:

[0049] Figure 1 This is a flowchart illustrating an image augmentation method according to certain embodiments of this application;

[0050] Figure 2 This is a flowchart illustrating an image augmentation method according to certain embodiments of this application;

[0051] Figure 3 This is a flowchart illustrating an image augmentation method according to certain embodiments of this application;

[0052] Figure 4 This is a flowchart illustrating an image augmentation method according to certain embodiments of this application;

[0053] Figure 5 This is a flowchart illustrating an image augmentation method according to certain embodiments of this application;

[0054] Figure 6 This is a flowchart illustrating an image augmentation method according to certain embodiments of this application;

[0055] Figure 7 This is a flowchart illustrating an image augmentation method according to certain embodiments of this application;

[0056] Figure 8 This is a flowchart illustrating an image augmentation method according to certain embodiments of this application;

[0057] Figure 9 This is a schematic diagram of the module of an image augmentation device according to certain embodiments of this application;

[0058] Figure 10 This is a schematic diagram of the module of an image augmentation device according to certain embodiments of this application;

[0059] Figure 11 This is a schematic diagram of the module of an image augmentation system according to certain embodiments of this application;

[0060] Figure 12 This is a schematic diagram illustrating the connection state between a computer-readable storage medium and a processor according to certain embodiments of this application. Detailed Implementation

[0061] The embodiments of this application will be further described below with reference to the accompanying drawings. The same or similar reference numerals in the drawings denote the same or similar elements or elements having the same or similar functions throughout. Furthermore, the embodiments of this application described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of this application, and should not be construed as limiting this application.

[0062] Please see Figure 1 This application provides an image augmentation method. The image augmentation method includes:

[0063] 010: Determine multiple augmentation strategies for the training dataset;

[0064] 020: Define the parameter range for each augmentation strategy to determine the search space for multiple augmentation strategies;

[0065] 030: Based on a predetermined optimization algorithm, iteratively optimize multiple augmentation strategies in the search space to determine the optimal strategy combination for the training dataset;

[0066] 040: Based on the optimal strategy combination, image augmentation is performed on the training dataset to obtain the first augmented dataset;

[0067] 050: The industrial detection model is trained based on the first augmented dataset.

[0068] In the image augmentation method of this application, multiple augmentation strategies are iteratively optimized based on a predetermined optimization algorithm within the search space corresponding to multiple augmentation strategies in the training dataset to obtain the optimal strategy combination corresponding to the training dataset. Then, image augmentation is performed on the training dataset based on the optimal strategy combination to obtain a first augmented dataset, which is used to train an industrial detection model. This approach offers strong flexibility and generalization; the determined optimal strategy combination provides good and stable augmentation effects on the training dataset; and training the model based on the first augmented dataset improves the training effect of the industrial detection model, thereby adapting to different types of industrial detection scenarios and segmentation tasks.

[0069] Specifically, multiple augmentation strategies can be combined to determine the augmentation strategy combinations. Each position in the combination can be any of the multiple augmentation strategies. It can be understood that different permutations and combinations of multiple augmentation strategies will result in different augmentation effects. Multiple augmentation strategies for the training dataset are determined, and the parameter range for each strategy is defined. For example, if the augmentation strategy is rotation, the corresponding angle parameter range can be defined as [-45°, 45°]; if the augmentation strategy is scaling, the corresponding scale parameter range can be defined as [0.8, 1.2].

[0070] Multiple parameter ranges corresponding to multiple augmentation strategies constitute the search space of parameters for these strategies. Based on a predetermined optimization algorithm, the multiple augmentation strategies are iteratively optimized within this search space. This iterative optimization includes permutations and combinations of the strategies and parameter optimization, determining the optimal combination of the strategies and the optimal parameters for each strategy, thereby identifying the optimal strategy combination for the training dataset.

[0071] After obtaining the optimal strategy combination, a configuration file is generated based on the optimal strategy combination, which describes in detail how multiple augmentation strategies are combined and the optimal parameters of each augmentation strategy, so as to facilitate subsequent augmentation of the training dataset and realize the expansion of the training dataset.

[0072] The training dataset can include multiple training images. In an industrial inspection project, the training images are industrial images. By employing the optimal strategy combination, multiple training images in the training dataset can be augmented to obtain the first augmented dataset. The first augmented dataset includes the multiple training images in the training dataset and the augmented images obtained from the first augmentation.

[0073] The industrial inspection model is trained using the first augmented dataset, enabling its application in industrial inspection. For example, in an industrial defect detection project, the training images are defect images. The industrial inspection model is trained using the first augmented dataset, and the trained model can then be applied to industrial defect detection.

[0074] In related technologies, the characteristics of specific datasets are not taken into account. Predefined combinations of augmentation strategies with fixed parameters or combinations of augmentation strategies with randomly selected parameters are used. These combinations of augmentation strategies cannot adapt to the characteristics of different datasets and are difficult to extend to other industrial detection scenarios, resulting in poor dataset augmentation effects or ineffective or even harmful augmentation effects.

[0075] For example, in industrial inspection tasks, some augmentation strategies may alter the shape features of the target object, causing the subsequent model to learn incorrect features; or, they may over-distort the image, causing the augmented data to deviate from the original data distribution; or, they may change the contrast between the target object and the background, blurring the boundaries of the target object and affecting the accuracy of the segmentation task.

[0076] In this embodiment, multiple augmentation strategies are iteratively optimized in the search space based on a predetermined optimization algorithm to obtain the optimal strategy combination corresponding to the training dataset. For different training datasets, the characteristics of the training dataset are adaptively learned, and the optimal strategy combination is found. Thus, adaptive adjustments can be made according to the characteristics of the training dataset, resulting in strong flexibility and generalization. The determined optimal strategy combination provides good and stable augmentation effects on the training dataset. Furthermore, the industrial detection model trained on the first augmented dataset has high accuracy and strong generalization ability, and can adapt to different types of industrial detection scenarios and segmentation tasks.

[0077] Other related techniques require manual selection and adjustment of augmentation strategy combinations based on experience, which is time-consuming and highly subjective. Furthermore, the lack of specificity in the augmentation strategy combinations may generate a large amount of useless augmented data, wasting storage space and computational resources during subsequent training.

[0078] In the implementation of this application, the optimal strategy combination can be determined without manual intervention, and the optimal strategy combination has a good and stable augmentation effect on the training dataset, thus avoiding the waste of storage space and subsequent computing resources.

[0079] In some implementations, the augmentation strategies include any combination of geometric transformation strategies, color transformation strategies, noise addition strategies, and image filtering strategies.

[0080] Specifically, the multiple augmentation strategies can include any combination of geometric transformation strategies, color transformation strategies, noise addition strategies, and image filtering strategies. For example, the multiple augmentation strategies can include any two, three, or all of the geometric transformation strategies, color transformation strategies, noise addition strategies, and image filtering strategies.

[0081] Each augmentation strategy also includes multiple sub-strategies. For example, the geometric transformation strategy includes rotation, flip, scaling, shearing, and translation sub-strategies. The rotation sub-strategy refers to rotating the image by a certain angle. The flip sub-strategy refers to flipping the image horizontally or vertically. The scaling sub-strategy refers to enlarging or reducing the image. The shearing sub-strategy refers to skewing the image. The translation sub-strategy refers to moving the image along the x-axis or y-axis.

[0082] Color transformation strategies include brightness adjustment sub-strategies, contrast adjustment sub-strategies, hue shift sub-strategies, saturation adjustment sub-strategies, and channel blending sub-strategies. Among these, the brightness adjustment sub-strategy refers to increasing or decreasing the overall brightness of the image. The contrast adjustment sub-strategy refers to enhancing or weakening the contrast between light and dark areas of the image. The hue shift sub-strategy refers to changing the hue of the image. The saturation adjustment sub-strategy refers to increasing or decreasing the purity of colors in the image. The channel blending sub-strategy refers to adjusting the ratio of the image's RGB channels.

[0083] Noise addition strategies include Gaussian noise sub-strategies, salt-and-pepper noise sub-strategies, and Poisson noise sub-strategies. The Gaussian noise sub-strategy adds random noise following a Gaussian distribution to the image. The salt-and-pepper noise sub-strategy randomly sets the pixels of the image to black or white. The Poisson noise sub-strategy adds noise to the image to simulate noise caused by photon counting statistics.

[0084] Image filtering strategies include blurring sub-strategies and sharpening sub-strategies. Blurring sub-strategies refer to applying Gaussian blur or average blur to the image. Sharpening sub-strategies refer to enhancing image edges and details.

[0085] It can be understood that defining the parameter range for each augmenting strategy is equivalent to defining the parameter range for each sub-strategy within each augmenting strategy. The parameter ranges of all sub-strategies constitute the search space for the parameters of multiple augmenting strategies.

[0086] For example, for a geometric transformation strategy, the angular parameter range of the rotation sub-strategy can be defined as [-45°, 45°]. The scaling parameter range of the scaling sub-strategy can be defined as [0.8, 1.2]. The probability parameter range of the horizontal flip sub-strategy can be defined as [0, 1]. The probability parameter range of the vertical flip sub-strategy can be defined as [0, 1]. The scaling parameter range of the translation sub-strategy can be defined as [-10%, 10%].

[0087] For color transformation strategies, the scaling parameter range for the brightness adjustment sub-strategy can be defined as [-20%, 20%]. The scaling parameter range for the contrast adjustment sub-strategy can be defined as [0.8, 1.2]. The angle parameter range for the hue shift sub-strategy can be defined as [-20°, 20°]. The scaling parameter range for the saturation adjustment sub-strategy can be defined as [0.8, 1.2].

[0088] For the noise addition strategy, the intensity parameter range of the Gaussian noise sub-strategy can be defined as [0, 0.1]. The scale parameter range of the salt-and-pepper noise sub-strategy can be defined as [0, 0.05].

[0089] For image filtering strategies, the Gaussian kernel size parameter of the blurring sub-strategy can be defined in the range of [3, 7]. The intensity parameter of the sharpening sub-strategy can be defined in the range of [0, 2].

[0090] In one example, the search space also includes multiple combinations of multiple augmentation strategies.

[0091] In some implementations, the predetermined optimization algorithm includes any one of the Bayesian optimizer algorithm, reinforcement learning algorithm, and evolutionary algorithm.

[0092] Specifically, any one of the Bayesian optimizer algorithm, reinforcement learning algorithm, or evolutionary algorithm can be used as the predetermined optimization algorithm to iteratively optimize multiple augmentation strategies and determine the optimal strategy combination for the training dataset.

[0093] Please see Figure 2 In some implementations, multiple augmentation strategies are iteratively optimized in the search space based on a predetermined optimization algorithm to determine the optimal strategy combination (i.e., 030), including:

[0094] 031: Based on the Bayesian optimizer algorithm, iterative optimization is performed on multiple augmenting strategies in the search space to determine the optimal strategy combination.

[0095] Specifically, the Bayesian optimizer algorithm possesses global optimization capabilities and adaptability, while maintaining high optimization efficiency. It approximates the objective function by establishing a surrogate model and uses Bayes' theorem to infer the selection of the next sampling point, thereby reducing the uncertainty of the surrogate model and the expectation of the objective function. After each sampling, the surrogate model is updated based on the new observations, and the above process is repeated until predetermined conditions are met. Based on the Bayesian optimizer algorithm, by establishing a surrogate model and iteratively optimizing multiple augmenting strategies in the search space, continuously updating the surrogate model, the optimal strategy combination for the training dataset can be determined.

[0096] In some implementations, reinforcement learning algorithms iteratively optimize multiple augmenting policies in the search space to determine the optimal policy combination for the training dataset. Specifically, the reinforcement learning algorithm can be Q-learning or the policy gradient method. Reinforcement learning algorithms learn the optimal policy combination by modeling the selection of augmenting policies as a sequential decision problem. Reinforcement learning algorithms can learn the dependencies and long-term effects between multiple augmenting policies, possessing autonomous learning capabilities and strong adaptability and flexibility.

[0097] In some implementations, evolutionary algorithms iteratively optimize multiple augmentation strategies in the search space to determine the optimal strategy combination for the training dataset. Evolutionary algorithms simulate the natural biological evolution process through genetic algorithms or evolutionary strategies, continuously evolving and searching for the optimal strategy combination. Evolutionary algorithms can effectively handle discrete or continuous augmentation strategy parameters, are easily parallelized, and possess self-organizing, adaptive, and self-learning characteristics.

[0098] Thus, by using any one of the Bayesian optimizer algorithm, reinforcement learning algorithm, or evolutionary algorithm as the predetermined optimization algorithm, it is possible to adaptively adjust according to the characteristics of the training dataset. The determined optimal strategy combination has a good and stable augmentation effect on the training dataset.

[0099] The following section details the process of using the Bayesian optimizer algorithm as the predetermined optimization algorithm to determine the optimal strategy combination.

[0100] Please see Figure 3 In some implementations, the optimal strategy combination (i.e., 031) is determined by iteratively optimizing multiple augmenting strategies in the search space based on the Bayesian optimizer algorithm, including:

[0101] 0311: Initialize the Bayesian optimizer and determine its surrogate model and acquisition function;

[0102] 0312: Based on the surrogate model, a composite transformation strategy corresponding to multiple augmentation strategies is selected in the search space using a collection function;

[0103] 0313: Determine the first evaluation result of the composite transformation strategy;

[0104] 0314: Update the surrogate model based on the first evaluation result and return the steps of selecting the composite transformation strategy corresponding to multiple augmentation strategies in the search space based on the surrogate model and using the acquisition function;

[0105] 0315: When the preset conditions are met, stop iterative optimization and take the composite transformation strategy corresponding to the best first evaluation result as the optimal strategy combination.

[0106] Specifically, the Bayesian optimizer is initialized by determining its surrogate model and acquisition function. The surrogate model can model the relationship between the augmentation strategy and model performance. The surrogate model can be any of the following: Gaussian process regression model, random forest, or Gaussian mixture model. The acquisition function guides the iterative optimization process, determining which region of the search space should be explored next. The acquisition function can be any of the following: Expected Improvement (EI) function, Upper Confidence Bound (UCB) function, or Probability of Improvement (PI) function.

[0107] Please see Figure 4 In some implementations, the Bayesian optimizer is initialized, and the surrogate model and acquisition function (i.e., 0311) of the Bayesian optimizer are determined, including:

[0108] 03111: Select the Gaussian process regression model as the surrogate model for the Bayesian optimizer, and determine the kernel function of the surrogate model;

[0109] 03112: Select the desired improvement function as the acquisition function for the Bayesian optimizer;

[0110] 03113: Determine the prior distribution of the surrogate model.

[0111] Specifically, Gaussian process regression refers to fitting a corresponding Gaussian process to finite high-dimensional data, thereby predicting the function value of any random variable. The core of Gaussian process regression is utilizing the properties of Gaussian processes to model and predict data by specifying an appropriate kernel function. The kernel function can capture the smoothness and discontinuities in the search space and determines the smoothness, periodicity, and other characteristics of the Gaussian process regression model. Kernel functions can include radial basis function kernels, Matérn kernels, or rational quadratic kernels. In one example, the Matérn 5 / 2 kernel is chosen as the kernel function for the surrogate model.

[0112] Choosing a Gaussian process regression model as the surrogate model for the Bayesian optimizer is advantageous because it not only provides predicted values ​​but also estimates of the uncertainty of those predictions, facilitating the selection of composite transformation strategies for the acquisition function. Furthermore, the kernel function of the Gaussian process regression model can be flexibly selected to adapt to the characteristics of different training datasets.

[0113] The expected improvement function can be calculated based on the maximum and minimum values ​​predicted by the current surrogate model, using the results inferred from Bayes' theorem to calculate the expected improvement value of each candidate composite transformation strategy in the search space. This expected improvement value represents the potential benefit of further evaluating the candidate composite transformation strategy given the current optimal solution. By comparing the expected improvement values ​​of each candidate composite transformation strategy, the candidate composite transformation strategy that maximizes the expected improvement can be selected as the next composite transformation strategy.

[0114] Choosing the expectation improvement function as the sampling function for the Bayesian optimizer algorithm allows for several advantages. In the exploration phase, the expectation improvement function tends to select points with higher prediction uncertainty to explore potential better solutions. In the utilization phase, it tends to select points with higher prediction values ​​to further optimize the currently known optimal solution. This balances the needs of exploration and utilization. Furthermore, because the expectation improvement function comprehensively considers both prediction values ​​and uncertainty, it can find the global optimum or a solution close to the global optimum in fewer iterations.

[0115] After determining the surrogate model, it is also necessary to determine its prior distribution, which is used to fit the unknown distribution of the true objective function. Reasonable initial points can be set for the surrogate model based on prior knowledge in the industrial inspection field to initialize its parameters. For example, the pixel averages of multiple industrial images in an industrial inspection scenario can be used as initial points.

[0116] After initialization, the Bayesian optimizer selects a composite transformation strategy corresponding to multiple augmentation strategies in the search space based on the surrogate model and the acquisition function. The acquisition function can weigh the uncertain regions of the surrogate model (i.e., exploration) and the known high-performance regions (i.e., exploitation), selecting the composite transformation strategy that maximizes the acquisition function.

[0117] After obtaining the composite transformation strategy, it is evaluated to obtain the first evaluation result. Based on the first evaluation result, the surrogate model can be updated to improve its prediction accuracy. Then, the process returns to the step of selecting composite transformation strategies corresponding to multiple augmentation strategies in the search space using a collection function based on the surrogate model, and a new round of iterative optimization is performed based on the updated surrogate model.

[0118] When the iterative optimization process reaches the preset conditions, the iterative optimization stops, and the optimal strategy combination is determined based on the first evaluation result of each composite transformation strategy. For example, the composite transformation strategy corresponding to the best first evaluation result can be selected as the optimal strategy combination.

[0119] Please see Figure 5In some implementations, based on the surrogate model, a composite transformation strategy (i.e., 0312) corresponding to multiple augmentation strategies is selected in the search space using an acquisition function, including:

[0120] 03121: Based on the surrogate model, a collection function is used to select from multiple augmentation strategies in the search space to obtain a sequence of strategies to be evaluated, wherein each position in the sequence of strategies to be evaluated is any one of the multiple augmentation strategies;

[0121] 03122: In the policy sequence to be evaluated, assign probability values ​​to each augmentation policy to obtain the composite transformation policy.

[0122] Specifically, the Bayesian optimizer, based on the surrogate model, uses a collection function to select a combination of multiple augmenting strategies in the search space, along with the parameters corresponding to each augmenting strategy, to obtain a sequence of policies to be evaluated. Each position in the sequence of policies to be evaluated can be any of the multiple augmenting strategies, specifically any sub-strategy of any augmenting strategy.

[0123] In the strategy sequence to be evaluated, a probability value is assigned to each augmentation strategy, resulting in a composite transformation strategy. This composite transformation strategy includes the strategy sequence to be evaluated and the probability value of each augmentation strategy within the sequence. The probability value ranges from [0, 1], representing the probability that an augmentation strategy will be applied to the image for augmentation. In other words, when using the optimal strategy combination to augment the image, there may be augmentation strategies within the optimal strategy combination that are not applied to the image. Therefore, the optimal strategy combination can repeatedly augment the same training image, resulting in different augmented images.

[0124] Please see Figure 6 In some implementations, determining the first evaluation result (i.e., 0313) of the composite transformation strategy includes:

[0125] 03131: Augment the training dataset using a composite transformation strategy to obtain a second augmented dataset;

[0126] 03132: Train the first preset model based on the second augmented dataset;

[0127] 03133: The performance of the first preset model is evaluated based on the first validation dataset to obtain the first evaluation result.

[0128] Specifically, the composite transformation strategy can be evaluated based on the first preset model. The specific process is as follows: The training dataset is augmented using the composite transformation strategy to obtain a second augmented dataset. The second augmented dataset includes multiple training images from the training dataset and the augmented second augmented images. The first preset model is then trained based on the second augmented dataset. The first preset model can be any model applicable to industrial inspection projects.

[0129] After the first preset model is trained, a first validation dataset is obtained. The first validation dataset includes multiple first validation images; in an industrial inspection project, the first validation images are industrial images. Based on the first validation dataset, the performance of the first preset model can be evaluated to obtain a first evaluation result. For example, if the first preset model is a segmentation model, the model detection rate on the first validation dataset can be used as the evaluation metric. The higher the model detection rate, the better the model performance, and the better the first evaluation result.

[0130] In related technologies, there is a lack of quantitative evaluation and optimization mechanisms for augmentation effects, making it impossible to adaptively adjust augmentation strategies and difficult to determine whether augmentation truly improves model performance.

[0131] In this embodiment, a first preset model is used to evaluate the composite transformation strategy. The augmentation effect is directly evaluated through the model performance of the first preset model. The evaluation index of the first preset model is also used as the target of Bayesian optimization, which can ensure the effectiveness of the augmentation strategy combination. It can provide detailed performance indicators and visualization results, which are easy to analyze and understand, and can simplify the overall task. In addition, the process of finding the augmentation strategy combination is highly bound to a specific training dataset, thereby effectively improving the augmentation effect.

[0132] In some implementations, the first preset model is a lightweight segmentation model.

[0133] Specifically, a lightweight segmentation model can be used as the first preset model to evaluate the composite transformation strategy. Compared with the complete segmentation model, the lightweight segmentation model has fewer parameters, occupies less storage space, and requires less computation. This can accelerate the evaluation process of each composite transformation strategy, further simplify the overall task, and improve the efficiency of the iterative optimization process.

[0134] Please see Figure 7 In some implementations, the iterative optimization is stopped when a preset condition is met (i.e., 0315), including:

[0135] 03151: Stop iterative optimization when the number of iterations reaches the preset number of iterations; or when the change between the two most recent first evaluation results is less than the preset threshold; or when the computing resources reach the preset computing resources; or when the computing time reaches the preset time.

[0136] Specifically, the preset conditions can be set based on the number of iterations, the change in the first evaluation result, computational resources, and computation time. The maximum allowed number of iterations can be used as the preset number of iterations; iteration optimization stops when the preset number of iterations is reached. Alternatively, after obtaining the current first evaluation result, the change between the current first evaluation result and the previous first evaluation result can be calculated; iteration optimization stops when the change in the first evaluation result is less than a preset threshold. Alternatively, the maximum allowed computational resources can be used as the preset computational resources; iteration optimization stops when the preset computational resources are reached. Alternatively, the maximum allowed computation time can be used as the preset time; iteration optimization stops when the preset time is reached.

[0137] For the above-mentioned preset conditions, iterative optimization can be stopped when any one of the preset conditions is met. In this way, the efficiency of iterative optimization can be improved to a certain extent.

[0138] Please see Figure 8 In some implementations, the image augmentation method further includes:

[0139] 040: The training dataset is augmented using the optimal strategy combination to obtain the third augmented dataset;

[0140] 050: Train the second preset model based on the third augmented dataset;

[0141] 060: The performance of the second preset model is evaluated based on the second validation dataset to obtain the second evaluation result;

[0142] 070: Fine-tune the optimal strategy combination based on the results of the second evaluation.

[0143] Specifically, after obtaining the optimal strategy combination through iterative optimization based on a predetermined optimization algorithm, the optimal strategy combination can be fine-tuned. The training dataset is then augmented using the optimal strategy combination to obtain a third augmented dataset, which includes multiple industrial images from the training dataset and the augmented third-amplified images. A second pre-defined model is then trained based on this third augmented dataset. This second pre-defined model can be any model applicable to industrial inspection projects.

[0144] In one example, the second pre-defined model is larger than the first pre-defined model; for instance, if the first pre-defined model is a lightweight segmentation model, then the second pre-defined model is a full segmentation model. The third augmented dataset is larger than the second augmented dataset.

[0145] After the second preset model is trained, a second validation dataset is obtained. This dataset includes multiple validation images; in industrial inspection projects, these validation images are industrial images. The second validation dataset may or may not be the same as the first validation dataset. The performance of the second preset model is then evaluated based on the second validation dataset to obtain a second evaluation result. For example, if the second preset model is a segmentation model, its detection rate on the second validation dataset can be used as the evaluation metric. A higher detection rate indicates better model performance, resulting in a better second evaluation result.

[0146] Based on the second evaluation results, you can choose whether to fine-tune the optimal strategy combination. If fine-tuning is required, you can manually fine-tune the optimal strategy combination within a small range based on the second evaluation results to improve the second evaluation results.

[0147] Please see Figure 9 This application also provides an image augmentation device 100. The image augmentation device 100 includes a determining module 10, a defining module 20, an optimizing module 30, an augmentation module 40, and a training module 50. The determining module 10 is used to determine multiple augmentation strategies for a training dataset. The defining module 20 is used to define the parameter range corresponding to each augmentation strategy to determine the search space of the multiple augmentation strategies. The optimizing module 30 is used to iteratively optimize the multiple augmentation strategies in the search space based on a predetermined optimization algorithm to determine the optimal strategy combination. The augmentation module 40 is used to perform image augmentation on the training dataset based on the optimal strategy combination to obtain a first augmented dataset. The training module 50 is used to train an industrial detection model based on the first augmented dataset.

[0148] In some implementations, the predetermined optimization algorithm includes any one of the Bayesian optimizer algorithm, reinforcement learning algorithm, and evolutionary algorithm.

[0149] In some implementations, the optimization module 30 is specifically used to iteratively optimize multiple augmentation strategies in the search space based on the Bayesian optimizer algorithm to determine the optimal strategy combination.

[0150] In some implementations, the optimization module 30 is specifically used to initialize the Bayesian optimizer, determine the surrogate model and acquisition function of the Bayesian optimizer; select a composite transformation strategy corresponding to multiple augmentation strategies in the search space based on the surrogate model and the acquisition function; determine the first evaluation result of the composite transformation strategy; update the surrogate model according to the first evaluation result, and return the step of selecting a composite transformation strategy corresponding to multiple augmentation strategies in the search space based on the surrogate model and the acquisition function; stop iterative optimization when a preset condition is met, and take the composite transformation strategy corresponding to the best first evaluation result as the optimal strategy combination.

[0151] In some implementations, the optimization module 30 is specifically used to select a Gaussian process regression model as the surrogate model of the Bayesian optimizer and determine the kernel function of the surrogate model; select the expected improvement function as the acquisition function of the Bayesian optimizer; and determine the prior distribution of the surrogate model.

[0152] In some implementations, the optimization module 30 is specifically used to select from multiple augmentation strategies in the search space based on a surrogate model using an acquisition function to obtain a sequence of strategies to be evaluated. Each position in the sequence of strategies to be evaluated is any one of the multiple augmentation strategies, which include any combination of geometric transformation strategies, color transformation strategies, noise addition strategies, and image filtering strategies. In the sequence of strategies to be evaluated, a probability value is assigned to each augmentation strategy to obtain a composite transformation strategy.

[0153] In some implementations, the optimization module 30 is specifically used to augment the training dataset based on a composite transformation strategy to obtain a second augmented dataset; to train a first preset model based on the second augmented dataset; and to evaluate the performance of the first preset model based on a first validation dataset to obtain a first evaluation result.

[0154] In some implementations, the first preset model is a lightweight segmentation model.

[0155] In some implementations, the optimization module 30 is specifically used to stop iterative optimization when the number of iterations reaches a preset number of iterations; or when the change between the two most recent first evaluation results is less than a preset threshold; or when the computing resources reach a preset number of computing resources; or when the computing time reaches a preset time.

[0156] Please see Figure 10 In some embodiments, the image augmentation device 100 further includes a fine-tuning module 60, which is used to augment the training dataset with the optimal strategy combination to obtain a third augmented dataset; train a second preset model based on the third augmented dataset; evaluate the performance of the second preset model based on the second validation dataset to obtain a second evaluation result; and fine-tune the optimal strategy combination according to the second evaluation result.

[0157] It should be noted that the explanation of the image augmentation method in the foregoing embodiments also applies to the image augmentation device 100 in the embodiments of this application, and will not be elaborated here.

[0158] Please see Figure 11This application also provides an image augmentation system 200. The image augmentation system 200 includes one or more processors 210 and a memory 220. The memory 220 stores a computer program, which, when executed by the processor 210, implements the image augmentation method of any of the above embodiments.

[0159] For example, when the computer program is executed by the processor 210, the following image augmentation method is implemented:

[0160] 010: Determine multiple augmentation strategies for the training dataset;

[0161] 020: Define the parameter range for each augmentation strategy to determine the search space for multiple augmentation strategies;

[0162] 030: Based on a predetermined optimization algorithm, iteratively optimize multiple augmentation strategies in the search space to determine the optimal strategy combination for the training dataset;

[0163] 040: Based on the optimal strategy combination, image augmentation is performed on the training dataset to obtain the first augmented dataset;

[0164] 050: The industrial detection model is trained based on the first augmented dataset.

[0165] For example, when the computer program is executed by the processor 210, the following image augmentation method is implemented:

[0166] 031: Based on the Bayesian optimizer algorithm, iterative optimization is performed on multiple augmenting strategies in the search space to determine the optimal strategy combination.

[0167] It should be noted that the explanations of the image augmentation method and image augmentation device 100 in the foregoing embodiments also apply to the image augmentation system 200 of the embodiments of this application, and will not be elaborated here.

[0168] Please see Figure 12 This application also provides a computer-readable storage medium 300 storing a computer program 310, which, when executed by a processor 320, implements the image augmentation method of any of the above embodiments.

[0169] For example, when computer program 310 is executed by processor 320, the following image augmentation method is implemented:

[0170] 010: Determine multiple augmentation strategies for the training dataset;

[0171] 020: Define the parameter range for each augmentation strategy to determine the search space for multiple augmentation strategies;

[0172] 030: Based on a predetermined optimization algorithm, iteratively optimize multiple augmentation strategies in the search space to determine the optimal strategy combination for the training dataset;

[0173] 040: Based on the optimal strategy combination, image augmentation is performed on the training dataset to obtain the first augmented dataset;

[0174] 050: The industrial detection model is trained based on the first augmented dataset.

[0175] For example, when computer program 310 is executed by processor 320, the following image augmentation method is implemented:

[0176] 031: Based on the Bayesian optimizer algorithm, iterative optimization is performed on multiple augmenting strategies in the search space to determine the optimal strategy combination.

[0177] It should be noted that the explanations of the image augmentation method and image augmentation device 100 in the foregoing embodiments also apply to the computer-readable storage medium 300 of the embodiments of this application, and will not be elaborated here.

[0178] In summary, the image augmentation method, image augmentation apparatus 100, image augmentation system 200, and computer-readable storage medium 300 of this application, in the search space corresponding to multiple augmentation strategies in the training dataset, iteratively optimize multiple augmentation strategies based on a predetermined optimization algorithm to obtain the optimal strategy combination corresponding to the training dataset. Then, image augmentation is performed on the training dataset based on the optimal strategy combination to obtain the first augmented dataset, which is used to train the industrial detection model. This approach offers strong flexibility and generalization; the determined optimal strategy combination provides good and stable augmentation effects on the training dataset; and model training based on the first augmented dataset can improve the training effect of the industrial detection model, thereby adapting to different types of industrial detection scenarios and segmentation tasks.

[0179] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0180] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0181] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable storage medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, a computer-readable storage medium can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable storage medium could be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0182] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0183] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it includes one or a combination of the steps of the method embodiments. Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc.

[0184] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. An image augmentation method, characterized in that: include: Determine multiple augmentation strategies for the training dataset; Respectively define a parameter range corresponding to each of the augmentation strategies to determine a search space of the plurality of augmentation strategies; Iteratively optimizing the plurality of augmentation strategies in the search space based on a predetermined optimization algorithm to determine an optimal strategy combination; Performing image augmentation on the training data set based on the optimal strategy combination to obtain a first augmented data set; The industrial detection model is trained based on the first augmented data set.

2. The image augmentation method according to claim 1, characterized in that: The predetermined optimization algorithm includes any one of a Bayesian optimizer algorithm, a reinforcement learning algorithm, and an evolutionary algorithm.

3. The image augmentation method according to claim 1, characterized in that: The iterative optimization of the plurality of augmentation strategies in the search space based on a predetermined optimization algorithm to determine an optimal strategy combination includes: Based on the Bayesian optimizer algorithm, multiple augmentation strategies are iteratively optimized in the search space to determine the optimal strategy combination.

4. The image augmentation method according to claim 3, characterized in that: The iterative optimization of the plurality of augmentation strategies in the search space based on the Bayesian optimizer algorithm to determine the optimal strategy combination includes: Initializing a Bayesian optimizer and determining a proxy model and an acquisition function of the Bayesian optimizer; Based on the proxy model, the acquisition function is used to select a plurality of composite transformation strategies corresponding to the augmentation strategies in the search space; Determining a first evaluation result of the composite transformation strategy; The proxy model is updated according to the first evaluation result, and the step of selecting a composite transformation strategy corresponding to a plurality of the augmentation strategies in the search space by using the acquisition function based on the proxy model is returned; When the preset condition is reached, the iterative optimization is stopped, and the composite transformation strategy corresponding to the optimal first evaluation result is used as the optimal strategy combination.

5. The image augmentation method according to claim 4, characterized in that: The initializing of the Bayesian optimizer and determining the proxy model and acquisition function of the Bayesian optimizer include: Selecting a Gaussian process regression model as the proxy model of the Bayesian optimizer, and determining a kernel function of the proxy model; Selecting an expected improvement function as the acquisition function of the Bayesian optimizer; A prior distribution for the surrogate model is determined.

6. The image augmentation method according to claim 4, characterized in that: The step of selecting, based on the proxy model, a composite transformation strategy corresponding to a plurality of the augmentation strategies in the search space by using the acquisition function comprises: Based on the proxy model, the acquisition function is used to select multiple augmentation strategies in the search space to obtain a strategy sequence to be evaluated, wherein each position of the strategy sequence to be evaluated is any one of the multiple augmentation strategies, and the multiple augmentation strategies include any multiple of geometric transformation strategies, color transformation strategies, noise addition strategies, and image filtering strategies; In the sequence of strategies to be evaluated, a probability value is assigned to each of the augmented strategies to obtain a composite transformation strategy.

7. The image augmentation method according to claim 4, characterized in that: The determining of the first evaluation result of the composite transformation strategy comprises: Augmenting the training data set based on the composite transformation strategy to obtain a second augmented data set; Training the first preset model based on the second augmented data set; A performance evaluation is performed on the first preset model based on a first validation data set to obtain the first evaluation result.

8. The image augmentation method according to claim 7, characterized in that: The first preset model is a lightweight segmentation model.

9. The image augmentation method according to claim 4, characterized in that: When the preset condition is reached, stopping the iterative optimization includes: When the number of iterations reaches a preset number of iterations; or when the change between the two most recent first evaluation results is less than a preset threshold; or when the computing resources reach a preset computing resources; or when the computing time reaches a preset time, stop the iterative optimization.

10. The image augmentation method according to claim 1, characterized in that: The image augmentation method further comprises: Augmenting the training data set using the optimal strategy combination to obtain a third augmented data set; Training a second preset model based on the third augmented data set; Performing a performance evaluation on the second preset model based on a second validation data set to obtain a second evaluation result; The optimal strategy combination is fine-tuned according to the second evaluation result.

Images