Mask image-based model training method and device, and storage medium

Abstract

The application discloses a model training method and device based on a mask image and a storage medium, and the method comprises the following steps: performing occlusion processing on a first image region of a training image by using a mask region of an initial image mask to obtain a first mask image; performing model training on a first recognition model by using the first mask image to update model parameters of the first recognition model and obtain a second recognition model; adjusting parameter values of image mask parameters based on a loss corresponding to the second recognition model and the first mask image to obtain a target image mask, performing occlusion processing on a second image region of the training image by using a mask region of the target image mask to obtain a second mask image, and the loss corresponding to the second recognition model and the first mask image is lower than a loss corresponding to the second recognition model and the second mask image; and performing model training on the second recognition model by using the second mask image to update model parameters of the second recognition model and obtain a third recognition model.

Description

Technical Field

[0001] This application relates to the field of computer science, and more specifically, to a model training method, apparatus, and storage medium based on mask images. Background Technology

[0002] Deep neural networks, due to their numerous model parameters, are prone to overfitting during training. This results in a neural network model performing well on the training set but performing poorly in real-world testing, lacking generalization ability. To address this, data augmentation can be used to expand the training data in the training set, thereby altering the distribution of the data.

[0003] For scenarios where the training data consists of training images, the commonly used data augmentation method is mask-based data augmentation. This involves manually selecting a mask of a specific shape and placing it on the training images, then modifying the image content within the mask area. This allows the model to see more images during training, thereby achieving stronger generalization ability.

[0004] However, the aforementioned method of model training using mask images suffers from a lack of relevance to the goals of reducing overfitting and improving generalization ability, as mask selection and filling are largely based on randomness. This means the masked and filled portions can appear anywhere in the original image. Therefore, mask-based model training methods in related technologies suffer from weak generalization ability due to the randomness of mask selection. Summary of the Invention

[0005] This invention provides a model training method, apparatus, and storage medium based on mask images, to at least solve the problem that the generalization ability of models based on mask images in related technologies is weak due to the randomness of mask selection.

[0006] According to one aspect of the embodiments of this application, a model training method based on a mask image is provided, comprising: occluding a first image region of a training image using a mask region of an initial image mask to obtain a first mask image; training a first recognition model using the first mask image to update the model parameters of the first recognition model to obtain a second recognition model; adjusting the parameter values ​​of the image mask parameters based on the loss corresponding to the second recognition model and the first mask image to obtain a target image mask, and occluding a second image region of the training image using a mask region of the target image mask to obtain a second mask image, wherein the image mask parameters are used to represent the mask region of the image mask corresponding to the training image, and the loss of the second recognition model corresponding to the first mask image is lower than the loss of the second recognition model corresponding to the second mask image; and training the second recognition model using the second mask image to update the model parameters of the second recognition model to obtain a third recognition model.

[0007] According to another aspect of the embodiments of this application, a model training apparatus based on a mask image is also provided, comprising: a first processing unit, configured to occlude a first image region of a training image using a mask region of an initial image mask to obtain a first mask image; a first training unit, configured to train a first recognition model using the first mask image to update the model parameters of the first recognition model to obtain a second recognition model; an adjustment unit, configured to adjust the parameter values ​​of the image mask parameters based on the loss corresponding to the second recognition model and the first mask image to obtain a target image mask, wherein the image mask parameters are used to represent the mask region of the image mask corresponding to the training image; a second processing unit, configured to occlude a second image region of the training image using the mask region of the target image mask to obtain a second mask image, wherein the loss corresponding to the second recognition model and the first mask image is lower than the loss corresponding to the second recognition model and the second mask image; and a second training unit, configured to train the second recognition model using the second mask image to update the model parameters of the second recognition model to obtain a third recognition model.

[0008] As an optional solution, the adjustment unit includes: a first input module, used to input the first mask image into the second recognition model to obtain a first recognition result output by the second recognition model; a first determination module, used to determine a first function value corresponding to a preset loss function and the second recognition result and the preset recognition result, wherein the preset recognition result is a labeled recognition result corresponding to the training image, and the first function value is used to represent the error between the first recognition result and the preset recognition result; and a first update module, used to update the parameter values ​​of the image mask parameters based on the first function value along the direction of the first parameter gradient increase to obtain the target image mask, wherein the input parameters of the preset loss function include the image mask parameters, and the first parameter gradient is the gradient of the preset loss function corresponding to the image mask parameters.

[0009] As an optional solution, the device further includes: a determining unit, configured to determine the first parameter gradient as the product of the derivative of the preset loss function with respect to the image mask corresponding to the training image and the derivative of the image mask corresponding to the training image with respect to the image mask parameters.

[0010] As an optional solution, the apparatus further includes: a sampling unit, configured to randomly sample the image mask parameters to obtain initial parameter values ​​of the image mask parameters before occluding the first image region of the training image using the mask region of the initial image mask to obtain the first mask image; and an execution unit, configured to perform an image mask generation operation using a differentiable activation function based on the initial parameter values ​​of the image mask parameters to obtain the initial image mask, wherein the differentiable activation function is a monotonically increasing function.

[0011] As an optional solution, the sampling unit includes: a first sampling module, used to randomly sample the region shape parameters in the image mask parameters to obtain an initial shape parameter value, wherein the region shape parameters are used to represent the region shape of the mask region of the image mask corresponding to the training image; and a second sampling module, used to randomly sample the region position parameters in the image mask parameters to obtain an initial position parameter value, wherein the region position parameters are used to represent the region position of the mask region of the image mask corresponding to the training image.

[0012] As an optional solution, the first sampling module includes: a first sampling submodule, used to randomly sample the region size parameter in the image mask parameters to obtain an initial size parameter value, wherein the region size parameter is used to represent the region size of the mask region of the image mask corresponding to the training image; and a second sampling submodule, used to randomly sample the rotation angle parameter in the image mask parameters to obtain an initial rotation angle value, wherein the rotation angle parameter is used to represent the angle of rotation of the mask region of the image mask corresponding to the training image along a preset direction with the region center point of the mask region of the image mask corresponding to the training image as the center.

[0013] As an optional solution, the second sampling module includes: a third sampling submodule, used to randomly sample the center point position parameter in the image mask parameters to obtain the initial position parameter value, wherein the center point position parameter is used to represent the position of the region center point of the mask area of ​​the image mask corresponding to the training image.

[0014] As an optional scheme, the mask region indicated by the initial parameter values ​​of the image mask parameters is the initial mask region. The initial parameter values ​​of the image mask parameters include: an initial region length value for the region length of the initial mask region, an initial region width value for the region width of the initial mask region, an initial x-coordinate and an initial y-coordinate for the center point position of the initial mask region, and an initial rotation angle value for representing the rotation angle of the initial mask region. The rotation angle of the initial mask region refers to the angle by which the initial mask region rotates along a preset direction with the region center point as the center point. The execution unit includes: a first execution module, used to perform the following operations on each pixel position in the initial image mask as the current pixel position to obtain the initial image mask, wherein the current pixel position includes the current pixel x-coordinate. The pixel value at the current pixel position is determined by multiplying the function value corresponding to the differentiable activation function with the first reference value and the function value corresponding to the differentiable activation function with the second reference value. The first reference value is the product of the first coordinate difference and the cosine of the initial rotation angle, plus the product of the second coordinate difference and the sine of the initial rotation angle, minus half the initial region length. The second reference value is the product of the second coordinate difference and the cosine of the initial rotation angle, minus the product of the first coordinate difference and the sine of the initial rotation angle, minus half the initial region width. The first coordinate difference is the coordinate difference between the current pixel's horizontal coordinate and the initial center point's horizontal coordinate, and the second coordinate difference is the coordinate difference between the initial center point's vertical coordinate and the current pixel's vertical coordinate.

[0015] As an optional approach, the execution unit includes: a second execution module, configured to perform an image mask generation operation based on the initial parameter values ​​of the image mask parameters, using the differentiable activation function according to the resolution of the training image, to obtain the initial image mask, wherein the resolution of the initial image mask is equal to the resolution of the training image.

[0016] As an optional solution, the execution unit includes one of the following: a third execution module, used to perform an image mask generation operation using a logistic function based on the initial parameter values ​​of the image mask parameters to obtain the initial image mask; and a fourth execution module, used to perform an image mask generation operation using a preset hyperbolic tangent function based on the initial parameter values ​​of the image mask parameters to obtain the initial image mask, wherein the preset hyperbolic tangent function is a hyperbolic tangent function with bias and scaling terms.

[0017] As an optional approach, the first training unit includes: a second input module, used to input the first mask image into the first recognition model to obtain a second recognition result output by the first recognition model; a second determination module, used to determine a preset loss function and a second function value corresponding to the second recognition result and the preset recognition result, wherein the preset recognition result is a labeled recognition result corresponding to the training image, and the second function value is used to represent the error between the first recognition result and the preset recognition result; and a second update module, used to update the model parameters of the first recognition model based on the second function value along the direction of gradient descent of the second parameter to obtain the second recognition model, wherein the input parameters of the preset loss function include the model parameters of the first recognition model, and the gradient of the second parameter is the gradient of the preset loss function corresponding to the model parameters of the first recognition model.

[0018] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer-readable storage medium, wherein the computer program is configured to execute the above-described model training method based on mask images at runtime.

[0019] According to another aspect of the embodiments of this application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the model training method based on the mask image as described above.

[0020] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-described mask image-based model training method through the computer program.

[0021] In this embodiment, a method based on model loss-guided image mask drifting on training images is adopted. First, the mask region of the initial image mask is used to occlude the corresponding image region in the training image to obtain a first mask image. The first mask image is then used to train a first recognition model to obtain a second recognition model. Through this training, the model parameters of the recognition model are updated, and the training objective is to minimize the loss. Then, the image mask parameters of the initial image mask are updated with the objective of increasing the loss to obtain a target image mask. The mask region of the target image mask is then used to occlude the corresponding image region in the training image. The process involves obtaining a second mask image and using it to train a second recognition model, resulting in a third recognition image. Through this training, the model parameters of the recognition model are updated. The training objective is to minimize the loss. Since the mask image is controlled to drift on the training image with the goal of increasing the loss, the constructed mask image is more difficult for the recognition model to recognize, thus assisting in model training and avoiding overfitting. This achieves the technical effect of improving the generalization ability of the recognition model, thereby solving the problem of weak generalization ability of the model training method based on mask image in related technologies due to the randomness of mask selection. Attached Figure Description

[0022] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0023] Figure 1 This is a schematic diagram of an application environment for an optional mask image-based model training method according to an embodiment of this application;

[0024] Figure 2 This is a schematic flowchart of an optional mask image-based model training method according to an embodiment of this application;

[0025] Figure 3 This is a schematic diagram of an optional mask-based data augmentation method according to an embodiment of this application;

[0026] Figure 4 This is a schematic diagram of another optional mask-based data augmentation method according to an embodiment of this application;

[0027] Figure 5 This is a schematic diagram of another optional mask-based data augmentation method according to an embodiment of this application;

[0028] Figure 6 This is a schematic diagram of another optional mask-based data augmentation method according to an embodiment of this application;

[0029] Figure 7 This is a schematic diagram of another optional mask-based data augmentation method according to an embodiment of this application;

[0030] Figure 8 This is a schematic diagram of another optional mask-based data augmentation method according to an embodiment of this application;

[0031] Figure 9 This is a schematic diagram of another optional mask-based data augmentation method according to an embodiment of this application;

[0032] Figure 10 This is a schematic flowchart of another optional mask image-based model training method according to an embodiment of this application;

[0033] Figure 11 This is a schematic flowchart of another optional mask image-based model training method according to an embodiment of this application;

[0034] Figure 12 This is a schematic diagram of another optional mask-based data augmentation method according to an embodiment of this application;

[0035] Figure 13 This is a structural block diagram of an optional mask image-based model training device according to an embodiment of this application;

[0036] Figure 14 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of this application;

[0037] Figure 15 This is a structural block diagram of a computer system for an optional electronic device according to an embodiment of this application. Detailed Implementation

[0038] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0039] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0040] According to one aspect of the embodiments of this application, a model training method based on mask images is provided. Optionally, as an optional implementation, the above-described model training method based on mask images can be applied to, but is not limited to, [examples of other methods]. Figure 1 The environment shown may include, but is not limited to, a model training device 102, a network 110, and a server 112. The model training device 102 may include, but is not limited to, a display 108, a processor 106, and a memory 104. The specific process can be summarized in the following steps:

[0041] In step S102, server 112 sends the initial recognition model to be trained and the training image set used to train the initial recognition model to the model training device 102 via network 110.

[0042] The recognition model to be trained can be trained on server 112 or model training device 102 (e.g., a terminal device of a developer or other relevant personnel). In the scenario where model training is performed on model training device 102, the database 114 of server 112 can store the initial recognition model to be trained and a set of labeled training images used to train the initial recognition model. Based on a data retrieval request from model training device 102, the processing engine 116 of server 112 can send the initial recognition model and training image set stored in the database 114 of server 112 to model training device 102 via network 110.

[0043] In step S104, the model training device 102 uses training images from the training image set to train the initial recognition model and obtain an intermediate recognition model.

[0044] The model training device 102 can use training images from the training image set to train the initial recognition model multiple times until the iteration termination condition is met. The recognition model obtained after training can be used as the final recognition model. Alternatively, to improve the model's generalization ability, data augmentation (or data augmentation) can be performed on the training images to change the distribution of data in the training image set, and the augmented training images can be used to train intermediate recognition models. Here, the data augmentation technique used can be a region mask-based augmentation technique.

[0045] In step S106, the model training device 102 performs data augmentation on the training image based on the region mask to obtain the mask image, and uses the mask image to train the intermediate recognition model to obtain the target recognition model.

[0046] The model training device 102 can perform data augmentation on the training image based on a region mask to obtain an augmented training image (i.e., a mask image). After obtaining the intermediate recognition model, the model training device 102 can use the augmented training image to train the intermediate recognition model to obtain the target recognition model.

[0047] In step S108, the model training device 102 sends the model parameters of the target recognition model to the server 112 via the network 110.

[0048] After obtaining the target recognition model, the model training device 102 can send the model parameters of the target recognition model to the server 112 via the network 110. The processing engine 116 of the server 112 can save the received model parameters of the target recognition model to the database 114 of the server 112.

[0049] Optionally, the model training device 102 may include, but is not limited to, at least one of the following: desktop computers, workstations, AR (Augmented Reality), VR (Virtual Reality) and other virtual reality devices, etc. Portable devices with abundant computing resources, smart home appliances, in-vehicle devices, etc., may also be used as model training devices or auxiliary devices for model training (e.g., devices for sending instructions, displaying augmented training images, displaying recognition results, etc.) to train the recognition model. Portable devices may include, but are not limited to, at least one of the following: mobile phones (such as Android phones, iOS phones, etc.), laptops, tablets, handheld computers, MID (Mobile Internet Devices), PADs, etc. The aforementioned network 110 may include, but is not limited to, wired networks and wireless networks. The wired network includes local area networks (LANs), metropolitan area networks (MANs), and wide area networks (WANs). The wireless network includes Bluetooth, Wireless Fidelity (WIFI), and other networks that enable wireless communication. The aforementioned server 112 may be a single server, a server cluster consisting of multiple servers, or a cloud server. The above is merely an example, and no limitation is made in this embodiment.

[0050] Optionally, the above-mentioned model training method based on mask images can be executed by the model training device 102 or the server 112 alone, or by the model training device 102 and the server 112 working together, or by other processing devices besides the model training device 102 and the server 112, as long as the obtained model parameters can be applied to the recognition model.

[0051] As an optional implementation, taking the mask image-based model training method of this embodiment executed by the model training device 102 as an example, Figure 2 This is a flowchart illustrating an optional mask image-based model training method according to an embodiment of this application, as shown below. Figure 2 As shown, the process of this mask image-based model training method may include the following steps:

[0052] Step S202: Use the mask area of ​​the initial image mask to occlude the first image area of ​​the training image to obtain the first mask image.

[0053] The mask image-based model training method in this embodiment can be applied to the process of training a neural network model using training data. The training data can be training images, and the mask image refers to the image obtained by performing region masking on the image to be masked (i.e., the image obtained by occluding the image to be masked using an image mask). The neural network model can be a recognition model (e.g., a classification model) or other types of neural network models. In this embodiment, the model training of a recognition model using training images is used as an example for illustration.

[0054] Neural network models are models that use neural networks to perform specified tasks. Deep neural networks, due to their numerous parameters, are prone to overfitting during training. This results in models performing well on the training set but performing poorly in real-world testing, lacking generalization ability. Therefore, data augmentation techniques can be applied to neural network training to improve the generalization ability of the trained model and achieve better testing results. Here, data augmentation refers to using specific methods to augment the data in the training set, thereby changing the distribution of the data. Data augmentation techniques achieve stronger generalization ability by exposing the model to more images during training; at a deeper level, data augmentation can be seen as regularization of the model's output, preventing the neural network from focusing on outliers that deviate from half of the data distribution during training, thus achieving better generalization results.

[0055] As a commonly used data augmentation technique, region masking is not only simple and intuitive to implement, but also has a very significant effect on the training and regularization of neural networks. Currently, region masking is usually done by manually selecting a mask of a specific shape (e.g., a rectangle of a certain area) and placing it on the image, and then modifying the image content within the mask area (e.g., randomly filling in noise or a part of another image) to generate a new training image.

[0056] Here, region masking refers to using a black-and-white binary image (i.e., a binarized mask) to occlude a specific region (i.e., the masked region) in the original image. The black portion in the binary image corresponds to a portion of the original image that becomes black, while other portions remain unchanged, thereby extracting the part of interest from the original image. For example, a region mask-based data augmentation method can be represented by formula (1):

[0057]

[0058] in, Let I be the augmented image (i.e., the mask image), M be the original image (i.e., the training image), M be a binary mask (image mask, where each pixel can only take one value, 0 or 1) with the same resolution as the original image, and Δ be the content to be filled. The augmented image... By default, it shares the same label as I. Region masks typically select rectangles as mask regions; that is, one rectangular region in M ​​is 0, and other regions are 1. Different region masking methods differ in the selection of the fill content Δ. For example, the CutOut method fills the mask region with Gaussian random noise; the CutMix method crops another image in the training set to the same area as the mask as the fill; and the ResizeMix method scales another image in the training set to the same area and aspect ratio as the mask as the fill.

[0059] like Figure 3 As shown, for the original image (a), a rectangular mask is first placed on the image, and then noise is randomly filled in the mask area to obtain the mask image (b), or a part of another image is randomly filled in the mask area to obtain the mask image (c).

[0060] However, the aforementioned data augmentation methods based on region masks largely rely on randomness to amplify training data. Mask selection and content filling are both random; the masked and filled portions may appear anywhere in the original image, lacking guidance. Furthermore, they lack coupling with the final training objective of enhancing the generalization ability of the neural network training model. Here, the lack of guidance means that the masked regions are random, potentially obscuring non-critical areas in the original image (e.g., areas unrelated to or with low relevance to the recognition task of the model). Therefore, using masked images for model training has little effect on improving the model's generalization ability.

[0061] To at least partially address the aforementioned problems, this embodiment improves upon region-mask-based data augmentation techniques by optimizing the mask region based on randomly selected masks. This ensures that, after mask optimization, the area obscured by the optimized mask becomes more challenging to identify (e.g., obscuring areas such as...). Figure 3 The approach uses adversarial learning to update the geometric parameters of the mask by increasing the network loss (or maximizing the network loss), which involves generating a mask image using the image mask and training the recognition model with the generated mask image. This allows the image mask to more effectively occlude key areas in the training image, thereby achieving more guided and targeted data augmentation to improve the model's generalization ability and reduce the risk of overfitting.

[0062] When training a recognition model using training images, the image regions of the training images can be occluded using the mask region of an image mask to obtain a mask image. For a training image, the image mask used for region occlusion processing can be an initial image mask. The model training device can use the mask region of the initial image mask to occlude the corresponding image region (i.e., the first image region) in the training image to obtain a first mask image. The resolution of the initial image mask can be the same as the resolution of the training image. Here, the image mask can be a binary mask (binary image) used for region occlusion processing of the training image, and the mask region in the image mask can be configured through image mask parameters. The initial image mask is used to occlude the first image region of the training image. The mask region of the initial image mask can be randomly selected, and its shape can be preset, such as a rectangular region, or other shapes. In this embodiment, there are no limitations on the generation method of the initial image mask or the shape of the mask region.

[0063] For example, such as Figure 4 As shown, the mask region in image mask 1 is a rectangular region. By using the mask region of image mask 1 to occlude the image region corresponding to the training image, mask image 1 can be obtained.

[0064] Step S204: Use the first mask image to train the first recognition model to update the model parameters of the first recognition model and obtain the second recognition model.

[0065] After obtaining the first mask image, it can be applied to the model training process of the recognition model: the first recognition model is trained using the first mask image to update its model parameters. The first recognition model can be a recognition model from any model training stage from the initial recognition model to the target recognition model, such as the aforementioned intermediate recognition model, a recognition model obtained after training an intermediate recognition model using one or more mask images, or a recognition model from other stages.

[0066] Here, the training process of a deep neural network model is essentially the optimization of the loss function. The loss function determines the performance of the recognition model by comparing the deviation between its predicted and expected outputs, thereby identifying areas for optimization. A larger deviation results in a larger loss, and vice versa. To minimize the loss during model training, the model parameters can be adjusted to reduce the deviation between the predicted and expected outputs. Taking a deep neural network classifier as an example, the original labeled training dataset can be represented as set D. t ={(I x,c)}, where x∈R H×W This represents a two-dimensional data array with height H and width W; each training sample corresponds to a label c∈{0,1}. K , is a K-dimensional vector with one-hot encoding (one dimension is 1, and the rest are 0), where K is the number of classes in the training set. The training samples (training images) are input into the deep neural network classifier f(·; φ):R H×W →R K The training samples are mapped to K-dimensional score vectors, where each element of the K-dimensional score vector represents the classification score for the corresponding class. This allows deep neural network classifiers to be trained using cross-entropy loss.

[0067]

[0068] Where φ is the weight of the neural network (i.e., the model parameters), and the training process is the process of minimizing the cross-entropy loss.

[0069] Step S206: Adjust the parameter values ​​of the image mask parameters based on the loss corresponding to the second recognition model and the first mask image to obtain the target image mask, and use the mask area of ​​the target image mask to occlude the second image area of ​​the training image to obtain the second mask image.

[0070] To avoid the problem that training a model based on a mask image is not very effective in improving the model's generalization ability because the mask region occludes non-critical areas in the training image, this embodiment can use reparameterization and adversarial optimization to guide the mask to drift on the image: the parameter values ​​of the image mask are adjusted based on the loss corresponding to the second recognition model and the first mask image to update the image mask and obtain the target image mask. The mask region of the target image mask is then used to occlude the image region corresponding to the training image (i.e., the second image region) to obtain the second mask image. The region position of the second image region in the second mask image has drifted relative to the region position of the first image region in the first mask image. The goal of updating the image mask parameters is to increase the loss, that is, the loss corresponding to the second recognition model and the first mask image is lower than the loss corresponding to the second recognition model and the second mask image.

[0071] It should be noted that adjusting the image mask parameters is to construct more challenging data to assist in neural network training. For example, if the loss of the second recognition model corresponding to the second mask image is higher than the loss of the second recognition model corresponding to the first mask image, then the second mask image is more difficult for the first recognition model to recognize. Using a more challenging mask image to train the recognition model can improve the generalization ability of the recognition model.

[0072] Here, image mask parameters are used to represent the mask region of the image mask corresponding to the training image. The types of image mask parameters can be one or more, such as region shape parameters, region position parameters, etc. When adjusting the image mask parameters, a preset adjustment strategy can be attempted to adjust the mask parameters. Whether or not to allow adjustment of the model parameters is determined based on the loss between the second recognition model and the adjusted mask image, and the loss between the second recognition model and the first mask image. The preset adjustment strategy can be one or more, such as adjusting the parameter values ​​of the image mask parameters along a specified direction or distance, or other adjustment strategies (e.g., selecting the adjusted parameter values ​​from a set of candidate parameter values). This embodiment does not limit this approach.

[0073] For example, such as Figure 5 As shown, after adjusting the parameter values ​​of image mask 1, image mask 2 is obtained, and the mask image corresponding to image mask 2 is called mask image 2. Since the mask area in mask image 2 obscures the representative head of the animal object, the loss of the recognition model corresponding to mask image 2 is higher than the loss of the recognition model corresponding to mask image 1.

[0074] Step S208: Use the second mask image to train the second recognition model to update the model parameters of the second recognition model and obtain the third recognition model.

[0075] After obtaining the second mask image, it can be applied to the model training process of the recognition model: the second mask image can be used to train the second recognition model to update its model parameters. The recognition model obtained after updating the model parameters is the third recognition model. The goal of training the second recognition model is to minimize the loss, that is, the loss of the third recognition model corresponding to the second mask image is lower than the loss of the second recognition model corresponding to the second mask image.

[0076] Here, the training image set may include a group of training images. For each training image, the recognition model can be trained using the same model training method as described above based on region masks. This has already been explained and will not be repeated here.

[0077] Optionally, training the recognition model using training images and augmented training images can be performed sequentially. That is, first, the initial recognition model is trained using training images from the training image set to obtain an intermediate recognition model, and then the intermediate recognition model is trained using the augmented training images to obtain the target recognition model. Alternatively, it can be performed alternately. For example, training images and augmented training images can be used alternately to train the model and obtain the target recognition model. In this case, the training images and augmented training images are input sequentially. Or, the training image set can be updated by mixing training images and augmented training images, and the initial recognition model can be trained using the updated training images to obtain the target recognition model. In this case, the input order of training images and augmented training images is not fixed.

[0078] For example, such as Figure 6 As shown, when training the initial recognition model using the training image set, the initial recognition model can be trained sequentially using each training image in the training image set (training image 1, training image 2, etc.) to obtain an intermediate recognition model. Then, the mask-enhanced images obtained by data augmentation (mask image 1, mask image 2, etc.) can be used sequentially to train the intermediate recognition model to obtain an updated intermediate recognition model. Finally, the mask-enhanced images obtained by data augmentation again (mask image 1', mask image 2', etc.) can be used sequentially to train the updated intermediate recognition model to obtain the target recognition model.

[0079] For example, such as Figure 7 As shown, when training the initial recognition model using the training image set, the initial recognition model can be trained sequentially using each training image in the training image set to obtain an intermediate recognition model. Then, the mask enhancement image obtained by data augmentation and the mask enhancement image obtained by re-data augmentation can be used alternately to train the intermediate recognition model to obtain the target recognition model.

[0080] For example, such as Figure 8 As shown, when training the initial recognition model using the training image set, the initial recognition model can be trained by alternately using each training image in the training image set, the mask enhancement image obtained by data augmentation, and the mask enhancement image obtained by re-data augmentation, thus obtaining the target recognition model.

[0081] The embodiments provided in this application involve occluding a first image region of a training image using the mask region of an initial image mask to obtain a first mask image; training a first recognition model using the first mask image to update the model parameters of the first recognition model to obtain a second recognition model; adjusting the parameter values ​​of the image mask parameters based on the loss corresponding to the second recognition model and the first mask image to obtain a target image mask; and occluding a second image region of the training image using the mask region of the target image mask to obtain a second mask image. The image mask parameters represent the mask region of the image mask corresponding to the training image, and the loss of the second recognition model corresponding to the first mask image is lower than the loss of the second recognition model corresponding to the second mask image. Training a second recognition model using the second mask image to update the model parameters of the second recognition model to obtain a third recognition model solves the problem of weak generalization ability of mask image-based model training methods in related technologies due to the randomness of mask selection, thus improving the generalization ability of the model.

[0082] As an optional approach, a first recognition model is trained using a first mask image to update its model parameters, resulting in a second recognition model, including:

[0083] S11, input the first mask image into the first recognition model to obtain the second recognition result output by the first recognition model;

[0084] S12, determine the second function value corresponding to the preset loss function and the first recognition result and the preset recognition result, wherein the preset recognition result is the labeled recognition result corresponding to the training image, and the second function value is used to represent the error between the second recognition result and the preset recognition result;

[0085] S13, based on the second function value, update the model parameters of the first recognition model along the direction of gradient descent of the second parameter to obtain the second recognition model. The input parameters of the preset loss function include the model parameters of the first recognition model, and the gradient of the second parameter is the gradient of the preset loss function and the model parameters of the first recognition model.

[0086] The input parameters of the preset loss function include the model parameters of the recognition model. To improve the efficiency of model training, the model parameters of the recognition model can be updated along the direction of gradient descent corresponding to the model parameters of the recognition model. Here, gradient refers to the gradient of the multivariate function. Partial derivatives are vectors representing the partial derivatives of the obtained parameters. Gradient descent is mainly used to update weights (model parameters) in neural network models, that is, to update and adjust the model parameters in one direction to minimize the loss function.

[0087] It's important to note that the gradient of the loss function with respect to the parameters is the direction in which the loss function rises the fastest. To minimize the loss function, moving the parameters in the opposite direction of the gradient will decrease the loss function. In other words, during the backpropagation of the network, relevant errors can be backpropagated, and gradient descent can be used to update the model parameters in the opposite direction of the loss function's gradient to minimize the loss function.

[0088] In this embodiment, the first mask image is input into the first recognition model to obtain the second recognition result output by the first recognition model. The second recognition result is the predicted output of the recognition model, while the expected output corresponding to the first mask image is the preset recognition result (i.e., the labeled recognition result corresponding to the training image is the expected recognition result). There is a deviation between the second recognition result and the preset recognition result. After obtaining the second recognition result, a preset loss function and a second function value corresponding to the second recognition result and the preset recognition result can be determined. The second function value is used to represent the error between the second recognition result and the preset recognition result. The preset loss function can be the cross-entropy loss function or other loss functions. In this embodiment, the type of loss function is not limited.

[0089] Based on the obtained second function value, the model parameters of the first recognition model can be updated along the gradient descent direction corresponding to the preset loss function and the model parameters of the first recognition model, thus obtaining the second recognition model. Here, the input parameters of the preset loss function include the model parameters of the first recognition model. In other words, the preset loss function is a function of the model parameters of the first recognition model, and the gradient of the preset loss function with respect to the model parameters of the first recognition model is the direction in which the preset loss function rises the fastest. Updating the model parameters of the first recognition model along the opposite direction of the gradient can minimize the model loss.

[0090] Through the embodiments provided in this application, based on the error between the recognition result output by the recognition model and the expected recognition result, the model parameters of the recognition model are updated along the gradient descent direction corresponding to the loss function and the model parameters of the recognition model, thereby optimizing the model parameters and improving the efficiency of model training.

[0091] As an optional approach, the parameter values ​​of the image mask are adjusted based on the loss corresponding to the second recognition model and the first mask image to obtain the target image mask, including:

[0092] S21, input the first mask image into the second recognition model to obtain the first recognition result output by the second recognition model;

[0093] S22, determine the preset loss function and the first function value corresponding to the first recognition result and the preset recognition result, wherein the preset recognition result is the labeled recognition result corresponding to the training image, and the first function value is used to represent the error between the first recognition result and the preset recognition result;

[0094] S23, based on the first function value, update the parameter values ​​of the image mask parameters along the direction of the gradient of the first parameter to obtain the target image mask. The input parameters of the preset loss function include the image mask parameters, and the gradient of the second parameter is the gradient of the preset loss function corresponding to the image mask parameters.

[0095] To improve the accuracy of image mask parameter adjustment, the image mask parameters can be used as input parameters to a preset loss function. The mask parameters of the training image's mask are then updated based on gradient ascent, resulting in the updated image mask, i.e., the target image mask. Here, the goal of optimizing the mask region is to make the occluded areas more challenging to recognize after mask optimization. Therefore, updating the image mask parameters along the gradient ascent direction corresponding to the preset loss function and the mask parameters of the training image's mask increases the model loss. In this case, the mask image generated using the updated image mask is more difficult for the second recognition model to identify, thus avoiding overfitting and improving the generalization ability of the recognition model.

[0096] In this embodiment, after obtaining the second recognition model, the first mask image can be input into the second recognition model to obtain the first recognition result, and a first function value corresponding to the preset loss function and the first recognition result can be determined. The first recognition result can be the recognition result output by the second recognition model corresponding to the first mask image. Compared with the second recognition result, the error between the first recognition result and the preset recognition result is smaller, that is, the first function value is smaller than the second function value. Based on the obtained first function value, the mask parameters of the initial image mask can be updated along the gradient rising direction corresponding to the preset loss function and the image mask parameters to obtain the target image mask. Here, updating the mask parameters of the initial image mask along the gradient rising direction can maximize the loss.

[0097] It should be noted that although gradient ascent / descent operations are used for updating in the embodiments of this application, other update strategies can also be used, such as gradient updates with momentum, Adam (adaptive moment estimation) optimizer, etc.

[0098] Through the embodiments provided in this application, based on the error between the recognition result output by the recognition model and the expected recognition result, the mask parameters of the image mask of the training image are updated along the direction of gradient ascent corresponding to the loss function and the image mask parameters, which can achieve more guided and targeted data augmentation.

[0099] As an optional approach, the above method also includes:

[0100] S31, the product of the derivative of the preset loss function with respect to the image mask corresponding to the training image and the derivative of the image mask corresponding to the training image with respect to the image mask parameters is determined as the gradient of the first parameter.

[0101] In this embodiment, the loss function can be expressed as a function of the mask image and the model parameters of the recognition model. The goal of adjusting the image mask parameters is to maximize the training loss function of the recognition model, i.e.,

[0102]

[0103] Where, Φ geo For image mask parameters, M(Φ) geo ) represents the image mask, I⊙M(Φ geo )+Δ⊙.1-M(Φ geo )) is a mask image.

[0104] By inverting the gradient and using the chain rule, the gradient of the loss L with respect to the image mask parameters can be obtained. That is, the product of the derivative of the preset loss function with respect to the image mask corresponding to the training image and the derivative of the image mask corresponding to the training image with respect to the image mask parameters is determined as the gradient of the first parameter. Thus, the parameter values ​​of the image mask parameters can be updated based on the gradient of the first parameter. For example, the update method of the parameter values ​​of the image mask parameters can be as shown in formula (2):

[0105]

[0106] Where Φ0 can be the current parameter value of the image mask parameters (e.g., the initial parameter value). For the current mask image (e.g., the initialized mask enhancement image), γ represents the update step.

[0107] The embodiments provided in this application improve the accuracy and efficiency of image model parameter updates by updating the parameter values ​​of image mask parameters through gradient inversion and chaining.

[0108] As an optional approach, before occluding the first image region of the training image using the mask region of the initial image mask to obtain the first mask image, the above method further includes:

[0109] S41, Randomly sample the image mask parameters to obtain the initial parameter values ​​of the image mask parameters;

[0110] S42, based on the initial parameter values ​​of the image mask parameters, perform the image mask generation operation using a differentiable activation function to obtain the initial image mask, wherein the differentiable activation function is a monotonically increasing function.

[0111] In this embodiment, to obtain an initial image mask, the image mask parameters can be randomly sampled to obtain initial parameter values. The image mask parameters can include one or more mask parameters. When the image mask parameters include multiple mask parameters, all mask parameters can be randomly sampled, or at least some of the mask parameters can be randomly sampled while the parameter values ​​of the other mask parameters remain unchanged. For example, if the image mask parameters include five mask parameters, three of them can be randomly sampled while the parameter values ​​of the other two mask parameters remain unchanged.

[0112] Based on the initial parameter values ​​of the image mask, an image mask generation operation can be performed to obtain an initial image mask. To ensure that the geometric parameters of the mask are optimized along with the network training objective, the image mask can be configured as a differentiable structure to facilitate solving the gradient between the preset loss function and the image mask parameters. Therefore, when generating the image mask, a differentiable activation function can be used to perform the image mask generation operation to obtain the corresponding image mask. For the initial image mask, the initial image mask can be obtained by performing the image mask generation operation using a differentiable activation function based on the initial parameter values ​​of the image mask.

[0113] Here, the differentiable activation function δ(·) can be any differentiable activation function that satisfies the following condition:

[0114] lim x→∞ δ(x)=1,lim x→-∞ δ(x)=0

[0115] It can be a monotonically increasing function, such as the sigmoid function, or other monotonically increasing functions with the same mathematical properties. In this embodiment, there is no limitation on differentiable activation functions.

[0116] The embodiments provided in this application improve the ease of generating the initial image mask by randomly sampling the image mask parameters; and by using a monotonically increasing differentiable activation function to perform the image mask generation operation, the gradient of the loss function with respect to the image mask parameters can be easily determined, thus improving the ease of adjusting the image mask.

[0117] As an optional approach, the image mask parameters are randomly sampled to obtain initial parameter values, including:

[0118] S51, Randomly sample the region shape parameter in the image mask parameters to obtain the initial shape parameter value, wherein the region shape parameter is used to represent the region shape of the mask region of the image mask corresponding to the training image;

[0119] S52, randomly sample the region position parameters in the image mask parameters to obtain initial position parameter values, where the region position parameters are used to represent the region position of the mask region of the image mask corresponding to the training image.

[0120] In this embodiment, the image mask parameters may include a variety of mask parameters, including but not limited to at least one of the following: a region shape parameter representing the region shape of the mask area corresponding to the training image, and a region position parameter representing the region position of the mask area corresponding to the training image. Here, adjusting the region shape of the mask area in the image mask can change the region shape of the image region in the training image that is occluded by the mask area of ​​the image mask, thereby updating the mask image; adjusting the region position of the mask area in the image mask can change the region position of the image region in the training image that is occluded by the mask area of ​​the image mask, thereby updating the mask image.

[0121] As an optional implementation, the region shape parameters in the image mask parameters can be randomly sampled to obtain initial shape parameter values. The random sampling of the region shape parameters can be performed within a preset region shape parameter range. The preset region shape parameter range can be set according to at least one of the image shape and region size of the training image, so as to avoid the impact on the recognition accuracy of the recognition model due to the image region being too large or too small being obscured by the mask region of the image mask.

[0122] As another optional implementation, the region position parameters in the image mask parameters can be randomly sampled to obtain initial position parameter values. The random sampling of the region position parameters can be performed within a preset region position parameter range. The preset region position parameter range can be set according to at least one of the image shape and region size of the training image, so as to avoid the impact on the recognition accuracy of the recognition model due to the image region being too biased by the mask region of the image mask.

[0123] The embodiments provided in this application allow for greater flexibility in image mask initialization by randomly sampling at least one of the region shape parameters and region position parameters to obtain an initialized random mask.

[0124] As an optional approach, the region shape parameters in the image mask parameters are randomly sampled to obtain initial shape parameter values, including:

[0125] S61, Randomly sample the region size parameter in the image mask parameters to obtain the initial size parameter value, wherein the region size parameter is used to represent the region size of the mask region of the image mask corresponding to the training image;

[0126] S62, randomly sample the rotation angle parameter in the image mask parameters to obtain an initial rotation angle value, wherein the rotation angle parameter is used to represent the angle by which the mask region of the image mask corresponding to the training image rotates along a preset direction with the center point of the mask region of the image mask corresponding to the training image as the center.

[0127] In this embodiment, the region shape parameter may include one or more, including but not limited to at least one of the following: region size parameter, rotation angle parameter, wherein the region size parameter is used to represent the region size of the mask region of the image mask corresponding to the training image, and the rotation angle parameter is used to represent the angle at which the mask region of the image mask corresponding to the training image rotates along a preset direction with the region center point of the mask region of the image mask corresponding to the training image as the center.

[0128] As an optional implementation, the region shape parameter may include the region size parameter. For mask regions of different shapes, the corresponding type of region size parameter may be different. For example, if the mask region is a rectangular region (e.g., ... Figure 9 (As shown on the left), the region size parameter can include at least one of the region length and region width. If the mask region is a circular region (e.g., Figure 9 As shown on the right), the region size parameter can include the region radius. Similar region size parameters can also be configured for mask regions of other shapes (e.g., ellipse, triangle, or other shapes). This embodiment does not limit this.

[0129] As an alternative implementation, the region shape parameter may include a rotation angle parameter. Since the rotation angle parameter represents the angle of rotation of the mask region with its center point as the center, and the mask region rotating along a preset direction (which can be clockwise or counterclockwise), the region shape parameter may include a rotation angle parameter for mask regions whose shape changes with rotation (e.g., rectangular regions), while for mask regions whose shape does not change with rotation (e.g., circular regions), the region shape parameter may not include a rotation angle parameter.

[0130] The embodiments provided in this application allow for greater flexibility in image mask initialization by randomly sampling at least one of the region size parameters and rotation angle parameters to obtain an initialized random mask.

[0131] As an optional approach, the region location parameters in the image mask parameters are randomly sampled to obtain initial location parameter values, including:

[0132] S71, randomly sample the center point position parameter in the image mask parameters to obtain the initial position parameter value, wherein the center point position parameter is used to represent the position of the center point of the mask region of the image mask corresponding to the training image.

[0133] The region location parameters may include parameters used to identify any position within the mask region, such as the top-left vertex, top-right vertex, bottom-left vertex, and bottom-right vertex of the mask region. In this embodiment, the region location parameters may include center point location parameters that represent the location of the center point of the mask region corresponding to the training image. By combining the location of the center point of the mask region with the region shape parameter value, the mask region in the image mask can be determined.

[0134] Optionally, the center point location parameter may include at least one of the x-coordinate and y-coordinate of the region center point. If the x-coordinate of the region center point of the mask region is fixed, the center point location parameter may include only the y-coordinate of the region center point. If the y-coordinate of the region center point of the mask region is fixed, the center point location parameter may include only the x-coordinate of the region center point.

[0135] Taking a rectangular image mask as an example, the five geometric parameters Φ of the rectangular mask... geo =[x,y,w,h,θ- T , where (x,y) represents the coordinates of the center point, (w,h) represents the length and width, and θ represents the clockwise rotation angle of the rectangular mask portion relative to the horizontal axis with the center point (x,y) as the center.

[0136] The embodiments provided in this application allow for greater flexibility in image mask initialization by randomly sampling the center point position parameters to obtain an initialized random mask.

[0137] As an optional approach, the mask region can be a rectangular region, and the image mask parameters can include region length parameters, region width parameters, center point position parameters, and rotation angle parameters. The meanings of each parameter are similar to those in the previous embodiments and will not be repeated here. The mask region indicated by the initial parameter values ​​of the image mask parameters is the initial mask region. The initial parameter values ​​of the image mask parameters include: an initial region length value for the region length of the initial mask region, an initial region width value for the region width of the initial mask region, an initial x-coordinate and an initial y-coordinate for the center point position of the initial mask region, and an initial rotation angle value for representing the rotation angle of the initial mask region. The rotation angle of the initial mask region refers to the angle by which the initial mask region rotates along a preset direction with the center point of the initial mask region as the center point.

[0138] Correspondingly, based on the initial parameter values ​​of the image mask parameters, an image mask generation operation is performed using a differentiable activation function to obtain an initial image mask, including:

[0139] S81, each pixel position in the initial image mask is used as the current pixel position to perform pixel value determination operation to obtain the initial image mask.

[0140] For the current pixel position, it can include the current pixel's horizontal coordinate and vertical coordinate. When performing image mask generation operation using a differentiable activation function, the function value corresponding to the differentiable activation function with the first reference value and the function value corresponding to the differentiable activation function with the second reference value can be determined separately. The product of the two function values ​​is determined as the pixel value of the current pixel position. Here, the first reference value is the product of the first coordinate difference and the cosine of the initial rotation angle value, plus the product of the second coordinate difference and the sine of the initial rotation angle value, minus half of the initial region length value. The second reference value is the product of the second coordinate difference and the cosine of the initial rotation angle value, minus the product of the first coordinate difference and the sine of the initial rotation angle value, minus half of the initial region width value. The first coordinate difference is the coordinate difference between the current pixel's horizontal coordinate and the initial center point's horizontal coordinate, and the second coordinate difference is the coordinate difference between the initial center point's vertical coordinate and the current pixel's vertical coordinate.

[0141] For example, using the five geometric parameters of the rectangular mask, the pixel value of the mask M at each pixel position (u,v) can be as shown in formula (3):

[0142]

[0143] δ(·) can be any differentiable activation function, and this part can be instantiated using the sigmoid function as shown in equation (4):

[0144]

[0145] The embodiments provided in this application demonstrate how the pixel values ​​at each pixel position in an image mask can be determined by combining a differentiable activation function with the region parameters of a rectangular mask area, thereby improving the convenience of image mask generation.

[0146] As an optional approach, based on the initial parameter values ​​of the image mask parameters, an image mask generation operation is performed using a differentiable activation function to obtain an initial image mask, including:

[0147] S91, based on the initial parameter values ​​of the image mask parameters, use a differentiable activation function to perform an image mask generation operation according to the resolution of the training image to obtain an initial image mask, wherein the resolution of the initial image mask is equal to the resolution of the training image.

[0148] In this embodiment, the resolution of the generated initial image mask can be related to the resolution of the training image; for example, they can be the same or proportional. If they are proportional, the generated initial image mask can be scaled to make its resolution the same as the training image's resolution.

[0149] To simplify the region masking process and improve its efficiency, the resolution of the generated initial image mask can be controlled to be equal to the resolution of the training image. That is, based on the initial parameter values ​​of the image mask parameters, a differentiable activation function is used to perform the image mask generation operation according to the resolution of the training image, resulting in the initial image mask. In this case, no further processing is required on the generated initial image mask; it can be used directly for region masking.

[0150] The embodiments provided in this application demonstrate that using a differentiable activation function to perform image mask generation operations according to the resolution of the training image can simplify the region masking process and improve the efficiency of region masking.

[0151] As an optional approach, based on the initial parameter values ​​of the image mask parameters, an image mask generation operation is performed using a differentiable activation function to obtain an initial image mask, including one of the following:

[0152] S101, Based on the initial parameter values ​​of the image mask parameters, the image mask generation operation is performed using the logistic function to obtain the initial image mask;

[0153] S103, Based on the initial parameter values ​​of the image mask parameters, perform the image mask generation operation using a preset hyperbolic tangent function to obtain the initial image mask, wherein the preset hyperbolic tangent function is a hyperbolic tangent function with bias and scaling terms.

[0154] In this embodiment, the differentiable activation function can be any differentiable monotonically increasing function, including but not limited to one of the following: the logistic function (sigmoid function), the hyperbolic tangent function with bias and scaling terms (tanh function), i.e., a preset hyperbolic tangent function. Other monotonically increasing functions with the same mathematical properties can also be used as differentiable activation functions in the process of generating image masks.

[0155] When generating the initial image mask, the image mask generation operation can be performed using the logistic function based on the initial parameter values ​​of the image mask parameters to obtain the initial image mask, or the image mask generation operation can be performed using the preset hyperbolic tangent function based on the initial parameter values ​​of the image mask parameters to obtain the initial image mask.

[0156] The embodiments provided in this application improve the flexibility of image mask generation by using a differentiable monotonically increasing function to generate the image mask.

[0157] The model training method based on mask images in this application embodiment will be explained below with reference to optional examples. In this optional example, the mask region is a rectangular region, and the image mask parameter Φ geo There are 5 geometric parameters: [x,y,w,h,θ- T .

[0158] This optional example provides a data augmentation scheme based on gradient drift. Based on random mask selection, a gradient optimization strategy is used to optimize the selection of mask geometric parameters using the gradient of the network's training loss function, achieving better data augmentation and improving model generalization ability. Since the mask M itself has the same resolution as x, and each position is a discrete value {0,1}, it is difficult to directly optimize using gradient updates. Therefore, a reparameterization approach can be adopted, representing M as a differentiable structure, as shown in formula (2). Through the reparameterization described above, the mask can be optimized by optimizing the geometric parameters Φ. geo This is used to achieve the drifting of the mask geometry region.

[0159] Because it involves mask parameter Φ geo Instead of alternating updates of the neural network weights φ, and with the opposite optimization objective, this optional example uses gradient retracement and alternating updates to achieve the aforementioned optimization objective. See also Figure 10 and Figure 11 For a training sample (I,c), the iteration process of one training session can be divided into the following steps:

[0160] Step S1102, randomly sample Φ0=[x0,y0,w0,h0,θ0- T As the initialization mask (i.e., the initial image mask), the initialized mask-enhanced image is obtained. (i.e., initial mask image), mask enhancement image As shown in formula (5):

[0161]

[0162] Step S1104, will The input is used to train the neural network. The backpropagated parameter gradients are used to update the model parameters φ. At the same time, the loss L is obtained relative to the geometric parameters Φ through gradient inversion and the chain rule. geo The gradient is used to update the geometric parameters Φ. geo The update method of model parameter φ and geometric parameter Φ geo The update method can be shown in formula (6) and formula (2):

[0163]

[0164] like Figure 12 As shown, after adjusting the parameter values ​​of the mask parameters of image mask 1, image mask 3 is obtained, and the mask image corresponding to image mask 3 is mask image 3. Compared with image mask 1, image mask 3 has updated the region size, region position, and relative angle with the horizontal axis in the rectangular region.

[0165] Step S1106, update the geometric parameters Φ geo Reused for mask generation and data augmentation of the image The image is then fed into the neural network for retraining and to update the model parameters of the neural network. The update method for the model parameter φ can be as shown in formula (7):

[0166]

[0167] This optional example demonstrates how different granularities of embedding vector generation can be used to categorize newly added applications and discover applications of different types but with the same audience.

[0168] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0169] According to another aspect of the embodiments of this application, a mask-based model training apparatus for implementing the above-described mask-based model training method is also provided. Figure 13 This is a structural block diagram of an optional mask image-based model training device according to an embodiment of this application, such as... Figure 13 As shown, the device may include:

[0170] The first processing unit 1302 is used to perform occlusion processing on the first image region of the training image using the mask region of the initial image mask to obtain the first mask image.

[0171] The first training unit 1304 is connected to the first processing unit 1302 and is used to train the first recognition model using the first mask image to update the model parameters of the first recognition model and obtain the second recognition model.

[0172] The adjustment unit 1306 is connected to the first training unit 1304 and is used to adjust the parameter values ​​of the image mask parameters based on the loss corresponding to the second recognition model and the first mask image to obtain the target image mask. The image mask parameters are used to represent the mask area of ​​the image mask corresponding to the training image.

[0173] The second processing unit 1308 is connected to the adjustment unit 1306 and is used to occlude the second image region of the training image using the mask region of the target image mask to obtain the second mask image. The loss of the second recognition model corresponding to the first mask image is lower than the loss of the second recognition model corresponding to the second mask image.

[0174] The second training unit 1310, connected to the second processing unit 1308, is used to train the second recognition model using the second mask image to update the model parameters of the second recognition model and obtain the third recognition model.

[0175] It should be noted that the first processing unit 1302 in this embodiment can be used to execute the above step S202, the first training unit 1304 in this embodiment can be used to execute the above step S204, the adjustment unit 1306 and the second processing unit 1308 in this embodiment can be used to execute the above step S206, and the second training unit 1310 in this embodiment can be used to execute the above step S208.

[0176] The embodiments provided in this application involve occluding a first image region of a training image using the mask region of an initial image mask to obtain a first mask image; training a first recognition model using the first mask image to update the model parameters of the first recognition model to obtain a second recognition model; adjusting the parameter values ​​of the image mask parameters based on the loss corresponding to the second recognition model and the first mask image to obtain a target image mask; and occluding a second image region of the training image using the mask region of the target image mask to obtain a second mask image. The image mask parameters represent the mask region of the image mask corresponding to the training image, and the loss of the second recognition model corresponding to the first mask image is lower than the loss of the second recognition model corresponding to the second mask image. Training a second recognition model using the second mask image to update the model parameters of the second recognition model to obtain a third recognition model solves the problem of weak generalization ability of mask image-based model training methods in related technologies due to the randomness of mask selection, thus improving the generalization ability of the model.

[0177] As an optional solution, the adjustment unit includes:

[0178] The first input module is used to input the first mask image into the second recognition model to obtain the first recognition result output by the second recognition model;

[0179] The first determining module is used to determine the first function value corresponding to the preset loss function, the second recognition result, and the preset recognition result, wherein the preset recognition result is a labeled recognition result corresponding to the training image, and the first function value is used to represent the error between the first recognition result and the preset recognition result;

[0180] The first update module is used to update the parameter values ​​of the image mask parameters based on the first function value and along the direction of the gradient of the first parameter to obtain the target image mask. The input parameters of the preset loss function include the image mask parameters, and the gradient of the first parameter is the gradient of the preset loss function and the image mask parameters.

[0181] Optional examples of this implementation scheme can be found in the examples shown in the mask image-based model training method described above, and will not be repeated here.

[0182] As an optional solution, the above-mentioned device further includes:

[0183] The determining unit is used to determine the first parameter gradient as the product of the derivative of the preset loss function with respect to the image mask corresponding to the training image and the derivative of the image mask corresponding to the training image with respect to the image mask parameters.

[0184] Optional examples of this implementation scheme can be found in the examples shown in the mask image-based model training method described above, and will not be repeated here.

[0185] As an optional solution, the above-mentioned device further includes:

[0186] The sampling unit is used to perform occlusion processing on the first image region of the training image using the mask region of the initial image mask. Before obtaining the first mask image, the image mask parameters are randomly sampled to obtain the initial parameter values ​​of the image mask parameters.

[0187] The execution unit is used to perform an image mask generation operation based on the initial parameter values ​​of the image mask parameters using a differentiable activation function to obtain an initial image mask, wherein the differentiable activation function is a monotonically increasing function.

[0188] Optional examples of this implementation scheme can be found in the examples shown in the mask image-based model training method described above, and will not be repeated here.

[0189] As an optional scheme, the sampling unit includes:

[0190] The first sampling module is used to randomly sample the region shape parameters in the image mask parameters to obtain initial shape parameter values, wherein the region shape parameters are used to represent the region shape of the mask region of the image mask corresponding to the training image;

[0191] The second sampling module is used to randomly sample the region position parameters in the image mask parameters to obtain initial position parameter values, wherein the region position parameters are used to represent the region position of the mask region of the image mask corresponding to the training image.

[0192] Optional examples of this implementation scheme can be found in the examples shown in the mask image-based model training method described above, and will not be repeated here.

[0193] As an optional approach, the first sampling module includes:

[0194] The first sampling submodule is used to randomly sample the region size parameter in the image mask parameters to obtain the initial size parameter value, wherein the region size parameter is used to represent the region size of the mask region of the image mask corresponding to the training image;

[0195] The second sampling submodule is used to randomly sample the rotation angle parameter in the image mask parameters to obtain an initial rotation angle value. The rotation angle parameter is used to represent the angle by which the mask region of the image mask corresponding to the training image is rotated along a preset direction with the center point of the mask region of the image mask corresponding to the training image as the center.

[0196] Optional examples of this implementation scheme can be found in the examples shown in the mask image-based model training method described above, and will not be repeated here.

[0197] As an optional solution, the second sampling module includes:

[0198] The third sampling submodule is used to randomly sample the center point position parameter in the image mask parameters to obtain the initial position parameter value. The center point position parameter is used to represent the position of the center point of the mask region of the image mask corresponding to the training image.

[0199] Optional examples of this implementation scheme can be found in the examples shown in the mask image-based model training method described above, and will not be repeated here.

[0200] As an optional scheme, the mask region indicated by the initial parameter values ​​of the image mask parameters is the initial mask region. The initial parameter values ​​of the image mask parameters include: an initial region length value for the region length of the initial mask region, an initial region width value for the region width of the initial mask region, initial x-coordinate and y-coordinate of the initial center point for the position of the center point of the initial mask region, and an initial rotation angle value for representing the rotation angle of the initial mask region. The rotation angle of the initial mask region refers to the angle by which the initial mask region rotates along a preset direction with the center point of the initial mask region as the center point. The execution unit includes:

[0201] The first execution module is used to perform the following operations on each pixel position in the initial image mask as the current pixel position to obtain the initial image mask, wherein the current pixel position includes the current pixel's horizontal coordinate and the current pixel's vertical coordinate:

[0202] The pixel value at the current pixel position is determined by multiplying the function value corresponding to the first reference value and the function value corresponding to the second reference value of the differentiable activation function. The first reference value is the product of the first coordinate difference and the cosine of the initial rotation angle, plus the product of the second coordinate difference and the sine of the initial rotation angle, minus half of the initial region length. The second reference value is the product of the second coordinate difference and the cosine of the initial rotation angle, minus the product of the first coordinate difference and the sine of the initial rotation angle, minus half of the initial region width.

[0203] The first coordinate difference is the difference between the horizontal coordinate of the current pixel and the horizontal coordinate of the initial center point, and the second coordinate difference is the difference between the vertical coordinate of the initial center point and the vertical coordinate of the current pixel.

[0204] Optional examples of this implementation scheme can be found in the examples shown in the mask image-based model training method described above, and will not be repeated here.

[0205] As an optional solution, the execution unit includes:

[0206] The second execution module is used to perform an image mask generation operation based on the initial parameter values ​​of the image mask parameters, using a differentiable activation function according to the resolution of the training image, to obtain an initial image mask, wherein the resolution of the initial image mask is equal to the resolution of the training image.

[0207] Optional examples of this implementation scheme can be found in the examples shown in the mask image-based model training method described above, and will not be repeated here.

[0208] As an optional solution, the execution unit includes one of the following:

[0209] The third execution module is used to perform image mask generation operation using the logical stear function based on the initial parameter values ​​of the image mask parameters to obtain the initial image mask;

[0210] The fourth execution module is used to perform an image mask generation operation based on the initial parameter values ​​of the image mask parameters using a preset hyperbolic tangent function to obtain an initial image mask. The preset hyperbolic tangent function is a hyperbolic tangent function with bias and scaling terms.

[0211] Optional examples of this implementation scheme can be found in the examples shown in the mask image-based model training method described above, and will not be repeated here.

[0212] As an optional approach, the first training unit includes:

[0213] The first input module is used to input the first mask image into the first recognition model to obtain the second recognition result output by the first recognition model;

[0214] The first determining module is used to determine the preset loss function and the second function value corresponding to the second recognition result and the preset recognition result, wherein the preset recognition result is the labeled recognition result corresponding to the training image, and the second function value is used to represent the error between the first recognition result and the preset recognition result;

[0215] The first update module is used to update the model parameters of the first recognition model based on the second function value and along the direction of gradient descent of the second parameter to obtain the second recognition model. The input parameters of the preset loss function include the model parameters of the first recognition model, and the gradient of the second parameter is the gradient of the preset loss function and the model parameters of the first recognition model.

[0216] Optional examples of this implementation scheme can be found in the examples shown in the mask image-based model training method described above, and will not be repeated here.

[0217] According to another aspect of the embodiments of this application, an electronic device for implementing the above-described mask image-based model training method is also provided. This electronic device may be... Figure 1 The terminal device or server shown. This embodiment uses this electronic device as an example for illustration. Figure 14 As shown, the electronic device includes a memory 1402 and a processor 1404. The memory 1402 stores a computer program, and the processor 1404 is configured to execute the steps of any of the above method embodiments via the computer program.

[0218] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0219] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0220] S1, use the mask region of the initial image mask to occlude the first image region of the training image to obtain the first mask image;

[0221] S2, use the first mask image to train the first recognition model, so as to update the model parameters of the first recognition model and obtain the second recognition model;

[0222] S3, adjust the parameter values ​​of the image mask parameters based on the loss corresponding to the second recognition model and the first mask image to obtain the target image mask, and use the mask region of the target image mask to occlude the second image region of the training image to obtain the second mask image. Here, the image mask parameters are used to represent the mask region of the image mask corresponding to the training image, and the loss of the second recognition model corresponding to the first mask image is lower than the loss of the second recognition model corresponding to the second mask image.

[0223] S4. Use the second mask image to train the second recognition model to update the model parameters of the second recognition model and obtain the third recognition model.

[0224] Alternatively, as those skilled in the art will understand, Figure 14 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones (such as Android phones, iOS phones, etc.), tablets, PDAs, and other terminal devices such as MIDs and PADs. Figure 14 This does not limit the structure of the aforementioned electronic devices. For example, the electronic device may also include components that are more... Figure 14The more or fewer components shown (such as network interfaces, etc.), or having the same Figure 14 The different configurations shown.

[0225] The memory 1402 can be used to store software programs and modules, such as the program instructions / modules corresponding to the mask image-based model training method and apparatus in this embodiment. The processor 1404 executes various functional applications and data processing by running the software programs and modules stored in the memory 1402, thereby realizing the aforementioned mask image-based model training method. The memory 1402 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1402 may further include memory remotely located relative to the processor 1404, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 1402 may be used, but is not limited to, to store information such as recognition models, model parameters, image masks, and training images. As an example, such as... Figure 14 As shown, the memory 1402 may include, but is not limited to, the first processing unit 1302, the first training unit 1304, the adjustment unit 1306, the second processing unit 1308, and the second training unit 1310 in the mask image-based model training device. Furthermore, it may also include, but is not limited to, other module units in the mask image-based model training device, which will not be elaborated upon in this example.

[0226] Optionally, the transmission device 1406 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 1406 includes a Network Interface Controller (NIC), which can be connected to other network devices and routers via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 1406 is a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0227] In addition, the aforementioned electronic device also includes: a display 1408 for displaying training images, mask images, etc.; and a connection bus 1410 for connecting various module components in the aforementioned electronic device.

[0228] In other embodiments, the aforementioned terminal device or server can be a node in a distributed system, wherein the distributed system can be a blockchain system, which is a distributed system formed by connecting multiple nodes through network communication. The nodes can form a peer-to-peer (P2P) network, and any form of computing device, such as a server, terminal, or other electronic device, can become a node in the blockchain system by joining this peer-to-peer network.

[0229] According to one aspect of this application, a computer program product is provided, comprising a computer program / instructions containing program code for performing the methods shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via communication section 1509, and / or installed from removable medium 1511. When the computer program is executed by central processing unit 1501, it performs various functions provided in the embodiments of this application. The above embodiment numbers are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0230] Figure 15 This is a structural block diagram of a computer system for an optional electronic device according to an embodiment of this application. For example... Figure 15 As shown, the computer system 1500 includes a central processing unit (CPU) 1501, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 1502 or programs loaded from storage section 1508 into random access memory (RAM). The RAM 1503 also stores various programs and data required for system operation. The CPU 1501, ROM 1502, and RAM 1503 are interconnected via a bus 1504. An input / output interface 1505 (I / O interface) is also connected to the bus 1504.

[0231] The following components are connected to the input / output interface 1505: an input section 1506 including a keyboard, mouse, etc.; an output section 1507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1508 including a hard disk, etc.; and a communication section 1509 including a network interface card such as a local area network card, modem, etc. The communication section 1509 performs communication processing via a network such as the Internet. A drive 1510 is also connected to the input / output interface 1505 as needed. A removable medium 1511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1510 as needed so that computer programs read from it can be installed into the storage section 1508 as needed.

[0232] Specifically, according to embodiments of this application, the processes described in the various method flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1509, and / or installed from removable medium 1511. When the computer program is executed by central processing unit 1501, it performs various functions defined in the system of this application.

[0233] It should be noted that, Figure 15 The computer system 1500 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0234] According to one aspect of this application, a computer-readable storage medium is provided, from which a processor of a computer device reads computer instructions, and executes the computer instructions to cause the computer device to perform the methods provided in the various optional implementations of the above embodiments.

[0235] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0236] S1, use the mask region of the initial image mask to occlude the first image region of the training image to obtain the first mask image;

[0237] S2, use the first mask image to train the first recognition model, so as to update the model parameters of the first recognition model and obtain the second recognition model;

[0238] S3, adjust the parameter values ​​of the image mask parameters based on the loss corresponding to the second recognition model and the first mask image to obtain the target image mask, and use the mask region of the target image mask to occlude the second image region of the training image to obtain the second mask image. Here, the image mask parameters are used to represent the mask region of the image mask corresponding to the training image, and the loss of the second recognition model corresponding to the first mask image is lower than the loss of the second recognition model corresponding to the second mask image.

[0239] S4. Use the second mask image to train the second recognition model to update the model parameters of the second recognition model and obtain the third recognition model.

[0240] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0241] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0242] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0243] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0244] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0245] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or at least two units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0246] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A model training method based on mask images, characterized in that, include: The first image region of the training image is occluded using the mask region of the initial image mask to obtain the first mask image; The first recognition model is trained using the first mask image to update the model parameters of the first recognition model and obtain the second recognition model. The first mask image is input into the second recognition model to obtain the first recognition result output by the second recognition model. A first function value is determined corresponding to a preset loss function and the first recognition result and the preset recognition result, wherein the preset recognition result is a labeled recognition result corresponding to the training image, and the first function value is used to represent the error between the first recognition result and the preset recognition result; Based on the first function value, the parameter values ​​of the image mask parameters are updated along the direction of the gradient of the first parameter to obtain the target image mask. The image mask parameters are used to represent the mask region of the image mask corresponding to the training image. The input parameters of the preset loss function include the image mask parameters. The first parameter gradient is the gradient of the preset loss function and the image mask parameters. The second image region of the training image is occluded using the mask region of the target image mask to obtain the second mask image, wherein the loss of the second recognition model corresponding to the first mask image is lower than the loss of the second recognition model corresponding to the second mask image. The second recognition model is trained using the second mask image to update its model parameters, resulting in a third recognition model.

2. The method according to claim 1, characterized in that, The method further includes: The first parameter gradient is determined by multiplying the derivative of the preset loss function with respect to the image mask corresponding to the training image and the derivative of the image mask corresponding to the training image with respect to the image mask parameters.

3. The method according to claim 1, characterized in that, Before occluding the first image region of the training image using the mask region of the initial image mask to obtain the first mask image, the method further includes: The image mask parameters are randomly sampled to obtain initial parameter values. Based on the initial parameter values ​​of the image mask parameters, an image mask generation operation is performed using a differentiable activation function to obtain the initial image mask, wherein the differentiable activation function is a monotonically increasing function.

4. The method according to claim 3, characterized in that, The step of randomly sampling the image mask parameters to obtain the initial parameter values ​​of the image mask parameters includes: Randomly sample the region shape parameter in the image mask parameters to obtain an initial shape parameter value, wherein the region shape parameter is used to represent the region shape of the mask region of the image mask corresponding to the training image; The region position parameters in the image mask parameters are randomly sampled to obtain initial position parameter values, wherein the region position parameters are used to represent the region position of the mask region of the image mask corresponding to the training image.

5. The method according to claim 4, characterized in that, The step of randomly sampling the region shape parameters in the image mask parameters to obtain initial shape parameter values ​​includes: Randomly sample the region size parameter in the image mask parameters to obtain an initial size parameter value, wherein the region size parameter is used to represent the region size of the mask region of the image mask corresponding to the training image; The rotation angle parameter in the image mask parameters is randomly sampled to obtain an initial rotation angle value. The rotation angle parameter represents the angle at which the mask region of the image mask corresponding to the training image rotates along a preset direction with the center point of the mask region of the image mask corresponding to the training image as the center.

6. The method according to claim 4, characterized in that, The step of randomly sampling the region location parameters in the image mask parameters to obtain initial location parameter values ​​includes: The center point position parameter in the image mask parameters is randomly sampled to obtain the initial position parameter value, wherein the center point position parameter is used to represent the position of the center point of the mask region of the image mask corresponding to the training image.

7. The method according to claim 3, characterized in that, The mask region indicated by the initial parameter values ​​of the image mask parameters is the initial mask region. The initial parameter values ​​of the image mask parameters include: an initial region length value for the region length of the initial mask region, an initial region width value for the region width of the initial mask region, an initial x-coordinate and an initial y-coordinate for the center point position of the initial mask region, and an initial rotation angle value for representing the rotation angle of the initial mask region. The rotation angle of the initial mask region refers to the angle by which the initial mask region rotates along a preset direction with the center point of the initial mask region as the center point. The initial image mask is obtained by performing an image mask generation operation using a differentiable activation function based on the initial parameter values ​​of the image mask parameters, including: Perform the following operations on each pixel position in the initial image mask as the current pixel position to obtain the initial image mask, wherein the current pixel position includes the current pixel's horizontal coordinate and the current pixel's vertical coordinate: The pixel value at the current pixel position is determined by multiplying the function value corresponding to the first reference value of the differentiable activation function and the function value corresponding to the second reference value of the differentiable activation function. The first reference value is obtained by adding the product of the first coordinate difference and the cosine of the initial rotation angle value, plus the product of the second coordinate difference and the sine of the initial rotation angle value, and subtracting half of the initial region length value. The second reference value is obtained by subtracting the product of the second coordinate difference and the cosine of the initial rotation angle value, minus the product of the first coordinate difference and the sine of the initial rotation angle value, and subtracting half of the initial region width value. Wherein, the first coordinate difference is the coordinate difference between the horizontal coordinate of the current pixel and the horizontal coordinate of the initial center point, and the second coordinate difference is the coordinate difference between the vertical coordinate of the initial center point and the vertical coordinate of the current pixel.

8. The method according to claim 3, characterized in that, The initial image mask is obtained by performing an image mask generation operation using a differentiable activation function based on the initial parameter values ​​of the image mask parameters, including: Based on the initial parameter values ​​of the image mask parameters, the image mask generation operation is performed using the differentiable activation function according to the resolution of the training image to obtain the initial image mask, wherein the resolution of the initial image mask is equal to the resolution of the training image.

9. The method according to claim 3, characterized in that, The initial image mask is obtained by performing an image mask generation operation using a differentiable activation function based on the initial parameter values ​​of the image mask parameters, including one of the following: Based on the initial parameter values ​​of the image mask parameters, the image mask generation operation is performed using the logistic function to obtain the initial image mask; Based on the initial parameter values ​​of the image mask parameters, an image mask generation operation is performed using a preset hyperbolic tangent function to obtain the initial image mask, wherein the preset hyperbolic tangent function is a hyperbolic tangent function with bias and scaling terms.

10. The method according to any one of claims 1 to 9, characterized in that, The step of training the first recognition model using the first mask image to update the model parameters of the first recognition model and obtain the second recognition model includes: The first mask image is input into the first recognition model to obtain the second recognition result output by the first recognition model. Determine a second function value corresponding to a preset loss function and the second recognition result and the preset recognition result, wherein the preset recognition result is a labeled recognition result corresponding to the training image, and the second function value is used to represent the error between the first recognition result and the preset recognition result; Based on the second function value, the model parameters of the first recognition model are updated along the direction of gradient descent of the second parameter to obtain the second recognition model. The input parameters of the preset loss function include the model parameters of the first recognition model, and the gradient of the second parameter is the gradient of the preset loss function corresponding to the model parameters of the first recognition model.

11. A model training device based on a mask image, characterized in that, include: The first processing unit is used to occlude the first image region of the training image using the mask region of the initial image mask to obtain the first mask image. The first training unit is used to train the first recognition model using the first mask image to update the model parameters of the first recognition model and obtain the second recognition model. An adjustment unit is used to input the first mask image into the second recognition model to obtain a first recognition result output by the second recognition model; A first function value is determined corresponding to a preset loss function and the first recognition result, wherein the preset recognition result is a labeled recognition result corresponding to the training image, and the first function value is used to represent the error between the first recognition result and the preset recognition result; based on the first function value, the parameter values ​​of the image mask parameters are updated along the direction of the gradient of the first parameter to obtain the target image mask, wherein the image mask parameters are used to represent the mask region of the image mask corresponding to the training image, the input parameters of the preset loss function include the image mask parameters, and the first parameter gradient is the gradient of the preset loss function corresponding to the image mask parameters; The second processing unit is used to occlude the second image region of the training image using the mask region of the target image mask to obtain the second mask image, wherein the loss of the second recognition model corresponding to the first mask image is lower than the loss of the second recognition model corresponding to the second mask image. The second training unit is used to train the second recognition model using the second mask image to update the model parameters of the second recognition model and obtain the third recognition model.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method of any one of claims 1 to 10.

13. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1 to 10.

14. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method of any one of claims 1 to 10 through the computer program.

Images