Image recognition model training method, image recognition method, device and electronic equipment

Abstract

This disclosure relates to an image recognition model training method, image recognition method, apparatus, and electronic device. The image recognition model training method includes: acquiring an image set; inputting the image set into a preset machine learning model to obtain predicted sample attribute information for each image; training the preset machine learning model using the labeled sample attribute information and predicted sample attribute information corresponding to each image to obtain an initial image recognition model with preliminary recognition capabilities; determining a target image set from the image set based on the labeled sample attribute information and predicted sample attribute information corresponding to each image, thereby deleting certain non-target images that adversely affect the training process; and then training the initial image recognition model using the target image set to obtain a target image recognition model. This allows the trained target image recognition model to better focus on recognizing key images, improving the model's accuracy and recall.

Description

Technical Field

[0001] This disclosure relates to the field of machine learning technology, and in particular to an image recognition model training method, image recognition method, apparatus, and electronic device. Background Technology

[0002] In special scenarios such as security audits and abnormal behavior detection, the natural occurrence rate of a certain type of image (such as pornographic images, violent images, or certain images that are disgusting or offensive) is very low, usually only a few in ten thousand. That is to say, only a few of these types of images will appear out of ten thousand images.

[0003] In related technologies, when identifying such images from a large pool of images, these images (pornographic images, violent images, or other images that are offensive or repulsive) are typically treated the same as other images. However, since the natural occurrence rate of these images is very low, treating them the same as other images during image recognition inevitably leads to confusion with images that do not belong to this category due to certain characteristics, ultimately affecting the accuracy of image recognition. Therefore, a method capable of accurately identifying these images is needed. Summary of the Invention

[0004] This disclosure provides an image recognition model training method, an image recognition method, an apparatus, and an electronic device to at least address the problem of low image recognition accuracy in related technologies. The technical solution of this disclosure is as follows:

[0005] According to a first aspect of the present disclosure, an image recognition model training method is provided, comprising:

[0006] Obtain an image set; each image in the image set carries labeled sample attribute information;

[0007] The image set is input into the preset machine learning model to obtain the predicted sample attribute information for each image;

[0008] The preset machine learning model is trained using the attribute information of the labeled samples and the attribute information of the predicted samples corresponding to each image to obtain an initial image recognition model;

[0009] A target image set is determined from the image set based on the labeled sample attribute information and predicted sample attribute information corresponding to each image; the number of images in the target image set is less than or equal to the number of images in the image set.

[0010] The initial image recognition model is trained using the target image set to obtain the target image recognition model.

[0011] In an optional embodiment, the labeled sample attribute information includes labeled positive sample information and labeled negative sample information;

[0012] The step of training the preset machine learning model using the labeled sample attribute information and predicted sample attribute information corresponding to each image to obtain the initial image recognition model includes:

[0013] The sample loss of each image is determined based on a preset loss function, the labeled sample attribute information, and the predicted sample attribute information corresponding to each image; wherein, the influence of the predicted sample attribute information of the positive sample image on the sample loss of the positive sample image is greater than the influence of the predicted sample attribute information of the negative sample image on the sample loss of the negative sample image; the positive sample image is the image whose labeled sample attribute information is the labeled positive sample information, and the negative sample image is the image whose labeled sample attribute information is the labeled negative sample information;

[0014] The preset machine learning model is trained using the sample loss of each image to obtain an initial image recognition model.

[0015] In an optional embodiment, the expression for the preset loss function is:

[0016] S = -label*(1-p) a *log(p)-(1-label)*X*log(1-p);

[0017] X = (pt) n*a +t a ,p≥t

[0018] X = p a ,p <t

[0019] Where S is the sample loss, label is the labeled sample attribute information, P is the predicted sample attribute information, a is the exponent, t is the discrimination threshold, and n is the relaxation factor.

[0020] In an optional embodiment, the predicted sample attribute information includes predicted positive sample information and predicted negative sample information;

[0021] The step of determining the target image set from the image set based on the labeled sample attribute information and predicted sample attribute information corresponding to each image includes:

[0022] From the image set, images whose labeled sample attribute information is the labeled negative sample information and whose predicted sample attribute information is the predicted positive sample information are determined to obtain a non-target image set;

[0023] The non-target image set is obtained by removing the non-target image set from the image set.

[0024] In an optional embodiment, the step of determining from the image set that the labeled sample attribute information is the labeled negative sample information and the predicted sample attribute information is the predicted positive sample information, to obtain the non-target image set includes:

[0025] The labeled sample attribute information of the images in the image set is updated based on the sample loss of each image and a preset loss threshold to obtain the updated image set;

[0026] From the updated image set, images whose labeled sample attribute information is the labeled negative sample information and whose predicted sample attribute information is the predicted positive sample information are determined, thus obtaining a non-target image set.

[0027] In an optional embodiment, updating the labeled sample attribute information of the images in the image set according to the sample loss of each image and a preset loss threshold to obtain the updated image set includes:

[0028] Images to be updated are identified from the image set whose sample loss is greater than or equal to the preset loss threshold.

[0029] Update the attribute information of the labeled samples of the image to be updated to obtain the updated image;

[0030] The updated image set is obtained by replacing the image to be updated in the image set with the updated image to be updated.

[0031] According to a second aspect of the present disclosure, an image recognition method is provided, comprising:

[0032] Acquire the image to be recognized;

[0033] The image to be identified is input into the target image recognition model trained by the above image recognition model training method, and the image to be identified is used to perform image recognition and determine the predicted sample attribute information.

[0034] According to a third aspect of the present disclosure, an image recognition model training apparatus is provided, comprising:

[0035] The image acquisition module is configured to acquire an image set; each image in the image set carries labeled sample attribute information;

[0036] The prediction sample attribute information determination module is configured to input the image set into the preset machine learning model to obtain the prediction sample attribute information of each image;

[0037] The first training module is configured to train the preset machine learning model using the attribute information of the labeled samples and the attribute information of the predicted samples corresponding to each image, so as to obtain an initial image recognition model.

[0038] The target image set determination module is configured to determine a target image set from the image set based on the labeled sample attribute information and predicted sample attribute information corresponding to each image; the number of images in the target image set is less than or equal to the number of images in the image set.

[0039] The second training module is configured to train the initial image recognition model using the target image set to obtain the target image recognition model.

[0040] In an optional embodiment, the labeled sample attribute information includes labeled positive sample information and labeled negative sample information;

[0041] The first training module includes:

[0042] The sample loss determination submodule is configured to determine the sample loss of each image based on a preset loss function, the labeled sample attribute information, and the predicted sample attribute information corresponding to each image; wherein, the predicted sample attribute information of a positive sample image has a greater influence on the sample loss of the positive sample image than the predicted sample attribute information of a negative sample image; the positive sample image is the image whose labeled sample attribute information is the labeled positive sample information, and the negative sample image is the image whose labeled sample attribute information is the labeled negative sample information;

[0043] The first training submodule is configured to train the preset machine learning model using the sample loss of each image to obtain an initial image recognition model.

[0044] In an optional embodiment, the expression for the preset loss function is:

[0045] S = -label*(1-p) a *log(p)-(1-label)*X*log(1-p);

[0046] X = (pt) n*a +t a ,p≥t

[0047] X = p a ,p <t

[0048] Where S is the sample loss, label is the labeled sample attribute information, P is the predicted sample attribute information, a is the exponent, t is the discrimination threshold, and n is the relaxation factor.

[0049] In an optional embodiment, the predicted sample attribute information includes predicted positive sample information and predicted negative sample information;

[0050] The target image set determination module includes:

[0051] The non-target image set determination module is configured to determine images from the image set whose labeled sample attribute information is the labeled negative sample information and whose predicted sample attribute information is the predicted positive sample information, thereby obtaining a non-target image set;

[0052] The target image set determination submodule is configured to remove the non-target image set from the image set to obtain the target image set.

[0053] In an optional embodiment, the non-target image set determination module includes:

[0054] The image set update module is configured to update the labeled sample attribute information of the images in the image set according to the sample loss of each image and a preset loss threshold, so as to obtain the updated image set;

[0055] The non-target image set determination submodule is configured to execute the determination of images from the updated image set whose labeled sample attribute information is the labeled negative sample information and whose predicted sample attribute information is the predicted positive sample information, thereby obtaining a non-target image set.

[0056] In an optional embodiment, the image set update module includes:

[0057] The image to be updated determination module is configured to determine the images to be updated from the image set whose sample loss is greater than or equal to the preset loss threshold;

[0058] The image update module is configured to update the attribute information of the labeled samples of the image to be updated, so as to obtain the updated image to be updated;

[0059] The image set update submodule is configured to replace the image to be updated in the image set with the updated image to obtain the updated image set.

[0060] According to a fourth aspect of the present disclosure, an image recognition apparatus is provided, comprising:

[0061] The image acquisition module is configured to acquire the image to be recognized.

[0062] The recognition module is configured to perform image recognition on the image to be recognized by inputting the image to be recognized into the target image recognition model trained by the above image recognition model training method, and determine the predicted sample attribute information.

[0063] According to a fifth aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the image recognition model training method or the image recognition method as described above.

[0064] According to a sixth aspect of the present disclosure, a storage medium is provided that, when instructions in the storage medium are executed by a server's processor, enables the server to perform the image recognition model training method or the image recognition method as described above.

[0065] According to a seventh aspect of the present disclosure, a computer program product is provided, the computer program product including a computer program stored in a readable storage medium, wherein at least one processor of a computer device reads from the readable storage medium and executes the computer program, causing the device to perform the image recognition model training method or the image recognition method described above.

[0066] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:

[0067] An image set is acquired, where each image carries labeled sample attribute information. This image set is input into a pre-defined machine learning model to obtain predicted sample attribute information for each image. The pre-defined machine learning model is trained using the labeled and predicted sample attribute information for each image, resulting in an initial image recognition model with preliminary recognition capabilities. Subsequently, a target image set is determined from the image set based on the labeled and predicted sample attribute information for each image. The number of images in the target image set is less than or equal to the number of images in the image set, thus removing some non-target images that adversely affect the training process. The initial image recognition model is then trained using the target image set to obtain a target image recognition model. This allows the trained target image recognition model to focus more on recognizing key images, improving the model's accuracy and recall.

[0068] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0069] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0070] Figure 1 This is a schematic diagram of an implementation environment according to an exemplary embodiment;

[0071] Figure 2 This is a flowchart illustrating an image recognition model training method according to an exemplary embodiment;

[0072] Figure 3 This is a flowchart illustrating a method for determining prediction sample attribute information according to an exemplary embodiment;

[0073] Figure 4 This is a flowchart illustrating the determination of an initial image recognition model according to an exemplary embodiment;

[0074] Figure 5 This is a flowchart illustrating an image recognition method according to an exemplary embodiment;

[0075] Figure 6 This is a block diagram of an image recognition model training device according to an exemplary embodiment;

[0076] Figure 7 This is a block diagram of an image recognition device according to an exemplary embodiment;

[0077] Figure 8 This is a block diagram illustrating an electronic device for image recognition model training or for image recognition, according to an exemplary embodiment. Detailed Implementation

[0078] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0079] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0080] Please see Figure 1 , Figure 1 This is a schematic diagram illustrating an application environment according to an exemplary embodiment, such as... Figure 1 As shown, the application environment may include server 01 and terminal 02.

[0081] In an optional embodiment, server 01 can be used to train a target image recognition model capable of image recognition. Specifically, server 01 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0082] In an optional embodiment, terminal 02 can perform image recognition processing by combining the image recognition model trained by server 01. Specifically, terminal 02 can be, but is not limited to, electronic devices such as smartphones, desktop computers, tablets, laptops, smart speakers, digital assistants, augmented reality (AR) / virtual reality (VR) devices, and smart wearable devices. Optionally, the operating system running on the electronic device can be, but is not limited to, Android, iOS, Linux, Windows, and Unix systems.

[0083] In addition, it should be noted that, Figure 1 The example shown is merely one application environment provided by this disclosure. In practical applications, other application environments may also be included, such as the training of a target image recognition model, which can also be implemented on terminal 02.

[0084] In the embodiments described in this specification, the server 01 and the terminal 02 can be directly or indirectly connected through wired or wireless communication, and this disclosure does not impose any restrictions.

[0085] Figure 2 This is a flowchart illustrating an image recognition model training method according to an exemplary embodiment, such as... Figure 2 As shown, the image recognition model training method can be applied to electronic devices such as servers, terminals, and edge computing nodes, and includes the following steps.

[0086] In step S201, an image set is obtained; each image in the image set carries labeled sample attribute information.

[0087] In this embodiment of the disclosure, images in an image set can be acquired by an electronic device, or the electronic device can acquire images from other devices, such as camera devices, monitoring devices, etc. In some implementations, the aforementioned image may be a frame from a video. Similarly, the image set may be acquired by the electronic device, or it may be acquired by other devices and integrated into the electronic device.

[0088] In this embodiment of the disclosure, each image in the aforementioned image set may carry labeled sample attribute information, wherein the labeled sample attribute information may include labeled positive sample information or labeled negative sample information. That is, an image can only be labeled with one type of labeled sample attribute information, either labeled positive sample information or labeled negative sample information.

[0089] Optionally, the attribute information of the labeled samples carried on each image can be manually labeled by technicians. The main purpose of using manual labeling is to improve the accuracy of image labeling as much as possible, so that the image can avoid adverse consequences such as the trained model not meeting the requirements due to labeling errors during the later training process.

[0090] As mentioned above, the final target image recognition model aims to identify images with low natural occurrence probability, such as pornographic, violent, or offensive images, ensuring good accuracy and recall. Therefore, the image set can include images with low natural occurrence probability (e.g., pornographic, violent, or offensive images) and other images. We can refer to the images with low natural occurrence probability as positive sample images and the other images as negative sample images. Positive sample images are those labeled with positive attribute information, and negative sample images are those labeled with negative attribute information. Optionally, positive sample information can be represented by "label=1", and negative sample information by "label=0".

[0091] Specifically, assuming the target image recognition model is designed for pornographic images, the image set can include pornographic images and other images. Positive sample images can be pornographic images, while other images are negative sample images. In this case, the labeled sample attribute information carried on pornographic images is "label=1", while the labeled sample attribute information carried on other images is "label=0".

[0092] In step S203, the image set is input into a preset machine learning model to obtain the predicted sample attribute information for each image.

[0093] In this embodiment of the disclosure, the aforementioned preset machine learning model can be built based on machine learning and deep learning, which typically include techniques such as artificial neural networks, transfer learning, and inductive learning. Machine learning can include supervised machine learning, unsupervised machine learning, and semi-supervised machine learning. The aforementioned preset machine learning model can include, but is not limited to, machine learning models employing convolutional neural networks, residual neural networks, recurrent neural networks, and recurrent neural networks. Specifically, it can be a deep learning image recognition model such as ResNet50, Inception-v3, or Efficient-b3.

[0094] Figure 3 This is a flowchart illustrating a method for determining prediction sample attribute information according to an exemplary embodiment, such as... Figure 3 As shown, it includes:

[0095] In step S2031, a preset machine learning model is constructed, and the learning rate, optimizer type, and training cutoff condition of the preset machine learning model are determined.

[0096] The learning rate mentioned above is a crucial hyperparameter in supervised learning and deep learning, determining whether and when the objective function converges to a local minimum. A suitable learning rate enables the objective function to converge to a local minimum within a reasonable timeframe. When the learning rate is set too low, the convergence process becomes very slow. Conversely, when the learning rate is set too high, the gradient may oscillate around the minimum, or even fail to converge. With a fixed learning rate, when convergence is reached, the model will oscillate within a large region around the optimal value; conversely, decreasing the learning rate with increasing iterations will cause the model to oscillate within a smaller region around the optimal value during convergence. Therefore, choosing a suitable learning rate is crucial for model training. In this embodiment, the learning rate of the preset machine learning model can be a fixed value or a value that varies with the number of iterations. Optionally, the learning rate can be a fixed value of 0.001, which can be an experimental value.

[0097] The optimizer mentioned above is used to update and compute network parameters that affect model training and model output, making them approach or reach the optimal value, thereby minimizing (or maximizing) the loss function. The optimizer type can be the Radam optimizer, the Adam optimizer, or other suitable optimizers.

[0098] In this embodiment of the disclosure, the training cutoff condition mentioned above can be that the sample loss of the image meets a certain condition, or that the number of iterations meets a certain condition. When the training cutoff condition is met, the training of the current model can be stopped.

[0099] In step S2033, based on the pre-built machine learning model, the prediction of the predicted sample attribute information is performed on each image in the image set to obtain the predicted sample attribute information of each image.

[0100] In this embodiment of the disclosure, step S2033 is used to predict the probability that the input image is a positive sample image by using a preset machine learning model. If the positive sample image is a pornographic image, then the preset probability is that each image is a pornographic image.

[0101] In an optional embodiment, the predicted sample attribute information described above is represented using values ​​between 0 and 1, that is, the probability is between 0 and 1.

[0102] In step S205, the preset machine learning model is trained using the attribute information of the labeled samples and the attribute information of the predicted samples corresponding to each image to obtain the initial image recognition model.

[0103] In this embodiment of the disclosure, when constructing a preset machine learning model, in addition to setting the learning rate, optimizer type, and training cutoff condition mentioned above, it is also necessary to set a loss function so that it converges to a local minimum within a suitable time. Therefore, Figure 4 This is a flowchart illustrating the determination of an initial image recognition model according to an exemplary embodiment, such as... Figure 4 As shown, it includes:

[0104] In step S2051, the sample loss of each image is determined based on the preset loss function, the labeled sample attribute information and the predicted sample attribute information corresponding to each image.

[0105] In this embodiment of the disclosure, as mentioned above, the preset sample attribute information can be represented by probability values ​​between 0 and 1. Assuming that the predicted sample attribute information is distinguished using a probability value of 0.5 as the median, the predicted sample attribute information with probability values ​​between 0.5 and 1 can be identified as predicted positive sample information, while the predicted sample attribute information with probability values ​​between 0 and 0.5 can be considered as predicted negative sample information. Optionally, the aforementioned median value of 0.5 is an optional embodiment and can be varied according to actual circumstances.

[0106] Therefore, an image, based on its corresponding labeled sample attribute information and predicted sample attribute information, can generally fall into the following categories, including but not limited to:

[0107] The first type: The labeled sample attribute information of the image is the labeled positive sample information "label=1", and the predicted sample attribute information is the predicted positive sample information, such as 0.8. Taking a positive sample image as an image involving pornography as an example, the image in the first case is actually an image involving pornography, and the prediction is also biased towards an image involving pornography. Therefore, the image in this case can be considered a simple sample among the positive sample images.

[0108] The second type: The labeled sample attribute information of the image is the positive sample information "label=1", and the predicted sample attribute information is the predicted negative sample information, such as 0.2. Taking the positive sample image as an image involving pornography as an example, the image in the second case is actually an image involving pornography, but the prediction is biased towards other images, that is, non-pornographic images. Therefore, the image in this case can be considered a hard sample among the positive sample images, that is, an image involving pornography that is easily confused with other images.

[0109] The third type: The labeled sample attribute information of the image is the negative sample information "label=0", and the predicted sample attribute information is the predicted positive sample information, such as 0.8. Taking the positive sample image as an example of a pornographic image, the image in the third case is not actually a pornographic image, but it is predicted to be a pornographic image. Therefore, the image in this case can be considered a hard sample among the negative sample images, that is, other images that are easily confused with pornographic images. For example, this type of image might be an image of a shirtless male model.

[0110] The fourth type: The labeled sample attribute information of the image is the labeled negative sample information "label=0", and the predicted sample attribute information is the predicted negative sample information, such as 0.2. Taking the positive sample image as an example of a pornographic image, the image in the fourth case is not actually a pornographic image, and the prediction is also biased towards other images, that is, non-pornographic images. Therefore, the image in this case can be considered a simple sample among the negative sample images.

[0111] To ensure that the final target image recognition model can identify images with low natural probability, such as pornographic images, violent images, or images that cause disgust or aversion, and to make the trained target image recognition model perform well in terms of recognition accuracy and recall, since difficult samples are much more difficult to identify than simple samples, the preset loss function can be expressed as follows: the sample loss on difficult samples is greater than the sample loss on simple samples. The formula for the preset loss function is shown in the following formula (1):

[0112] S = -label*(1-p) a *log(p)-(1-label)*p a*log(1-p)……Formula (1)

[0113] Where S is the sample loss, label is the labeled sample attribute information, P is the predicted sample attribute information, and a is the exponent.

[0114] The following two examples will demonstrate that regardless of whether the labeled sample attribute information is positive or negative, formula (1) can satisfy the condition that the sample loss on difficult samples is greater than the sample loss on simple samples.

[0115] Example 1: Assume that the labeled sample attribute information of images A and B is positive sample information, the predicted sample attribute information of image A is 0.2, and the predicted sample attribute information of image B is 0.9. Image A is a difficult sample compared to image B. Since images A and B are positive sample images "label=1", the above formula (1) can be simplified to the following formula (1.1):

[0116] S = -(1-p) a *log(p)……Formula (1.1)

[0117] Assuming the exponent a is 2, substituting 0.2 and 0.9 into formula (1.1) above, we can obtain that the sample loss of image A is greater than that of image B, i.e., S(A) = -0.8. a *log(0.2)>S(B)=-0.1 a *log(0.9).

[0118] Example 2: Assume that the labeled sample attribute information of images C and D is the labeled negative sample information, the predicted sample attribute information of image C is 0.7, and the predicted sample attribute information of image D is 0.1. Image C is a hard sample compared to image D. Since images C and D are negative sample images with "label = 0", the above formula (1) can be simplified to the following formula (1.2):

[0119] S = -p a *log(1-p)……Formula (1.2)

[0120] Similarly, assuming a is 2, substituting 0.7 and 0.1 into formula (1.2) above, we can obtain that the sample loss of image C is greater than that of image D, i.e., S(C) = -0.7. a *log(0.3)>S(D)=-0.1 a *log(0.9).

[0121] In an optional embodiment, taking pornographic images as an example, the final target image recognition model identifies pornographic images from a set of images, especially those belonging to difficult samples. Difficult samples in non-pornographic images are not of great concern. Therefore, the preset loss function can be expressed as follows: while the sample loss on difficult samples is greater than the sample loss on simple samples, the influence of the predicted sample attribute information of positive sample images on the sample loss of positive sample images is greater than the influence of the predicted sample attribute information of negative sample images on the sample loss of negative sample images; positive sample images are those whose labeled sample attribute information is labeled with positive sample information, and negative sample images are those whose labeled sample attribute information is labeled with negative sample information. The above preset loss function can be expressed as follows: (2)

[0122] S = -label*(1-p) a *log(p)-(1-label)*X*log(1-p);

[0123] X = (pt) n*a +t a ,p≥t...Formula (2)

[0124] X = p a ,p <t

[0125] Where S is the sample loss, label is the labeled sample attribute information, P is the predicted sample attribute information, a is the exponent, t is the discrimination threshold, and n is the relaxation factor.

[0126] The following two examples will further demonstrate that regardless of whether the labeled sample attribute information is positive or negative, formula (2) can satisfy the condition that the sample loss on difficult samples is greater than that on easy samples, and that the influence of the predicted sample attribute information of the positive sample image on the sample loss of the positive sample image is greater than that of the predicted sample attribute information of the negative sample image on the sample loss of the negative sample image. Based on formula (1), formula (2) shows that the sample loss of difficult samples in the positive sample image remains unchanged, but the sample loss of difficult samples in the negative sample image is appropriately weakened.

[0127] Example 1: Assume that the labeled sample attribute information of images A and B is positive sample information, the predicted sample attribute information of image A is 0.2, and the predicted sample attribute information of image B is 0.9. Image A is a difficult sample compared to image B. Since images A and B are positive sample images "label=1", the above formula (1) can be simplified to the following formula (1.1):

[0128] S = -(1-p) a*log(p)……Formula (1.1)

[0129] Assuming the exponent a is 2, substituting 0.2 and 0.9 into formula (1.1) above, we can obtain that the sample loss of image A is greater than that of image B, i.e., S(A) = -0.8. a *log(0.2)>S(B)=-0.1 a *log(0.9). Obviously, compared to formula (1), the sample loss of the hard samples in the positive sample image corresponding to formula (2) remains unchanged.

[0130] Example 2: Assume that the labeled sample attribute information of images C and D is the labeled negative sample information, the predicted sample attribute information of image C is 0.7, and the predicted sample attribute information of image D is 0.1. Image C is a hard sample compared to image D. Since images C and D are negative sample images with "label = 0", the above formula (1) can be simplified to the following formula (1.2):

[0131] S=-X*log(1-p)……Formula (1.2)

[0132] Assuming a is 2 and the discrimination threshold t is 0.6, relaxation is applied because n is 2. Substituting 0.7 and 0.1 into formula (1.2) above, we can see that the sample loss of image C is greater than that of image D.

[0133] That is, S(C) = -(0.1) 4 +0.6 2 )*log(0.3)>S(D)=-0.1 a *log(0.9). Obviously, compared with formula (1), the sample loss of the hard sample in the negative sample image corresponding to formula (2) (corresponding to 0.7) is better than the sample loss of the hard sample in the negative sample image corresponding to formula (1) (corresponding to 0.7).

[0134] In summary, since Formula (2) maintains the same sample loss for difficult samples in positive sample images as Formula (1), but appropriately weakens the sample loss for difficult samples in negative sample images, it can be considered that the influence of the predicted sample attribute information of positive sample images on the sample loss of positive sample images is greater than the influence of the predicted sample attribute information of negative sample images on the sample loss of negative sample images.

[0135] In step S2053, the preset machine learning model is trained using the sample loss of each image to obtain the initial image recognition model.

[0136] In an optional embodiment, if the training cutoff condition is that the number of iterations meets a certain condition, such as 80 iterations, then the preset machine learning model is trained using the sample loss of each image. When the number of iterations reaches 80, the initial image recognition model is obtained. Optionally, the above-mentioned number of iterations of 80 is an example and can be set according to the actual situation.

[0137] In another optional embodiment, the training cutoff condition is that the sample loss of the image meets a certain condition, such as the sample loss being less than a preset threshold. When the sample loss is greater than the preset threshold, backpropagation is performed based on the sample loss to update the parameters of the preset machine learning model, resulting in an updated preset machine learning model. The above training steps are repeated until the sample loss of the image is less than the preset threshold, and the trained preset machine learning model is determined as the initial image recognition model.

[0138] Thus, in order to ensure that the final target image recognition model can identify some images with low natural probability, the initial image recognition model obtained in the first stage can have a greater sample loss on difficult samples than on simple samples, while keeping the sample loss of difficult samples in positive sample images unchanged, but appropriately weakening the sample loss of difficult samples in negative sample images. This lays the foundation for the final trained target image recognition model to have better performance in recognition accuracy and recall.

[0139] In step S207, a target image set is determined from the image set based on the labeled sample attribute information and predicted sample attribute information corresponding to each image; the number of images in the target image set is less than or equal to the number of images in the image set.

[0140] Since the purpose of this embodiment is to enable the final target image recognition model to better identify positive sample images, that is, to improve the recall rate of positive sample images, it is necessary to reduce the interference of difficult samples in negative sample images. Optionally, negative sample images that are easily confused with positive sample images can be deleted. In other words, images that are predicted as positive sample images but are actually negative sample images can be deleted from the image set.

[0141] In one optional embodiment, a non-target image set can be obtained by identifying images from the image set whose labeled sample attribute information is negative and whose predicted sample attribute information is positive. The non-target image set is then removed from the image set to obtain the target image set. For example, images predicted as pornographic (positive sample images) but actually depicting shirtless male models would be deleted. Since the example above uses a probability value of 0.5 as the median to distinguish between predicted positive and negative sample information, images with labeled sample attribute information of "label=0" but predicted sample attribute information between 0.5 and 1 can also be deleted.

[0142] In another alternative embodiment, a deletion threshold can be directly set, for example, 0.8. Based on this, images with labeled sample attribute information of "label=0" but predicted sample attribute information above 0.8 can be identified as non-target images and deleted. This yields the final target image set. In this way, while minimizing the interference of difficult samples in negative sample images on model training, the number of images used for training can be guaranteed.

[0143] In an optional embodiment, since the labeled sample attribute information carried by each image in the image set can be manually labeled as mentioned above, however, since a large number of images need to be labeled manually, it is still possible that the labeled sample attribute information of some images is incorrect during the process. In this case, the labeled sample attribute information of the images in the image set can be updated according to the sample loss of each image and a preset loss threshold to obtain an updated image set.

[0144] In an optional embodiment, the updated image set obtained above may include the following: identifying images to be updated from the image set whose sample loss is greater than or equal to a preset loss threshold; updating the labeled sample attribute information of the images to be updated to obtain updated images to be updated; and replacing the images to be updated in the image set with the updated images to be updated to obtain the updated image set. Optionally, the preset loss threshold can be set to a relatively large value. When the sample loss is greater than or equal to the preset loss threshold, it can be determined that the gap between the predicted information and the actual labeled information is large, which is very likely due to errors in manual labeling. Therefore, such images can be identified, their labeled sample attribute information updated to obtain updated images to be updated, and the updated images to be updated can be replaced with the updated images to be updated in the image set to obtain the updated image set.

[0145] Based on the above embodiments, images whose labeled sample attribute information is labeled as negative sample information and whose predicted sample attribute information is predicted as positive sample information can be identified from the updated image set, resulting in a non-target image set. The non-target image set is then removed from the image set to obtain the target image set. The process for obtaining the non-target image set can be found above and will not be repeated here.

[0146] In step S209, the initial image recognition model is trained using the target image set to obtain the target image recognition model.

[0147] In this embodiment of the disclosure, the training cutoff condition for the target image recognition model may include the number of iterations meeting a certain condition or the sample loss of the image meeting a certain condition.

[0148] In an optional embodiment, if the training cutoff condition is that the number of iterations meets a certain condition, such as 50 iterations, then the preset machine learning model is trained using the sample loss of each image. When the number of iterations reaches 20, the initial image recognition model is obtained. Optionally, the above-mentioned number of iterations of 50 is an example and can be set according to the actual situation.

[0149] In another optional embodiment, the training cutoff condition is that the sample loss of the image meets a certain condition, such as the sample loss being less than a preset threshold. When the sample loss is greater than the preset threshold, backpropagation is performed based on the sample loss to update the parameters of the initial image recognition model, resulting in an updated initial image recognition model. The above training steps are repeated until the sample loss of the image is less than the preset threshold, and the trained initial image recognition model is determined as the target image recognition model.

[0150] Based on the above-described embodiments of the image recognition model training method, the following describes an embodiment of an image recognition method disclosed herein. Figure 5 This is a flowchart illustrating an image recognition method according to an exemplary embodiment, with reference to... Figure 5 This method can be applied to electronic devices such as servers, terminals, and edge computing nodes, and includes the following steps.

[0151] In step S501, the image to be identified is acquired;

[0152] In step S503, the image to be recognized is input into the target image recognition model trained by the above image recognition model training method, and image recognition is performed on the image to be recognized to determine the predicted sample attribute information.

[0153] Optionally, if the predicted sample attribute information is greater than a certain set value, the image to be identified can be determined as the target image (such as an image involving pornography). Otherwise, it is ignored.

[0154] In the above embodiments, the preset loss function can make the sample loss on difficult samples greater than that on simple samples, while appropriately weakening the sample loss of difficult samples in negative sample images. In addition, by correcting mislabeled images and deleting negative sample images that are easily confused with positive sample images, the recognition accuracy and recall ability of the trained target image recognition model are comprehensively guaranteed, so that the image to be recognized can be identified more easily and the work efficiency can be improved.

[0155] Figure 6 This is a block diagram illustrating an image recognition model training apparatus according to an exemplary embodiment. (Refer to...) Figure 6 The device includes an image acquisition module 601, a predicted sample attribute information determination module 602, a first training module 603, a target image set determination module 604, and a second training module 605, wherein...

[0156] Image acquisition module 601 is configured to acquire an image set; each image in the image set carries labeled sample attribute information;

[0157] The prediction sample attribute information determination module 602 is configured to input the image set into the preset machine learning model to obtain the prediction sample attribute information of each image;

[0158] The first training module 603 is configured to train the preset machine learning model using the attribute information of the labeled samples and the attribute information of the predicted samples corresponding to each image, so as to obtain an initial image recognition model.

[0159] The target image set determination module 604 is configured to determine a target image set from the image set based on the labeled sample attribute information and predicted sample attribute information corresponding to each image; the number of images in the target image set is less than or equal to the number of images in the image set.

[0160] The second training module 605 is configured to train the initial image recognition model using the target image set to obtain the target image recognition model.

[0161] In an optional embodiment, the labeled sample attribute information includes labeled positive sample information and labeled negative sample information;

[0162] The first training module includes:

[0163] The sample loss determination submodule is configured to determine the sample loss of each image based on a preset loss function, the labeled sample attribute information, and the predicted sample attribute information corresponding to each image; wherein, the predicted sample attribute information of a positive sample image has a greater influence on the sample loss of the positive sample image than the predicted sample attribute information of a negative sample image; the positive sample image is the image whose labeled sample attribute information is the labeled positive sample information, and the negative sample image is the image whose labeled sample attribute information is the labeled negative sample information;

[0164] The first training submodule is configured to train the preset machine learning model using the sample loss of each image to obtain an initial image recognition model.

[0165] In an optional embodiment, the expression for the preset loss function is:

[0166] S = -label*(1-p) a *log(p)-(1-label)*X*log(1-p);

[0167] X = (pt) n*a +t a ,p≥t

[0168] X = p a ,p <t

[0169] Where S is the sample loss, label is the labeled sample attribute information, P is the predicted sample attribute information, a is the exponent, t is the discrimination threshold, and n is the relaxation factor.

[0170] In an optional embodiment, the predicted sample attribute information includes predicted positive sample information and predicted negative sample information;

[0171] The target image set determination module includes:

[0172] The non-target image set determination module is configured to determine images from the image set whose labeled sample attribute information is the labeled negative sample information and whose predicted sample attribute information is the predicted positive sample information, thereby obtaining a non-target image set;

[0173] The target image set determination submodule is configured to remove the non-target image set from the image set to obtain the target image set.

[0174] In an optional embodiment, the non-target image set determination module includes:

[0175] The image set update module is configured to update the labeled sample attribute information of the images in the image set according to the sample loss of each image and a preset loss threshold, so as to obtain the updated image set;

[0176] The non-target image set determination submodule is configured to execute the determination of images from the updated image set whose labeled sample attribute information is the labeled negative sample information and whose predicted sample attribute information is the predicted positive sample information, thereby obtaining a non-target image set.

[0177] In an optional embodiment, the image set update module includes:

[0178] The image to be updated determination module is configured to determine the images to be updated from the image set whose sample loss is greater than or equal to the preset loss threshold;

[0179] The image update module is configured to update the attribute information of the labeled samples of the image to be updated, so as to obtain the updated image to be updated;

[0180] The image set update submodule is configured to replace the image to be updated in the image set with the updated image to obtain the updated image set.

[0181] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0182] Figure 7 This is a block diagram illustrating an image recognition device according to an exemplary embodiment. (Refer to...) Figure 7 The device includes an image acquisition module 701 and an identification module 702, wherein,

[0183] The image acquisition module 701 is configured to acquire the image to be recognized.

[0184] The recognition module 702 is configured to perform image recognition on the image to be recognized by inputting the image to be recognized into the target image recognition model trained by the above image recognition model training method, and determine the predicted sample attribute information.

[0185] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0186] Figure 8 This is a block diagram illustrating an electronic device for image recognition model training or for image recognition, according to an exemplary embodiment. The electronic device may be a terminal, and its internal structure diagram may be as follows: Figure 8As shown, the electronic device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When executed by the processor, the computer program implements a method for determining or processing image defects by filling in a network. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the device's casing, or an external keyboard, touchpad, or mouse.

[0187] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present disclosure and does not constitute a limitation on the electronic device to which the present disclosure is applied. A specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0188] In an exemplary embodiment, an electronic device is also provided, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement an image recognition model training or an image recognition method as described in the embodiments of this disclosure.

[0189] In an exemplary embodiment, a storage medium is also provided, wherein when the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform image recognition model training or image recognition method as described in this disclosure.

[0190] In an exemplary embodiment, a computer program product including instructions is also provided. The computer program product includes a computer program stored in a readable storage medium. At least one processor of a computer device reads from the readable storage medium and executes the computer program, causing the device to perform the image recognition model training method or image recognition method described above.

[0191] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this disclosure can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0192] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0193] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for training an image recognition model, characterized in that, include: Get the image set; Each image in the image set carries labeled sample attribute information; The image set is input into a preset machine learning model to obtain the predicted sample attribute information for each image; The preset machine learning model is trained using the attribute information of the labeled samples and the attribute information of the predicted samples corresponding to each image to obtain an initial image recognition model; A target image set is determined from the image set based on the labeled sample attribute information and predicted sample attribute information corresponding to each image; the number of images in the target image set is less than or equal to the number of images in the image set. The initial image recognition model is trained using the target image set to obtain the target image recognition model; The labeled sample attribute information includes labeled positive sample information and labeled negative sample information; the predicted sample attribute information includes predicted positive sample information and predicted negative sample information. The step of determining the target image set from the image set based on the labeled sample attribute information and predicted sample attribute information corresponding to each image includes: From the image set, images whose labeled sample attribute information is the labeled negative sample information and whose predicted sample attribute information is the predicted positive sample information are determined to obtain a non-target image set; The non-target image set is obtained by removing the non-target image set from the image set.

2. The image recognition model training method according to claim 1, characterized in that, The step of training the preset machine learning model using the labeled sample attribute information and predicted sample attribute information corresponding to each image to obtain the initial image recognition model includes: The sample loss of each image is determined based on a preset loss function, the labeled sample attribute information, and the predicted sample attribute information corresponding to each image; wherein, the influence of the predicted sample attribute information of the positive sample image on the sample loss of the positive sample image is greater than the influence of the predicted sample attribute information of the negative sample image on the sample loss of the negative sample image; the positive sample image is the image whose labeled sample attribute information is the labeled positive sample information, and the negative sample image is the image whose labeled sample attribute information is the labeled negative sample information; The preset machine learning model is trained using the sample loss of each image to obtain an initial image recognition model.

3. The image recognition model training method of claim 2, wherein, The expression for the preset loss function is: Where S is the sample loss, label is the labeled sample attribute information, P is the predicted sample attribute information, a is the exponent, t is the discrimination threshold, and n is the relaxation factor.

4. The image recognition model training method according to claim 1, characterized in that, The step of determining from the image set that the labeled sample attribute information is the labeled negative sample information and the predicted sample attribute information is the predicted positive sample information, to obtain the non-target image set includes: The labeled sample attribute information of the images in the image set is updated based on the sample loss of each image and a preset loss threshold to obtain the updated image set; From the updated image set, images whose labeled sample attribute information is the labeled negative sample information and whose predicted sample attribute information is the predicted positive sample information are determined to obtain a non-target image set.

5. The image recognition model training method according to claim 4, characterized in that, The step of updating the labeled sample attribute information of the images in the image set according to the sample loss of each image and a preset loss threshold to obtain the updated image set includes: Images to be updated are identified from the image set whose sample loss is greater than or equal to the preset loss threshold. Update the attribute information of the labeled samples of the image to be updated to obtain the updated image; The updated image set is obtained by replacing the image to be updated in the image set with the updated image to be updated.

6. An image recognition method, characterized in that, include Acquire the image to be recognized; The image to be identified is input into the target image recognition model trained according to any one of the image recognition model training methods according to claims 1 to 5, and the image to be identified is used to perform image recognition to determine the predicted sample attribute information.

7. An image recognition model training device, characterized in that, include: The image acquisition module is configured to acquire an image set; each image in the image set carries labeled sample attribute information; The prediction sample attribute information determination module is configured to input the image set into a preset machine learning model to obtain the prediction sample attribute information of each image; The first training module is configured to train the preset machine learning model using the attribute information of the labeled samples and the attribute information of the predicted samples corresponding to each image, so as to obtain an initial image recognition model. The target image set determination module is configured to determine a target image set from the image set based on the labeled sample attribute information and predicted sample attribute information corresponding to each image; the number of images in the target image set is less than or equal to the number of images in the image set. The second training module is configured to train the initial image recognition model using the target image set to obtain the target image recognition model; The labeled sample attribute information includes labeled positive sample information and labeled negative sample information; the predicted sample attribute information includes predicted positive sample information and predicted negative sample information. The target image set determination module is configured to execute: From the image set, images whose labeled sample attribute information is the labeled negative sample information and whose predicted sample attribute information is the predicted positive sample information are determined to obtain a non-target image set; The non-target image set is obtained by removing the non-target image set from the image set.

8. The image recognition model training device according to claim 7, characterized in that, The first training module includes: The sample loss determination submodule is configured to determine the sample loss of each image based on a preset loss function, the labeled sample attribute information, and the predicted sample attribute information corresponding to each image; wherein, the predicted sample attribute information of the positive sample image has a greater influence on the sample loss of the positive sample image than the predicted sample attribute information of the negative sample image; the positive sample image is the image whose labeled sample attribute information is the labeled positive sample information, and the negative sample image is the image whose labeled sample attribute information is the labeled negative sample information; The first training submodule is configured to train the preset machine learning model using the sample loss of each image to obtain an initial image recognition model.

9. The image recognition model training device according to claim 8, characterized in that, The expression for the preset loss function is: Where S is the sample loss, label is the labeled sample attribute information, P is the predicted sample attribute information, a is the exponent, t is the discrimination threshold, and n is the relaxation factor.

10. The image recognition model training device according to claim 7, characterized in that, The non-target image set determination module includes: The image set update module is configured to update the labeled sample attribute information of the images in the image set according to the sample loss of each image and a preset loss threshold, so as to obtain the updated image set; The non-target image set determination submodule is configured to execute the determination of images from the updated image set whose labeled sample attribute information is the labeled negative sample information and whose predicted sample attribute information is the predicted positive sample information, thereby obtaining a non-target image set.

11. The image recognition model training device according to claim 10, characterized in that, The image set update module includes: The image to be updated determination module is configured to determine the images to be updated from the image set whose sample loss is greater than or equal to the preset loss threshold; The image update module is configured to update the attribute information of the labeled samples of the image to be updated, so as to obtain the updated image to be updated; The image set update submodule is configured to replace the image to be updated in the image set with the updated image to obtain the updated image set.

12. An image recognition device, characterized in that, include The image acquisition module is configured to acquire the image to be recognized. The recognition module is configured to perform image recognition on the image to be recognized by inputting the image to be recognized into a target image recognition model trained according to any one of the image recognition model training methods according to claims 1 to 5, and determine the predicted sample attribute information.

13. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the image recognition model training method as described in any one of claims 1 to 5, or to implement the image recognition method as described in claim 6.

14. A storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the image recognition model training method as described in any one of claims 1 to 5, or implement the image recognition method as described in claim 6.

15. A computer program product, characterized in that, The computer program product includes a computer program stored in a readable storage medium, wherein at least one processor of a computer device reads from and executes the computer program, causing the device to perform the image recognition model training method as described in any one of claims 1 to 5, or to implement the image recognition method as described in claim 6.

Images