Weak texture clue mining method, device, equipment and medium

Abstract

The present invention is applicable to the field of image processing technology, and in particular relates to a weak texture clue mining method, device, equipment and medium. In this application, an encoder is first used to extract weak texture difference features in a weak texture image, a weak texture difference feature map is obtained and a first loss is calculated, and then a generator is used to migrate the weak texture difference features to obtain a weak texture migration image and a second loss is calculated; then a discriminator is used to predict the category of the weak texture migration image and the weak texture image, a first probability and a second probability are obtained, and a third loss and a fourth loss are calculated; in addition, a classifier is used to distinguish the weak texture category of the weak texture difference feature map and a fifth loss is calculated; finally, the encoder, the generator, and the discriminator are trained according to the first loss, the second loss, the third loss and the fourth loss, and the classifier is trained according to the fifth loss; finally, the trained encoder and classifier are combined to improve the accuracy of weak texture image classification.

Description

Technical Field

[0001] The present invention is applicable to the field of image processing technology, and in particular relates to a weak texture clue mining method, device, equipment and medium. Background Art

[0002] Currently, most weak texture classification methods focus on weak texture clue mining, which can be roughly divided into three types: weak texture clue mining methods based on general classification, weak texture clue mining methods based on auxiliary supervision, and weak texture clue mining methods based on anomaly detection.

[0003] Specifically, weak texture clue mining methods based on general classification can be further divided into weak texture clue mining methods based on manual features and weak texture clue mining methods based on deep features. Among them, weak texture clue mining methods based on manual feature extraction mainly use feature gradient operators to extract some invariant gradient information from weak texture images, and use this as category clues to guide the classifier to complete the classification task; weak texture clue mining methods based on deep features mainly achieve an intuitive mapping from input images to category labels by establishing an end-to-end deep learning framework, and reduce the complexity of feature extraction in a data-driven manner.

[0004] However, this intuitive classification framework has two limitations: 1) Since coarse-grained category labels cannot accurately reflect fine-grained weak texture differences, the model is likely to learn invalid pseudo-features; 2) Due to the category closure of the Softmax function, the trained model is difficult to apply to untrained weak texture recognition scenarios (it cannot be adapted to open-set test scenarios).

[0005] To address the first issue, methods for mining weak texture cues based on auxiliary supervision primarily introduce prior strategies for auxiliary training. These strategies are then supervised and constrained during network optimization, improving the discriminability of model feature extraction. However, this method relies heavily on the prior strategy, resulting in limited model applicability and low accuracy in most scenarios.

[0006] To address the second issue, weak texture clue mining methods based on anomaly detection typically require first splitting the training set into two subsets, "normal" and "abnormal," corresponding to different weak texture types. Then, by introducing a contrastive learning strategy that only supervises and constrains normal samples while leaving abnormal samples unconstrained, it is possible to generate weak texture difference feature maps, which can then be used for subsequent classification tasks. However, this weak texture clue mining method only supervises and constrains normal samples, resulting in low model reliability and accuracy.

[0007] Therefore, in the weak texture clue mining scenario, how to improve the accuracy of the model becomes an urgent problem to be solved. Summary of the Invention

[0008] In view of this, embodiments of the present invention provide a weak texture clue mining method, apparatus, device and medium to solve the problem of low accuracy of weak texture clue mining models and mining results in the prior art.

[0009] In a first aspect, an embodiment of the present invention provides a weak texture clue mining method, which uses a trained mining model to extract weak texture difference features in a target weak texture image to be mined, and identifies the category of the weak texture difference features to obtain a weak texture classification result. The trained mining model includes: a trained encoder and a trained classifier. The training method of the trained mining model includes:

[0010] Obtain image sets corresponding to M category labels, each image set including at least one weak texture image, use an encoder to extract weak texture difference features of all weak texture images, obtain corresponding weak texture difference feature maps, and calculate a first loss of the encoder, where M is an integer greater than 1;

[0011] Select any weak texture image from each image set as a reference image to obtain M reference images, use any reference image as a target image, use a generator to migrate the weak texture difference features corresponding to the M reference images to the target image, obtain the corresponding weak texture migration image, traverse all reference images to obtain the weak texture migration image corresponding to the reference image, traverse all weak texture images to obtain the weak texture migration image corresponding to the weak texture image, and calculate the second loss of the generator;

[0012] Using the discriminator to perform category prediction on all weak texture migration images, obtain predicted labels corresponding to the weak texture migration images, calculate a first probability that the predicted labels of the weak texture migration images are consistent with the corresponding category labels, use the discriminator to perform category prediction on all weak texture images, obtain predicted labels corresponding to the weak texture images, calculate a second probability that the predicted labels of the weak texture images are consistent with the corresponding category labels, and calculate a third loss of the generator and a fourth loss of the discriminator;

[0013] Training the encoder, the generator, and the discriminator according to the first loss, the second loss, the third loss, and the fourth loss to obtain a trained encoder, a trained generator, and a trained discriminator;

[0014] Using the trained encoder to extract weak texture difference features of all weak texture images to obtain optimized weak texture difference feature maps, using a classifier to identify categories of all optimized weak texture difference feature maps to obtain classification results corresponding to the weak texture images, and calculating a fifth loss of the classifier;

[0015] The classifier is trained according to the fifth loss to obtain a trained classifier.

[0016] In a second aspect, an embodiment of the present invention provides an artificial intelligence-based weak texture clue mining device, the weak texture clue mining device comprising:

[0017] A feature extraction module is configured to obtain image sets corresponding to M class labels, each image set including at least one weak texture image, extract weak texture difference features from all weak texture images using an encoder to obtain corresponding weak texture difference feature maps, and calculate a first loss of the encoder, where M is an integer greater than 1;

[0018] A feature migration module is configured to select any weak texture image from each image set as a reference image to obtain M reference images, use any reference image as a target image, use a generator to migrate the weak texture difference features corresponding to the M reference images to the target image to obtain the corresponding weak texture migration image, traverse all reference images to obtain the weak texture migration image corresponding to the reference image, traverse all weak texture images to obtain the weak texture migration image corresponding to the weak texture image, and calculate the second loss of the generator;

[0019] a category prediction module, configured to use a discriminator to perform category prediction on all weak texture transfer images, obtain predicted labels for corresponding weak texture transfer images, calculate a first probability that the predicted labels of the weak texture transfer images are consistent with the corresponding category labels, use the discriminator to perform category prediction on all weak texture images, obtain predicted labels for corresponding weak texture images, calculate a second probability that the predicted labels of the weak texture images are consistent with the corresponding category labels, and calculate a third loss of the generator and a fourth loss of the discriminator;

[0020] a first model training module, configured to train the encoder, the generator, and the discriminator according to the first loss, the second loss, the third loss, and the fourth loss to obtain a trained encoder, a trained generator, and a trained discriminator;

[0021] a loss calculation module, configured to extract weak texture difference features of all weak texture images using the trained encoder to obtain optimized weak texture difference feature maps, identify the categories of all optimized weak texture difference feature maps using a classifier, obtain classification results corresponding to the weak texture images, and calculate a fifth loss of the classifier;

[0022] The second model training module is used to train the classifier according to the fifth loss to obtain a trained classifier.

[0023] In a third aspect, an embodiment of the present invention provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the weak texture clue mining method as described in the first aspect is implemented.

[0024] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the weak texture clue mining method as described in the first aspect is implemented.

[0025] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0026] By obtaining an image set corresponding to M category labels, each image set includes at least one weak texture image, using an encoder to extract the weak texture difference features of all weak texture images, obtaining the corresponding weak texture difference feature map, and calculating the first loss of the encoder, selecting any weak texture image from each image set as a reference image, obtaining M reference images, taking any reference image as the target image, using a generator to migrate the weak texture difference features of the weak texture difference feature map of the M reference images to the target image, obtaining the corresponding weak texture migration image, traversing all reference images, obtaining the weak texture migration image of the corresponding reference image, traversing all weak texture images, obtaining the weak texture migration image of the corresponding weak texture image, and calculating the second loss of the generator, using a discriminator to perform category prediction on all weak texture migration images, obtaining the predicted label of the corresponding weak texture migration image, calculating the first probability that the predicted label of the weak texture migration image is consistent with the category label to which the corresponding weak texture image belongs, using a discriminator to perform category prediction on all weak texture images, and obtaining the corresponding The predicted label of the weak texture image is calculated, and the second probability that the predicted label of the weak texture image is consistent with the corresponding category label is calculated, and the third loss of the generator and the fourth loss of the discriminator are calculated. According to the first loss, the second loss, the third loss and the fourth loss, the encoder, the generator and the discriminator are trained to obtain a trained encoder, a trained generator and a trained discriminator. The trained encoder is used to extract the weak texture difference features of all weak texture images to obtain an optimized weak texture difference feature map. The classifier is used to identify the category of all optimized weak texture difference feature maps to obtain the classification result of the corresponding weak texture image, and the fifth loss of the classifier is calculated. The classifier is trained according to the fifth loss to obtain a trained classifier. The weak texture difference features are extracted by the encoder, the weak texture difference features are transferred according to the generator, and classification is performed according to the discriminator for adversarial training. The trained encoder is used to train the classifier, thereby obtaining a trained mining model to identify the category of the target weak texture image to be mined, thereby improving the accuracy of weak texture image classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0028] Figure 1 This is a schematic diagram of an application environment of a weak texture clue mining method provided by the first embodiment of the present invention;

[0029] Figure 21 is a flow chart of a weak texture clue mining method provided in the first embodiment of the present invention;

[0030] Figure 3 This is a structural diagram of a difference capture module provided in the first embodiment of the present invention;

[0031] Figure 4 This is a schematic structural diagram of an artificial intelligence-based weak texture clue mining device provided in the second embodiment of the present invention;

[0032] Figure 5 This is a structural diagram of a computer device provided in Example 3 of the present invention. DETAILED DESCRIPTION

[0033] In the following description, specific details such as particular system structures and techniques are provided for purposes of illustration, not limitation, to facilitate a thorough understanding of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

[0034] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0035] It will also be understood that the term "and / or" used in the present description and appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0036] As used in the present specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0037] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0038] References to "one embodiment" or "some embodiments" in the present specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present invention. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0039] Embodiments of the present invention can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0040] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0041] It should be understood that the order of execution of the steps in the following embodiments does not necessarily mean the order in which they are executed. The order in which each process is executed should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0042] In order to illustrate the technical solution of the present invention, specific embodiments are provided below.

[0043] The weak texture clue mining method provided by the first embodiment of the present invention can be applied to Figure 1In an application environment, a client communicates with a server. The client includes but is not limited to a palmtop computer, a desktop computer, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a cloud computer device, a personal digital assistant (PDA), and other computer devices. The server can be implemented with an independent server or a server cluster consisting of multiple servers. The weak texture clue mining method uses a trained mining model to extract weak texture difference features of a target weak texture image to be mined, and identifies the category of the weak texture difference features to obtain a weak texture classification result. The trained mining model includes: a trained encoder and a trained classifier.

[0044] This weak texture clue mining method can be applied to multiple scenarios such as face recognition, remote sensing image classification, scene image classification, etc. For example, in the face recognition scenario in the field of electronic security, it is necessary to identify and verify the face image. Correspondingly, the face image is the target weak texture image to be mined. The weak texture clue mining method can use the trained mining model to extract the weak texture difference features of the face image, and identify the category of the weak texture difference features to obtain the recognition result of the face image, which serves as the basis for verifying whether the corresponding face image is safe, providing technical support for electronic security, thereby improving the safety of security work.

[0045] See also Figure 2 , is a flow chart of a weak texture clue mining method provided by the first embodiment of the present invention. The above-mentioned weak texture clue mining method can be applied to Figure 1 In the client, the training method of the trained mining model may include the following steps:

[0046] Step S201: obtain image sets corresponding to M category labels, each image set includes at least one weak texture image, use an encoder to extract weak texture difference features of all weak texture images, obtain corresponding weak texture difference feature maps, and calculate the first loss of the encoder.

[0047] To complete feature mining and classification of weak texture images, this embodiment obtains a trained encoder and a trained classifier, uses the trained encoder to extract weak texture difference features from the target weak texture image to be mined, and uses the trained classifier to identify the category of the weak texture difference features to obtain a weak texture classification result. For example, in a face recognition scenario in the electronic security field, the trained encoder is used to extract weak texture difference features from the target face image, and the trained classifier is used to identify the category of the weak texture difference features to obtain a face recognition result.

[0048] This embodiment trains the encoder and classifier to obtain a trained encoder and a trained classifier to mine weak texture clues, thereby improving the accuracy of the mining results. Specifically, during the training process, an image set corresponding to M category labels is obtained as a training sample, where M is an integer greater than 1, and each image set and all weak texture images in the image set correspond to the same category label, which is used to distinguish weak texture images of different categories. The encoder can then be used to extract weak texture difference features of all weak texture images, obtain weak texture difference feature maps corresponding to the weak texture images, and calculate the first loss of the encoder as the basis for model training.

[0049] Optionally, the encoder includes K downsampling layers and K upsampling layers;

[0050] Use the encoder to extract the weak texture difference features of all weak texture images and obtain the corresponding weak texture difference feature maps, including:

[0051] For any weak texture image, all weak texture images that are inconsistent with the category label of the weak texture image are determined as negative sample images corresponding to the weak texture image;

[0052] For any up-sampled feature of the weak texture image, determine the anchor feature and the positive sample feature in the up-sampled feature, which are the anchor feature and the positive sample feature corresponding to the weak texture image; for any up-sampled feature of any negative sample image, determine the negative sample feature in the up-sampled feature, which are the negative sample feature corresponding to the weak texture image;

[0053] Traverse all upsampled features of the weak texture image to obtain all anchor features and positive sample features corresponding to the weak texture image, and traverse all upsampled features of all negative sample images to obtain all negative sample features corresponding to the weak texture image;

[0054] Traverse all weak texture images and obtain all anchor features, positive sample features and negative sample features corresponding to all weak texture images;

[0055] Substitute all anchor features and positive sample features of all weak texture images, and all anchor features and negative sample features of all negative sample images into the preset first loss calculation formula to obtain the first loss of the encoder. The first loss calculation formula is:

[0056]

[0057] Where, L triplet is the first loss of the encoder, K is the number of upsampled features of each weak texture image, M is the total number of image sets, and I m is the total number of weak texture images in the mth image set, J miis the total number of negative sample images of the i-th weak texture image in the m-th image set, is the anchor feature in the kth upsampled feature of the i-th weak texture image in the m-th image set, is the positive sample feature in the kth upsampled feature of the i-th weak texture image in the m-th image set, is the negative sample feature in the kth up-sampled feature of the jth negative sample image corresponding to the i-th weak texture image in the m-th image set, and ∈ is a hyperparameter.

[0058] Based on the consistency between the class labels, all negative images corresponding to each weak texture image can be determined. Based on the K upsampled features corresponding to each weak texture object, the K anchor features and K positive features corresponding to each weak texture image, as well as the K negative features corresponding to each negative image, can be determined. During model training, it is hoped that the anchor features and positive features corresponding to the trained model will become increasingly close, while the corresponding negative features and positive features will be as far apart as possible.

[0059] Therefore, for the i-th weak texture image in the m-th image set, the first distance between the positive sample feature and the anchor feature in the k-th upsampling feature is calculated. For the i-th weak texture image and the corresponding j-th negative sample image in the m-th image set, the second distance between the positive sample feature and the negative sample feature in the k-th upsampling feature is calculated. The difference between the first distance and the second distance, as well as the sum of the difference and the preset hyperparameter are calculated. The obtained sum is compared with the preset value 0, and the larger value is used as the sub-loss corresponding to the i-th weak texture image in the m-th image set. The sum of the sub-losses corresponding to all weak texture images is calculated to obtain the first loss of the encoder.

[0060] Correspondingly, when the sum of the difference and the hyperparameter is greater than 0, the first distance is positively correlated with the first loss, and the second distance is negatively correlated with the first loss. Then, during the convergence of the first loss, the corresponding anchor point features and the positive sample features become closer and closer, and the corresponding negative sample features and the positive sample features become farther and farther away, making the features extracted from weak texture images of the same category labels closer and closer, and the features extracted from weak texture images of different category labels become farther and farther away, thereby improving the accuracy of feature extraction of the encoder.

[0061] The above-mentioned step of obtaining image sets corresponding to M category labels, each image set including at least one weak texture image, using an encoder to extract weak texture difference features of all weak texture images, obtaining corresponding weak texture difference feature maps, and calculating the first loss of the encoder, obtaining the weak texture difference feature map and the first loss as the basis for model training, thereby improving the accuracy of model training.

[0062] Step S202: Select any weak texture image from each image set as a reference image to obtain M reference images, use any reference image as the target image, use the generator to migrate the weak texture difference features corresponding to the M reference images to the target image, and obtain the corresponding weak texture migration image. Traverse all reference images to obtain the weak texture migration image corresponding to the reference image, traverse all weak texture images to obtain the weak texture migration image corresponding to the weak texture image, and calculate the second loss of the generator.

[0063] Among them, in order to improve the extraction accuracy of the weak texture difference features of the encoder, this embodiment uses a generator to migrate the extracted weak texture difference features to other weak texture images to generate weak texture migration images, so as to measure the extraction accuracy of the weak texture difference features of the encoder and the accuracy of the weak texture migration and image generation of the generator by identifying the category of the weak texture migration image, thereby obtaining an encoder with higher accuracy, which is used to accurately extract the weak texture difference features of the target weak texture image to be mined and train the classifier to improve the accuracy of the weak texture classification results.

[0064] Specifically, any weak texture image is selected from each image set as a reference image to obtain M reference images. Any reference image is used as the target image, and the generator is used to migrate the weak texture difference features corresponding to the M reference images to the target image to obtain the corresponding weak texture migration image.

[0065] Then traverse all reference images to obtain the weak texture migration image corresponding to the reference image, traverse all weak texture images to obtain the weak texture migration image corresponding to the weak texture image, and calculate the second loss of the generator as the basis for model training.

[0066] Optionally, the weak texture migration image includes a first weak texture migration image and a second weak texture migration image. Accordingly, a generator is used to migrate weak texture difference features of M-1 reference images that do not belong to the target image to the target image to obtain a corresponding first weak texture migration image. A generator is used to migrate weak texture difference features of the target image to the target image to obtain a corresponding second weak texture migration image. The weak texture clue mining method further includes:

[0067] Using the generator, the weak texture difference features of the target image are transferred to the corresponding first weak texture transfer image to obtain the corresponding third weak texture transfer image;

[0068] The second loss of the generator is calculated based on the second weak texture migration image and the third weak texture migration image corresponding to all weak texture images.

[0069] The weak texture migration image includes a first weak texture migration image, a second weak texture migration image, and a third weak texture migration image.

[0070] Specifically, a generator is used to migrate the weak texture difference features of M-1 reference images that do not belong to the target image to the target image respectively, and a corresponding first weak texture migration image is obtained. The first weak texture migration image is an image generated by the target image accepting the migration of weak texture difference features of other categories.

[0071] The generator is used to transfer the weak texture difference features of the target image to the target image to obtain the corresponding second weak texture transfer image. The second weak texture transfer image is an image generated by the target image accepting the transfer of its own weak texture difference features. The self-loop consistency loss of the generator can be measured based on the difference between the second weak texture transfer image and the corresponding target image.

[0072] The generator is further used to migrate the weak texture difference features of the target image to the corresponding first weak texture migration image to obtain the corresponding third weak texture migration image. The third weak texture migration image is an image generated by the target image successively accepting the migration of weak texture difference features of other categories and its own weak texture difference features. According to the difference between the third weak texture migration image and the corresponding target image, the inter-cycle consistency loss of the generator can be measured.

[0073] The sum of the self-loop consistency loss and the mutual-loop consistency loss of the generator can be determined as the second loss of the generator, which serves as the basis for model training.

[0074] Optionally, calculating the second loss of the generator according to the second weak texture migration image and the third weak texture migration image corresponding to all the weak texture images includes:

[0075] For any second weak texture migration image, calculate the first minimum mean square loss between the second weak texture migration image and the corresponding image, traverse all second weak texture migration images, and obtain all first minimum mean square losses;

[0076] The sum of all first least mean square losses is determined as the first sub-loss of the generator;

[0077] For any third weak texture transfer image, calculate the second minimum mean square loss between the third weak texture transfer image and the corresponding image, traverse all third weak texture transfer images, and obtain all second minimum mean square losses;

[0078] The sum of all second least mean square losses is determined as the second sub-loss of the generator;

[0079] The sum of the first sub-loss and the second sub-loss is determined as the second loss of the generator.

[0080] Among them, for any second weak texture migration image, the first minimum mean square loss between the second weak texture migration image and the corresponding image is calculated to characterize the difference between the corresponding second weak texture migration image and the corresponding image, then all second weak texture migration images are traversed to obtain all first minimum mean square losses, and the sum of all first minimum mean square losses is determined as the first sub-loss of the generator to characterize the self-loop consistency loss of the generator.

[0081] For any third weak texture transfer image, the second minimum mean square loss between the third weak texture transfer image and the corresponding image is calculated to characterize the difference between the corresponding third weak texture transfer image and the corresponding image. Then, all third weak texture transfer images are traversed to obtain all second minimum mean square losses. The sum of all second minimum mean square losses is determined as the second sub-loss of the generator to characterize the inter-cycle consistency loss of the generator.

[0082] Finally, the sum of the first sub-loss and the second sub-loss is determined as the second loss of the generator, which serves as the basis for model training.

[0083] In one embodiment, the first sub-loss of the generator is:

[0084]

[0085] Where, L self_cyc is the first sub-loss of the generator, V is the total number of weak texture images, x v is the vth weak texture image, is the second weak texture migration image of the vth weak texture image.

[0086] The second sub-loss of the generator is:

[0087]

[0088] Where, L mutual_cyc is the second sub-loss of the generator, V is the total number of weak texture images, M-1 is the total number of reference images corresponding to each weak texture image, x v is the vth weak texture image, is the third weak texture migration image corresponding to the qth reference image of the vth weak texture image.

[0089] Then the second loss of the generator is:

[0090] L cyc =L mutual_cyc +L self_cyc

[0091] Where, L cyc is the second loss of the generator, Lself_cyc is the first sub-loss of the generator, L mutual_cyc is the second sub-loss of the generator.

[0092] The above steps include selecting any weak texture image from each image set as a reference image to obtain M reference images, taking any reference image as the target image, using the generator to migrate the weak texture difference features corresponding to the M reference images to the target image, obtaining the corresponding weak texture migration image, traversing all reference images to obtain the weak texture migration image corresponding to the reference image, traversing all weak texture images to obtain the weak texture migration image corresponding to the weak texture image, and calculating the second loss of the generator, using the generator to migrate the extracted weak texture difference features to other weak texture images to generate a weak texture migration image, and calculating the second loss of the weak texture migration image generator, which serves as the basis for model training and improves the accuracy of the model.

[0093] Step S203: Use the discriminator to perform category prediction on all weak texture transfer images, obtain the predicted label of the corresponding weak texture transfer image, calculate the first probability that the predicted label of the weak texture transfer image is consistent with the corresponding category label, use the discriminator to perform category prediction on all weak texture images, obtain the predicted label of the corresponding weak texture image, calculate the second probability that the predicted label of the weak texture image is consistent with the corresponding category label, and calculate the third loss of the generator and the fourth loss of the discriminator.

[0094] Among them, the discriminator is used to make category judgments on weak texture transfer images and weak texture images, and to judge whether the weak texture transfer images and weak texture images are real images, so as to form adversarial training with the generator.

[0095] Specifically, this embodiment uses a discriminator to perform category prediction on all weak texture migration images, obtains the predicted label of the corresponding weak texture migration image, and then calculates the first probability that the predicted label of the weak texture migration image is consistent with the corresponding category label; and uses the discriminator to perform category prediction on all weak texture images, obtains the predicted label of the corresponding weak texture image, and calculates the second probability that the predicted label of the weak texture image is consistent with the corresponding category label to characterize the classification accuracy of the discriminator. The third loss of the generator and the fourth loss of the discriminator can be calculated based on the first probability and the second probability as the basis for model training to improve the accuracy of the model.

[0096] Optionally, the discriminator includes M discriminant units, the first probability includes a first sub-probability, a second sub-probability, and a third sub-probability, and the second probability includes a fourth sub-probability;

[0097] Using the discriminator to predict the category of all weak texture migration images, obtaining the predicted label of the corresponding weak texture migration image, calculating the first probability that the predicted label of the weak texture migration image is consistent with the corresponding category label, using the discriminator to predict the category of all weak texture images, obtaining the predicted label of the corresponding weak texture image, and calculating the second probability that the predicted label of the weak texture image is consistent with the corresponding category label, including:

[0098] For any first weak texture transfer image, determining the category label of the weak texture image corresponding to the first weak texture transfer image as a first category label, determining a first discriminant unit corresponding to the first weak texture transfer image based on the first category label, and classifying the first weak texture transfer image using the first discriminant unit to obtain a first predicted label for the first weak texture transfer image and a first sub-probability that the first predicted label is consistent with the first category label;

[0099] Determining the category label corresponding to the weak texture difference feature map corresponding to the first weak texture migration image as a second category label, determining a second discriminant unit corresponding to the first weak texture migration image based on the second category label, and classifying the first weak texture migration image using the second discriminant unit to obtain a second predicted label for the first weak texture migration image and a second sub-probability that the second predicted label is consistent with the second category label;

[0100] Traversing all first weak texture migration images, obtaining first sub-probabilities and second sub-probabilities corresponding to all weak texture migration images;

[0101] For any second weak texture transfer image, determining the category label corresponding to the weak texture difference feature map corresponding to the second weak texture transfer image as a third category label, determining a third discriminant unit corresponding to the second weak texture transfer image based on the third category label, and using the third discriminant unit to classify the second weak texture transfer image to obtain a third predicted label for the second weak texture transfer image and a third sub-probability that the third predicted label is consistent with the third category label;

[0102] Traverse all the second weak texture migration images to obtain the third sub-probabilities corresponding to all the weak texture migration images;

[0103] For any weak texture image, determining the category label of the weak texture image as a fourth category label, determining a fourth discriminant unit corresponding to the weak texture image based on the fourth category label, and classifying the weak texture image using the fourth discriminant unit to obtain a fourth predicted label of the weak texture image and a fourth sub-probability that the fourth predicted label is consistent with the fourth category label;

[0104] Traverse all weak texture images and obtain the fourth sub-probability corresponding to all weak texture images.

[0105] Among them, in order to improve the accuracy of judging weak texture migration images and weak texture images, the discriminator in this embodiment includes M discrimination units, each discrimination unit corresponds to a category label, and is used to perform category judgment on images belonging to the corresponding category label, thereby performing targeted classification of weak texture migration images and weak texture images, thereby improving the rationality and accuracy of the judgment results.

[0106] Specifically, the first weak texture migration image is obtained by the generator migrating the weak texture difference features of the weak texture difference feature maps of M-1 reference images that do not belong to the target image to the target image, then the category label of the weak texture image corresponding to the first weak texture migration image is inconsistent with the category label of the corresponding weak texture difference feature map.

[0107] For the generator, when the category label corresponding to the weak texture difference feature of the input first weak texture migration image is inconsistent with the category label corresponding to the target image, the purpose of the generator is to migrate the weak texture difference feature with inconsistent category labels to the target image. Then the category label corresponding to the corresponding weak texture difference feature map can be determined as the second category label, and the second discriminant unit corresponding to the first weak texture migration image is determined according to the second category label. The first weak texture migration image is classified using the second discriminant unit to obtain the second predicted label of the first weak texture migration image. The second discriminant unit needs to judge the result of the first weak texture migration image generated by the generator as "true", that is, the second predicted label should be close to the corresponding second category label to ensure the generation accuracy of the generator. Therefore, the second sub-probability that the second predicted label is consistent with the second category label is calculated, and the second sub-probability is used as the basis for calculating the third loss of the generator.

[0108] For the discriminator, when the category label corresponding to the weak texture difference feature of the input first weak texture transfer image is inconsistent with the category label corresponding to the target image, the purpose of the discriminator is to correctly judge the category to which the first weak texture transfer image belongs and to form a confrontation with the generator. The category label of the weak texture image corresponding to the first weak texture transfer image can be determined as the first category label, and the first discriminant unit corresponding to the first weak texture transfer image is determined according to the first category label. The first discriminant unit is used to classify the first weak texture transfer image to obtain the first predicted label of the first weak texture transfer image. The first discriminant unit needs to judge the result of the first weak texture transfer image generated by the generator as "false", that is, the first predicted label should be far away from the corresponding first category label to ensure the discriminant accuracy of the discriminator. Therefore, the first sub-probability that the first predicted label is consistent with the first category label is calculated, and the first sub-probability is used as the basis for calculating the fourth loss of the discriminator.

[0109] The second weak texture migration image is obtained by the generator migrating the weak texture difference features of the weak texture difference feature map of the target image to the target image, and the category label of the weak texture image corresponding to the second weak texture migration image is consistent with the category label of the corresponding weak texture difference feature map.

[0110] For the generator, when the category label corresponding to the weak texture difference feature of the input second weak texture migration image is consistent with the category label corresponding to the target image, the purpose of the generator is to migrate the weak texture difference feature with the consistent category label to the target image. Then the category label corresponding to the corresponding weak texture difference feature map can be determined as the third category label, and the third discriminant unit corresponding to the second weak texture migration image is determined according to the third category label. The second weak texture migration image is classified using the third discriminant unit to obtain the third predicted label of the second weak texture migration image. Then the third discriminant unit needs to judge the result of the second weak texture migration image generated by the generator as "true", that is, the third predicted label should be close to the corresponding third category label to ensure the generation accuracy of the generator. Therefore, the third sub-probability that the third predicted label is consistent with the third category label is calculated, and the third sub-probability is used as the basis for calculating the third loss of the generator.

[0111] For the discriminator, when the input image is a weak texture image, the purpose of the discriminator is to correctly determine the category to which the weak texture image belongs. The category label of the weak texture image can be determined as the fourth category label, and the fourth discriminant unit corresponding to the weak texture image is determined according to the fourth category label. The weak texture image is classified using the fourth discriminant unit to obtain the fourth predicted label of the weak texture image. The fourth discriminant unit needs to judge the result of the weak texture image as "true", that is, the fourth predicted label should be close to the corresponding fourth category label to ensure the discriminant accuracy of the discriminator. Therefore, the fourth sub-probability that the fourth predicted label is consistent with the fourth category label is calculated, and the fourth sub-probability is used as the basis for calculating the fourth loss of the discriminator.

[0112] Optionally, the third loss of the generator and the fourth loss of the discriminator are calculated, including:

[0113] Calculating a third loss of the generator according to the second sub-probability corresponding to all first weak texture transfer images and the third sub-probability corresponding to all second weak texture transfer images;

[0114] The fourth loss of the discriminator is calculated according to the first sub-probability corresponding to all the first weak texture migration images and the fourth sub-probability corresponding to all the weak texture images.

[0115] Among them, the second sub-probability is the probability that the corresponding second predicted label is consistent with the second category label, and the third sub-probability is the probability that the corresponding third predicted label is consistent with the third category label. Correspondingly, the second sub-probability is positively correlated with the accuracy of the generator, and the third sub-probability is positively correlated with the accuracy of the generator. Therefore, the second sub-probability corresponding to all first weak texture migration images and the third sub-probability corresponding to all second weak texture migration images are substituted into the preset third loss calculation formula to obtain the third loss of the generator.

[0116] The first sub-probability is the probability that the corresponding first predicted label is consistent with the first category label, and the fourth sub-probability is the probability that the corresponding fourth predicted label is consistent with the fourth category label. Correspondingly, the third sub-probability is negatively correlated with the accuracy of the discriminator, and the fourth sub-probability is positively correlated with the accuracy of the generator. Therefore, the first sub-probability corresponding to all first weak texture migration images and the fourth sub-probability corresponding to all weak texture images are substituted into the preset fourth loss calculation formula to obtain the fourth loss of the discriminator.

[0117] In one embodiment, the preset third loss calculation formula is:

[0118]

[0119] Where, L g_adv is the third loss of the generator, U is the total number of the first weak texture migration images, W is the total number of the second weak texture migration images, is the second sub-probability corresponding to the u-th first weak texture migration image, is the third sub-probability corresponding to the w-th second weak texture migration image.

[0120] The preset fourth loss calculation formula is:

[0121]

[0122] Where, L d_adv is the fourth loss of the discriminator, M is the total number of image sets, I m is the total number of weak texture images in the mth image set, S mi is the total number of first weak texture migration images corresponding to the i-th weak texture image in the m-th image set, is the sth image corresponding to the i-th weak texture image in the m-th image set mi The first sub-probability corresponding to the first weak texture migration image, is the fourth sub-probability corresponding to the i-th weak texture image in the m-th image set.

[0123] The above steps use the discriminator to perform category prediction on all weak texture transfer images, obtain the predicted label of the corresponding weak texture transfer image, calculate the first probability that the predicted label of the weak texture transfer image is consistent with the corresponding category label, use the discriminator to perform category prediction on all weak texture images, obtain the predicted label of the corresponding weak texture image, calculate the second probability that the predicted label of the weak texture image is consistent with the corresponding category label, and calculate the third loss of the generator and the fourth loss of the discriminator. Use the discriminator to perform category judgment on the weak texture transfer image and the weak texture image, and calculate the third loss and the fourth loss. Combined with the generator for adversarial training, the accuracy of the generator and the discriminator is improved.

[0124] Step S204: Train the encoder, generator, and discriminator according to the first loss, the second loss, the third loss, and the fourth loss to obtain a trained encoder, a trained generator, and a trained discriminator.

[0125] The first, second, third, and fourth losses are negatively correlated with the accuracy of the encoder, generator, and discriminator. Therefore, the encoder, generator, and discriminator can be trained based on the first, second, third, and fourth losses. For example, gradient backpropagation can be used to correct the model parameters of the encoder, generator, and discriminator until the sum of the first, second, third, and fourth losses converges, resulting in a trained encoder, a trained generator, and a trained discriminator.

[0126] The above steps of training the encoder, generator, and discriminator according to the first loss, the second loss, the third loss, and the fourth loss to obtain a trained encoder, a trained generator, and a trained discriminator, based on the negative correlation between the first loss, the second loss, the third loss, and the fourth loss and the accuracy of the encoder, the generator, and the discriminator, thereby improving the accuracy of the encoder, the generator, and the discriminator.

[0127] Step S205: Use the trained encoder to extract weak texture difference features of all weak texture images to obtain optimized weak texture difference feature maps, use a classifier to identify the categories of all optimized weak texture difference feature maps, obtain the classification results of the corresponding weak texture images, and calculate the fifth loss of the classifier.

[0128] The mining model includes an encoder and a classifier. After obtaining the trained encoder, the classifier is trained based on the trained encoder to improve the classification accuracy of the classifier, thereby obtaining a trained classifier.

[0129] Specifically, the image set corresponding to M category labels is used as training samples, and the trained encoder is used to extract the weak texture difference features of all weak texture images to obtain optimized weak texture difference feature maps. Then, a classifier is used to identify the categories of all optimized weak texture difference feature maps to obtain the classification results of the corresponding weak texture images, and the fifth loss of the classifier is calculated to characterize the classification accuracy of the classifier, and serve as the basis for training the classifier to improve the classification accuracy.

[0130] In one embodiment, the classifier includes Y difference capture modules, where Y is an integer greater than 0. The weak texture difference feature map is regarded as the input tensor of the difference capture module. Each difference capture module can be divided into three sub-processes, see Figure 3 , for the zth (z=1, 2, ..., V) weak texture difference feature map and the yth (y=1, 2, ..., Y-1) difference capture module:

[0131] In the first subprocess, the input tensor T z First, pass through two layers of 3×3 convolutional layers to generate tensors in turn and , and the tensor and The size is Then, by applying the tensor and By doing the difference, we can get the first residual tensor

[0132] In the second subprocess, the first residual tensor is fed into an upsampling layer to obtain a tensor Then, after another residual propagation, we get the tensor

[0133] In the third subprocess, the tensor After one residual propagation, we get the tensor Then, through the sum operation, the tensor and tensors Add and get the result of the yth DCB module (i.e. the input of the y+1th DCB module). Similarly, after Y DCB blocks, a low-dimensional feature vector T′ will be obtained. z ∈R 512 .

[0134] Then the fifth loss of the classifier is:

[0135]

[0136] Where, L clsis the fifth loss of the classifier, V is the total number of weak texture images, M is the total number of category labels, l z is the category label corresponding to the z-th weak texture image, T′ z is the low-dimensional feature vector of the z-th weak texture image, is the weight matrix corresponding to the category label of the z-th weak texture image, is the deviation vector corresponding to the category label of the z-th weak texture image, W e is the weight matrix corresponding to the e-th category label, is the deviation vector corresponding to the e-th category label.

[0137] The above-mentioned use of the trained encoder to extract the weak texture difference features of all weak texture images to obtain optimized weak texture difference feature maps, use the classifier to identify the categories of all optimized weak texture difference feature maps, obtain the classification results of the corresponding weak texture images, and calculate the fifth loss of the classifier. The classifier is trained based on the trained encoder to improve the accuracy of the classifier, and the trained encoder is combined to extract the weak texture difference features of the target weak texture image to be mined, and the classifier is trained to identify the category of the weak texture difference features to obtain the classification results, thereby improving the accuracy of the weak texture classification results of the target weak texture image to be mined.

[0138] Step S206: Train the classifier according to the fifth loss to obtain a trained classifier.

[0139] Among them, the fifth loss is used to characterize the classification accuracy of the classifier. Accordingly, the larger the fifth loss is, the lower the classification accuracy of the classifier is. Therefore, this embodiment corrects the parameters of the classifier based on the gradient descent method until the fifth loss converges to obtain a trained classifier.

[0140] The above step of training the classifier according to the fifth loss to obtain a trained classifier improves the accuracy of the classifier in identifying the category of weak texture images through training.

[0141] The embodiment of the present invention obtains an image set corresponding to M category labels, each image set includes at least one weak texture image, uses an encoder to extract weak texture difference features of all weak texture images, obtains corresponding weak texture difference feature maps, and calculates the first loss of the encoder, selects any weak texture image from each image set as a reference image, obtains M reference images, uses any reference image as a target image, uses a generator to migrate the weak texture difference features corresponding to the M reference images to the target image, obtains the corresponding weak texture migration image, traverses all reference images, obtains the weak texture migration image corresponding to the reference image, traverses all weak texture images, obtains the weak texture migration image corresponding to the weak texture image, and calculates the second loss of the generator, uses a discriminator to perform category prediction on all weak texture migration images, obtains the predicted label of the corresponding weak texture migration image, calculates the first probability that the predicted label of the weak texture migration image is consistent with the category label to which the corresponding weak texture image belongs, uses the discriminator to perform category prediction on all weak texture images, and obtains the corresponding weak texture migration image. The predicted label of the texture image is calculated, and the second probability that the predicted label of the weak texture image is consistent with the corresponding category label is calculated, and the third loss of the generator and the fourth loss of the discriminator are calculated. According to the first loss, the second loss, the third loss and the fourth loss, the encoder, the generator and the discriminator are trained to obtain a trained encoder, a trained generator and a trained discriminator. The trained encoder is used to extract the weak texture difference features of all weak texture images to obtain an optimized weak texture difference feature map. The classifier is used to identify the category of all optimized weak texture difference feature maps to obtain the classification result of the corresponding weak texture image, and the fifth loss of the classifier is calculated. The classifier is trained according to the fifth loss to obtain a trained classifier. The weak texture difference features are extracted by the encoder, the weak texture difference features are transferred according to the generator, and classification is performed according to the discriminator for adversarial training. The trained encoder is used to train the classifier, thereby obtaining a trained mining model to identify the category of the target weak texture image to be mined, thereby improving the accuracy of weak texture image classification.

[0142] Corresponding to the weak texture clue mining method of the above embodiment, Figure 4 A structural block diagram of an artificial intelligence-based weak texture clue mining device provided in the second embodiment of the present invention is given. For the sake of ease of explanation, only the parts related to the embodiment of the present invention are shown.

[0143] See also Figure 4 , the weak texture clue mining device includes:

[0144] A feature extraction module 41 is configured to obtain image sets corresponding to M class labels, each image set including at least one weak texture image, extract weak texture difference features from all weak texture images using an encoder, obtain corresponding weak texture difference feature maps, and calculate a first loss of the encoder, where M is an integer greater than 1;

[0145] A feature transfer module 42 is configured to select any weak texture image from each image set as a reference image to obtain M reference images, use any reference image as a target image, use a generator to transfer the weak texture difference features corresponding to the M reference images to the target image to obtain a corresponding weak texture transfer image, traverse all reference images to obtain a weak texture transfer image corresponding to the reference image, traverse all weak texture images to obtain a weak texture transfer image corresponding to the weak texture image, and calculate a second loss of the generator;

[0146] A category prediction module 43 is configured to use a discriminator to perform category prediction on all weak texture transfer images, obtain predicted labels corresponding to the weak texture transfer images, calculate a first probability that the predicted labels of the weak texture transfer images are consistent with the corresponding category labels, use the discriminator to perform category prediction on all weak texture images, obtain predicted labels corresponding to the weak texture images, calculate a second probability that the predicted labels of the weak texture images are consistent with the corresponding category labels, and calculate a third loss of the generator and a fourth loss of the discriminator;

[0147] a first model training module 44, configured to train the encoder, the generator, and the discriminator according to the first loss, the second loss, the third loss, and the fourth loss to obtain a trained encoder, a trained generator, and a trained discriminator;

[0148] a loss calculation module 45 for extracting weak texture difference features from all weak texture images using the trained encoder to obtain optimized weak texture difference feature maps, identifying the categories of all optimized weak texture difference feature maps using a classifier, obtaining classification results corresponding to the weak texture images, and calculating a fifth loss of the classifier;

[0149] The second model training module 46 is used to train the classifier according to the fifth loss to obtain a trained classifier.

[0150] Optionally, the weak texture migration image includes a first weak texture migration image and a second weak texture migration image. Accordingly, a generator is used to migrate weak texture difference features of M-1 reference images that do not belong to the target image to the target image to obtain a corresponding first weak texture migration image. A generator is used to migrate weak texture difference features of the target image to the target image to obtain a corresponding second weak texture migration image. The weak texture clue mining device further includes:

[0151] A third weak texture migration module is used to use the generator to migrate the weak texture difference features of the target image to the corresponding first weak texture migration image to obtain the corresponding third weak texture migration image;

[0152] The second loss calculation module is used to calculate the second loss of the generator based on the second weak texture migration image and the third weak texture migration image corresponding to all weak texture images.

[0153] Optionally, the second loss calculation module includes:

[0154] A first minimum mean square loss calculation submodule is used to calculate, for any second weak texture migration image, a first minimum mean square loss between the second weak texture migration image and the corresponding image, and traverse all second weak texture migration images to obtain all first minimum mean square losses;

[0155] A first sub-loss calculation submodule, configured to determine the sum of all first minimum mean square losses as the first sub-loss of the generator;

[0156] A second minimum mean square loss calculation submodule is used to calculate the second minimum mean square loss between the third weak texture migration image and the corresponding image for any third weak texture migration image, and traverse all third weak texture migration images to obtain all second minimum mean square losses;

[0157] A second sub-loss calculation submodule, configured to determine the sum of all second least mean square losses as the second sub-loss of the generator;

[0158] The second loss calculation submodule is used to determine the sum of the first sub-loss and the second sub-loss as the second loss of the generator.

[0159] Optionally, the discriminator includes M discriminant units, the first probability includes a first sub-probability, a second sub-probability, and a third sub-probability, the second probability includes a fourth sub-probability, and the category prediction module 43 includes:

[0160] a first sub-probability calculation submodule, configured to, for any first weak texture transfer image, determine the category label of the weak texture image corresponding to the first weak texture transfer image as a first category label, determine a first discriminant unit corresponding to the first weak texture transfer image based on the first category label, classify the first weak texture transfer image using the first discriminant unit, and obtain a first predicted label for the first weak texture transfer image, as well as a first sub-probability that the first predicted label is consistent with the first category label;

[0161] A second sub-probability calculation submodule is used to determine the category label corresponding to the weak texture difference feature map corresponding to the first weak texture migration image as a second category label, determine a second discriminant unit corresponding to the first weak texture migration image based on the second category label, and use the second discriminant unit to classify the first weak texture migration image to obtain a second predicted label for the first weak texture migration image, and a second sub-probability that the second predicted label is consistent with the second category label;

[0162] A first image traversal submodule is used to traverse all first weak texture migration images to obtain first sub-probabilities and second sub-probabilities corresponding to all weak texture migration images;

[0163] a third sub-probability calculation submodule, configured to, for any second weak texture migration image, determine, as a third category label, a category label corresponding to a weak texture difference feature map corresponding to the second weak texture migration image; determine, based on the third category label, a third discriminant unit corresponding to the second weak texture migration image; and classify the second weak texture migration image using the third discriminant unit to obtain a third predicted label for the second weak texture migration image and a third sub-probability that the third predicted label is consistent with the third category label.

[0164] The second image traversal submodule is used to traverse all second weak texture migration images to obtain third sub-probabilities corresponding to all weak texture migration images;

[0165] a second probability calculation submodule, configured to, for any weak texture image, determine the category label of the weak texture image as a fourth category label, determine a fourth discriminant unit corresponding to the weak texture image based on the fourth category label, classify the weak texture image using the fourth discriminant unit, and obtain a fourth predicted label for the weak texture image, as well as a fourth sub-probability that the fourth predicted label is consistent with the fourth category label;

[0166] The third image traversal submodule is used to traverse all weak texture images to obtain fourth sub-probabilities corresponding to all weak texture images.

[0167] Optionally, the category prediction module 43 includes:

[0168] A third loss calculation submodule, configured to calculate a third loss of the generator based on the second sub-probability corresponding to all first weak texture migration images and the third sub-probability corresponding to all second weak texture migration images;

[0169] The fourth loss calculation submodule is used to calculate the fourth loss of the discriminator according to the first sub-probability corresponding to all the first weak texture migration images and the second probabilities corresponding to all the weak texture images.

[0170] Optionally, the encoder includes K downsampling layers and K upsampling layers, and the feature extraction module 41 includes:

[0171] The downsampling submodule is used to downsample all weak texture images using K downsampling layers to obtain downsampled features of the corresponding weak texture images;

[0172] A first upsampling submodule is configured to perform an upsampling operation on the downsampled features using the upsampling layer for the first upsampling layer, thereby obtaining a first upsampled feature corresponding to the weak texture image;

[0173] The second upsampling submodule is used to perform an upsampling operation on the s-1th upsampling feature using the upsampling layer for the s-th upsampling layer to obtain the s-th upsampling feature corresponding to the weak texture image, where s=2, 3, ...K-1;

[0174] The third upsampling submodule is used to perform an upsampling operation on the K-1th upsampling feature using K upsampling layers to obtain the Kth upsampling feature and the weak texture difference feature map corresponding to the weak texture image.

[0175] Optionally, the feature extraction module 41 includes:

[0176] A negative sample image determination submodule is used to determine, for any weak texture image, all weak texture images that have inconsistent category labels with the weak texture image as negative sample images corresponding to the weak texture image;

[0177] The first feature extraction submodule is used to determine, for any up-sampled feature of the weak texture image, the anchor feature and the positive sample feature in the up-sampled feature, which are the anchor feature and the positive sample feature corresponding to the weak texture image; and for any up-sampled feature of any negative sample image, determine the negative sample feature in the up-sampled feature, which are the negative sample feature corresponding to the weak texture image;

[0178] The second feature extraction submodule is used to traverse all upsampled features of the weak texture image to obtain all anchor features and positive sample features corresponding to the weak texture image, and traverse all upsampled features of all negative sample images to obtain all negative sample features corresponding to the weak texture image;

[0179] The third feature extraction submodule is used to traverse all weak texture images and obtain all anchor point features, positive sample features and negative sample features corresponding to all weak texture images;

[0180] The first loss calculation submodule is used to substitute all anchor features and positive sample features of all weak texture images, and all anchor features and negative sample features of all negative sample images into the preset first loss calculation formula to obtain the first loss of the encoder. The first loss calculation formula is:

[0181]

[0182] Where, Ltriplet is the first loss of the encoder, K is the number of upsampled features of each weak texture image, M is the total number of image sets, and I m is the total number of weak texture images in the mth image set, J mi is the total number of negative sample images of the i-th weak texture image in the m-th image set, is the anchor feature in the kth upsampled feature of the i-th weak texture image in the m-th image set, is the positive sample feature in the kth upsampled feature of the i-th weak texture image in the m-th image set, is the negative sample feature in the kth up-sampled feature of the jth negative sample image corresponding to the i-th weak texture image in the m-th image set, and ∈ is a hyperparameter.

[0183] Among them, the hyperparameter ∈ is used to control the minimum Euclidean distance of weak texture images with different category labels in the feature space, and the specific value of the hyperparameter ∈ can be set according to actual conditions.

[0184] It should be noted that the information interaction, execution process and other contents between the above modules are based on the same concept as the embodiment of the method of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0185] Figure 5 This is a schematic diagram of the structure of a computer device provided in the third embodiment of the present invention. Figure 5 As shown, the computer device of this embodiment includes: at least one processor ( Figure 5 Only one is shown in the figure), a memory, and a computer program stored in the memory and executable on at least one processor. When the processor executes the computer program, the steps of any of the above-mentioned weak texture clue mining method embodiments are implemented.

[0186] The computer device may include, but is not limited to, a processor and a memory. It will be understood by those skilled in the art that Figure 5 The above is merely an example of a computer device and does not constitute a limitation on the computer device. The computer device may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include a network interface, a display screen, and an input device.

[0187] The processor may be a CPU, or other general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor.

[0188] The memory includes a readable storage medium, an internal memory, etc., wherein the internal memory can be the memory of a computer device, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The readable storage medium can be the hard disk of the computer device. In other embodiments, it can also be an external storage device of the computer device, for example, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Furthermore, the memory can also include both the internal storage unit of the computer device and the external storage device. The memory is used to store the operating system, application programs, boot loaders (BootLoader), data, and other programs, such as the program code of the computer program. The memory can also be used to temporarily store data that has been output or is about to be output.

[0189] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned method embodiment. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include at least: any entity or device capable of carrying computer program code, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0190] The present invention may implement all or part of the processes in the above-mentioned method embodiments, and may also be completed through a computer program product. When the computer program product runs on a computer device, the computer device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0191] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0192] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0193] In the embodiments provided by the present invention, it should be understood that the disclosed apparatus / computer equipment and methods can be implemented in other ways. For example, the apparatus / computer equipment embodiments described above are merely illustrative. For example, the division of modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0194] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0195] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A weak texture clue mining method, characterized in that: Using the trained mining model to extract weak texture difference features in the target weak texture image to be mined, and identifying the category of the weak texture difference features to obtain a weak texture classification result; The trained mining model includes: a trained encoder and a trained classifier, and the training method of the trained mining model includes: Obtain image sets corresponding to M category labels, each image set including at least one weak texture image, use an encoder to extract weak texture difference features of all weak texture images, obtain corresponding weak texture difference feature maps, and calculate a first loss of the encoder, where M is an integer greater than 1; Select any weak texture image from each image set as a reference image to obtain M reference images, use any reference image as a target image, use a generator to migrate the weak texture difference features corresponding to the M reference images to the target image, obtain the corresponding weak texture migration image, traverse all reference images to obtain the weak texture migration image corresponding to the reference image, traverse all weak texture images to obtain the weak texture migration image corresponding to the weak texture image, and calculate the second loss of the generator; Using the discriminator to perform category prediction on all weak texture migration images, obtain predicted labels corresponding to the weak texture migration images, calculate a first probability that the predicted labels of the weak texture migration images are consistent with the corresponding category labels, use the discriminator to perform category prediction on all weak texture images, obtain predicted labels corresponding to the weak texture images, calculate a second probability that the predicted labels of the weak texture images are consistent with the corresponding category labels, and calculate a third loss of the generator and a fourth loss of the discriminator; Training the encoder, the generator, and the discriminator according to the first loss, the second loss, the third loss, and the fourth loss to obtain a trained encoder, a trained generator, and a trained discriminator; Using the trained encoder to extract weak texture difference features of all weak texture images to obtain optimized weak texture difference feature maps, using a classifier to identify categories of all optimized weak texture difference feature maps to obtain classification results corresponding to the weak texture images, and calculating a fifth loss of the classifier; The classifier is trained according to the fifth loss to obtain a trained classifier.

2. The weak texture clue mining method according to claim 1, characterized in that: The weak texture migration image includes a first weak texture migration image and a second weak texture migration image. Accordingly, a generator is used to migrate weak texture difference features of M-1 reference images that do not belong to the target image to the target image to obtain a corresponding first weak texture migration image; Using a generator to migrate the target image's weak texture difference features to the target image to obtain a corresponding second weak texture migration image, the weak texture clue mining method further comprising: Using a generator, migrating the weak texture difference features of the target image to the corresponding first weak texture migration image to obtain a corresponding third weak texture migration image; The second loss of the generator is calculated according to the second weak texture migration image and the third weak texture migration image corresponding to all the weak texture images.

3. The weak texture clue mining method according to claim 2, characterized in that: The calculating the second loss of the generator according to the second weak texture migration image and the third weak texture migration image corresponding to all the weak texture images includes: For any second weak texture migration image, calculate the first minimum mean square loss between the second weak texture migration image and the corresponding weak texture image, traverse all second weak texture migration images, and obtain all first minimum mean square losses; Determine the sum of all first least mean square losses as the first sub-loss of the generator; For any third weak texture migration image, calculate the second minimum mean square loss between the third weak texture migration image and the corresponding weak texture image, traverse all third weak texture migration images, and obtain all second minimum mean square losses; Determine the sum of all second least mean square losses as the second sub-loss of the generator; The sum of the first sub-loss and the second sub-loss is determined as the second loss of the generator.

4. The weak texture clue mining method according to claim 2, characterized in that: The discriminator includes M discriminant units, the first probability includes a first sub-probability, a second sub-probability and a third sub-probability, and the second probability includes a fourth sub-probability; The method comprises: using a discriminator to perform category prediction on all weak texture migration images to obtain predicted labels corresponding to the weak texture migration images, calculating a first probability that the predicted labels of the weak texture migration images are consistent with the corresponding category labels, using the discriminator to perform category prediction on all weak texture images to obtain predicted labels corresponding to the weak texture images, and calculating a second probability that the predicted labels of the weak texture images are consistent with the corresponding category labels. For any first weak texture transfer image, determining the category label of the weak texture image corresponding to the first weak texture transfer image as a first category label, determining a first discriminant unit corresponding to the first weak texture transfer image based on the first category label, and classifying the first weak texture transfer image using the first discriminant unit to obtain a first predicted label for the first weak texture transfer image and a first sub-probability that the first predicted label is consistent with the first category label; Determining a category label corresponding to a weak texture difference feature map corresponding to the first weak texture migration image as a second category label, determining a second discriminant unit corresponding to the first weak texture migration image based on the second category label, and classifying the first weak texture migration image using the second discriminant unit to obtain a second predicted label for the first weak texture migration image and a second sub-probability that the second predicted label is consistent with the second category label; Traversing all first weak texture migration images, obtaining first sub-probabilities and second sub-probabilities corresponding to all weak texture migration images; For any second weak texture transfer image, determining the category label corresponding to the weak texture difference feature map corresponding to the second weak texture transfer image as a third category label, determining a third discriminant unit corresponding to the second weak texture transfer image based on the third category label, and using the third discriminant unit to classify the second weak texture transfer image to obtain a third predicted label of the second weak texture transfer image and a third sub-probability that the third predicted label is consistent with the third category label; traversing all second weak texture transfer images to obtain third sub-probabilities corresponding to all weak texture transfer images; For any weak texture image, the category label of the weak texture image is determined as a fourth category label, a fourth discriminant unit corresponding to the weak texture image is determined according to the fourth category label, the weak texture image is classified using the fourth discriminant unit to obtain a fourth predicted label of the weak texture image, and a fourth sub-probability that the fourth predicted label is consistent with the fourth category label; all weak texture images are traversed to obtain the fourth sub-probabilities corresponding to all weak texture images.

5. The weak texture clue mining method according to claim 4, characterized in that: The calculating the third loss of the generator and the fourth loss of the discriminator includes: Calculating a third loss of the generator according to the second sub-probability corresponding to all first weak texture migration images and the third sub-probability corresponding to all second weak texture migration images; The fourth loss of the discriminator is calculated according to the first sub-probability corresponding to all the first weak texture migration images and the fourth sub-probability corresponding to all the weak texture images.

6. The weak texture clue mining method according to claim 1, characterized in that: The encoder includes K downsampling layers and K upsampling layers; The encoder is used to extract weak texture difference features of all weak texture images to obtain corresponding weak texture difference feature maps, including: Performing a downsampling operation on all weak texture images using the K downsampling layers to obtain downsampling features corresponding to the weak texture images; For a first upsampling layer, use the upsampling layer to perform an upsampling operation on the downsampled feature to obtain a first upsampled feature corresponding to the weak texture image; For the s-th upsampling layer, use the upsampling layer to perform an upsampling operation on the s-1-th upsampling feature to obtain the s-th upsampling feature corresponding to the weak texture image, where s=2, 3, ...K-1; An upsampling operation is performed on the K-1th upsampling feature using the K upsampling layers to obtain a Kth upsampling feature and a weak texture difference feature map corresponding to the weak texture image.

7. The method for mining weak texture clues according to claim 6, characterized in that: The calculating the first loss of the encoder comprises: For any weak texture image, all weak texture images whose category labels are inconsistent with the weak texture image are determined as negative sample images corresponding to the weak texture image; For any up-sampled feature of the weak texture image, determine the anchor feature and the positive sample feature in the up-sampled feature, which are the anchor feature and the positive sample feature corresponding to the weak texture image; for any up-sampled feature of any negative sample image, determine the negative sample feature in the up-sampled feature, which are the negative sample feature corresponding to the weak texture image; Traversing all upsampled features of the weak texture image to obtain all anchor features and positive sample features corresponding to the weak texture image, and traversing all upsampled features of all negative sample images to obtain all negative sample features corresponding to the weak texture image; Traverse all weak texture images and obtain all anchor features, positive sample features and negative sample features corresponding to all weak texture images; Substitute all anchor features and positive sample features of all weak texture images, and all anchor features and negative sample features of all negative sample images into a preset first loss calculation formula to obtain the first loss of the encoder. The first loss calculation formula is: Where, L triplet is the first loss of the encoder, K is the number of upsampled features of each weak texture image, M is the total number of image sets, I m is the total number of weak texture images in the mth image set, J mi is the total number of negative sample images of the i-th weak texture image in the m-th image set, is the anchor feature in the kth upsampled feature of the i-th weak texture image in the m-th image set, is the positive sample feature in the kth upsampled feature of the i-th weak texture image in the m-th image set, is the negative sample feature in the kth up-sampled feature of the jth negative sample image corresponding to the i-th weak texture image in the m-th image set, and ∈ is a hyperparameter.

8. A device for mining weak texture clues based on image processing technology, characterized in that: The weak texture clue mining device comprises: A feature extraction module is configured to obtain image sets corresponding to M class labels, each image set including at least one weak texture image, extract weak texture difference features from all weak texture images using an encoder to obtain corresponding weak texture difference feature maps, and calculate a first loss of the encoder, where M is an integer greater than 1; A feature migration module is configured to select any weak texture image from each image set as a reference image to obtain M reference images, use any reference image as a target image, use a generator to migrate the weak texture difference features corresponding to the M reference images to the target image to obtain the corresponding weak texture migration image, traverse all reference images to obtain the weak texture migration image corresponding to the reference image, traverse all weak texture images to obtain the weak texture migration image corresponding to the weak texture image, and calculate the second loss of the generator; a category prediction module, configured to use a discriminator to perform category prediction on all weak texture transfer images, obtain predicted labels for corresponding weak texture transfer images, calculate a first probability that the predicted labels of the weak texture transfer images are consistent with the corresponding category labels, use the discriminator to perform category prediction on all weak texture images, obtain predicted labels for corresponding weak texture images, calculate a second probability that the predicted labels of the weak texture images are consistent with the corresponding category labels, and calculate a third loss of the generator and a fourth loss of the discriminator; a first model training module, configured to train the encoder, the generator, and the discriminator according to the first loss, the second loss, the third loss, and the fourth loss to obtain a trained encoder, a trained generator, and a trained discriminator; a loss calculation module, configured to extract weak texture difference features of all weak texture images using the trained encoder to obtain optimized weak texture difference feature maps, identify the categories of all optimized weak texture difference feature maps using a classifier, obtain classification results corresponding to the weak texture images, and calculate a fifth loss of the classifier; The second model training module is used to train the classifier according to the fifth loss to obtain a trained classifier.

9. A computer device, characterized in that: The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the weak texture clue mining method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the weak texture clue mining method according to any one of claims 1 to 7 is implemented.

Images