Image recognition method and device, storage medium and electronic device

Abstract

The application discloses an image recognition method and device, a storage medium and an electronic device. The method comprises the following steps: obtaining a target image to be recognized; inputting the target image into an image recognition model, wherein the image recognition model is a neural network model for recognizing an image subject, which is obtained by training a plurality of original samples and a plurality of target samples, the target sample is a noise sample obtained by combining image subject information corresponding to a first sample and a second sample, the plurality of original samples comprise the first sample and the second sample, and the image subject information is information corresponding to an image subject in the first sample; and obtaining an image recognition result output by the image recognition model, wherein the image recognition result is used for indicating a subject type to which an image subject in the target image belongs, and can be applied to a cloud technology scene. The application solves the technical problem of low image recognition accuracy.

Description

Technical Field

[0001] This application relates to the field of computers, and more specifically, to an image recognition method, apparatus, storage medium, and electronic device. Background Technology

[0002] In image recognition scenarios, models are typically used to identify images. However, for certain scenarios, collecting a large number of defective, noisy samples can be challenging. Further training the image recognition model with a small number of noisy samples may lead to overfitting, resulting in decreased generalization performance and lower image recognition accuracy. Therefore, the problem of low image recognition accuracy exists.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This application provides an image recognition method, apparatus, storage medium, and electronic device to at least solve the technical problem of low image recognition accuracy.

[0005] According to one aspect of the embodiments of this application, an image recognition method is provided, comprising: acquiring a target image to be recognized; inputting the target image into an image recognition model, wherein the image recognition model is a neural network model for recognizing image subjects, trained using multiple original samples and multiple target samples, the target samples being noise samples obtained by combining image subject information corresponding to a first sample and a second sample, the multiple original samples including the first sample and the second sample, the image subject information being information corresponding to the image subject in the first sample; and acquiring an image recognition result output by the image recognition model, wherein the image recognition result is used to indicate the subject type to which the image subject in the target image belongs.

[0006] According to another aspect of the embodiments of this application, an image recognition apparatus is also provided, comprising: a first acquisition unit for acquiring a target image to be recognized; an input unit for inputting the target image into an image recognition model, wherein the image recognition model is a neural network model for recognizing image subjects, trained using multiple original samples and multiple target samples, the target samples being noise samples obtained by combining image subject information corresponding to a first sample and a second sample, the multiple original samples including the first sample and the second sample, and the image subject information being information corresponding to the image subject in the first sample; and a second acquisition unit for acquiring an image recognition result output by the image recognition model, wherein the image recognition result is used to indicate the subject type to which the image subject in the target image belongs.

[0007] As an optional solution, the above-mentioned apparatus further includes: a third acquisition unit, configured to acquire the plurality of original samples before acquiring the target image to be identified; a first training unit, configured to input the training samples from the plurality of original samples into an initial first recognition model for training once before acquiring the target image to be identified, until a trained second recognition model is obtained; an extraction unit, configured to extract information from the first sample from the plurality of original samples based on the model parameters of the second recognition model before acquiring the target image to be identified, to obtain the image subject information; a combination unit, configured to combine the image subject information and the second sample from the plurality of original samples before acquiring the target image to be identified, to obtain the plurality of target samples; and a second training unit, configured to perform secondary training using the plurality of target samples before acquiring the target image to be identified, until a trained image recognition model is obtained.

[0008] As an optional solution, the extraction unit includes: an extraction module, used to extract features from the first sample among the plurality of original samples using the first model parameters of the second recognition model to obtain a plurality of image feature points corresponding to the first sample, wherein the first model parameters are parameters corresponding to the feature processing structure in the second recognition model, and the image feature points are used to represent the image representation of each pixel on the first sample; a weighting module, used to perform weighting processing on the plurality of image feature points using the second model parameters of the second recognition model to obtain the weight value corresponding to each feature point among the plurality of image features, wherein the second model parameters are parameters corresponding to the recognition output layer in the second recognition model; a filtering module, used to filter out a plurality of target features from the plurality of features using the weight values, wherein the weight value corresponding to the target features is greater than or equal to a preset threshold; and an acquisition module, used to acquire the image subject information based on the plurality of target features.

[0009] As an optional solution, the above-mentioned acquisition module includes: a first acquisition submodule, used to acquire image masks corresponding to the above-mentioned multiple target features; a processing submodule, used to perform intersection processing on the above-mentioned image mask and the above-mentioned first sample to obtain an image portion of the above-mentioned first sample that matches the above-mentioned image mask, and to determine the above-mentioned image portion as the image subject of the above-mentioned first sample; and a second acquisition submodule, used to acquire information corresponding to the above-mentioned image portion, wherein the above-mentioned image subject information includes information corresponding to the above-mentioned image portion.

[0010] As an optional solution, the extraction module includes: an extraction submodule, used to extract features from the first sample among the plurality of original samples using the first model parameters to obtain an initial feature map, wherein the initial feature map represents the image features; the weighting module includes: a weighting submodule, used to perform weighting processing on the initial feature map using the second model parameters to obtain a feature activation map, wherein the feature activation map indicates the weight values ​​corresponding to each channel in the initial feature map; the device further includes: a sampling submodule, used to upsample the feature activation map according to the image size of the first sample after performing weighting processing on the initial feature map using the second model parameters to obtain the feature activation map, thereby obtaining a feature response map with the same image size as the first sample; the filtering module includes: a segmentation submodule, used to perform threshold segmentation on the feature response map using the value of each feature point of the feature activation map to obtain an image mask, wherein the plurality of target features include each feature in the image mask.

[0011] As an optional solution, the second training unit includes: a first training module for inputting the plurality of target samples into the second recognition model for training until a trained image recognition model is obtained; or, a second training module for inputting the plurality of target samples into the first recognition model for training until a trained image recognition model is obtained.

[0012] As an optional solution, the above-mentioned apparatus further includes: a fourth acquisition unit, configured to acquire a sample quality label carried by each of the multiple original samples before extracting information from the first sample among the multiple original samples based on the model parameters of the second recognition model to obtain the image subject information, wherein the sample quality label is used to indicate whether the original sample is a clean sample or a noisy sample; and a first screening unit, configured to screen at least one clean original sample from the multiple original samples according to the sample quality label before extracting information from the first sample among the multiple original samples based on the model parameters of the second recognition model to obtain the image subject information, and to determine the clean original sample as the first sample.

[0013] As an optional solution, the above-mentioned apparatus further includes: a fifth acquisition unit, configured to acquire sample type labels carried by each of the multiple original samples before extracting information from the first sample among the multiple original samples based on the model parameters of the second recognition model to obtain the image subject information, wherein the sample type labels are used to indicate that the image subject in the original sample belongs to any one of at least two subject types; and a second filtering unit, configured to filter at least one target original sample from the multiple original samples according to the sample type labels before extracting information from the first sample among the multiple original samples based on the model parameters of the second recognition model to obtain the image subject information, and to determine the target original sample as the first sample, wherein the image subject in the target original sample belongs to the target type, and the image subject in the second sample belongs to the target type.

[0014] According to another aspect of the embodiments of this application, a computer-readable storage medium is provided, characterized in that the computer-readable storage medium includes a computer program, wherein the computer program is executed by an electronic device (such as a user device or a server) to perform the image recognition method described above.

[0015] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the image recognition method described above through the computer program.

[0016] In this embodiment, a target image to be identified is obtained; the target image is input into an image recognition model, wherein the image recognition model is a neural network model trained using multiple original samples and multiple target samples for identifying the subject of an image, the target sample is a noise sample obtained by combining the subject information of the first sample with the second sample, the multiple original samples include the first sample and the second sample, and the subject information is the information corresponding to the subject in the first sample; the image recognition result output by the image recognition model is obtained, wherein the image recognition result is used to indicate the subject type to which the subject in the target image belongs. By adding noise to the original samples, a large number of noise samples are obtained to participate in model training, thereby improving the training quality of the model; furthermore, since the noise addition operation in this embodiment is performed by combining the subject information of different original samples, the difference between the obtained noise samples and the original samples is reduced, thereby not only achieving the goal of significantly increasing the amount of noise data in the training samples, but also, because the enhanced image training is more difficult, the model is less prone to overfitting and the model generalization is improved, thereby achieving the technical effect of improving the accuracy of image recognition using the image recognition model, and thus solving the technical problem of low image recognition accuracy. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 This is a schematic diagram of an application environment for an optional image recognition method according to an embodiment of this application;

[0019] Figure 2 This is a schematic diagram of the flow of an optional image recognition method according to an embodiment of this application;

[0020] Figure 3 This is a schematic diagram of an optional image recognition method according to an embodiment of this application;

[0021] Figure 4 This is a schematic diagram of another optional image recognition method according to an embodiment of this application;

[0022] Figure 5 This is a schematic diagram of another optional image recognition method according to an embodiment of this application;

[0023] Figure 6 This is a schematic diagram of another optional image recognition method according to an embodiment of this application;

[0024] Figure 7 This is a schematic diagram of another optional image recognition method according to an embodiment of this application;

[0025] Figure 8 This is a schematic diagram of another optional image recognition method according to an embodiment of this application;

[0026] Figure 9 This is a schematic diagram of another optional image recognition method according to an embodiment of this application;

[0027] Figure 10 This is a schematic diagram of an optional image recognition device according to an embodiment of this application;

[0028] Figure 11 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] For ease of understanding, the following terms are explained:

[0032] Cloud computing is a computing model that distributes computing tasks across a large pool of computers, enabling various application systems to access computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." From the user's perspective, resources in the "cloud" appear infinitely scalable, readily available, on-demand, and expandable, with payment based on usage.

[0033] As a provider of fundamental cloud computing capabilities, a cloud resource pool (referred to as a cloud platform, generally called an IaaS (Infrastructure as a Service) platform) is established. Various types of virtual resources are deployed in the resource pool for external customers to choose from. The cloud resource pool mainly includes: computing devices (virtualized machines containing operating systems), storage devices, and network devices.

[0034] Based on logical function, a PaaS (Platform as a Service) layer can be deployed on top of the IaaS (Infrastructure as a Service) layer, and a SaaS (Software as a Service) layer can be deployed on top of the PaaS layer. Alternatively, SaaS can be deployed directly on top of IaaS. PaaS is a platform for running software, such as databases and web containers. SaaS refers to various types of business software, such as web portals and bulk SMS senders. Generally speaking, SaaS and PaaS are upper layers compared to IaaS.

[0035] According to one aspect of the embodiments of this application, an image recognition method is provided. Optionally, as an optional implementation, the above-described image recognition method may be applied to, but is not limited to, [examples of other methods]. Figure 1 The environment shown may include, but is not limited to, user equipment 102 and server 112. User equipment 102 may include, but is not limited to, a display 104, a processor 106 and a memory 108. Server 112 includes a database 114 and a processing engine 116.

[0036] The specific process can be summarized in the following steps:

[0037] Step S102: User equipment 102 acquires the target image to be identified;

[0038] Steps S104-S106: Send the target image to the server 112 via network 110;

[0039] In steps S108-S110, server 112 inputs the target image into the image recognition model through processing engine 116 and obtains the image recognition result output by the image recognition model;

[0040] In steps S112-S114, the image recognition result is sent to the user equipment 102 via the network 110. The user equipment 102 displays the image recognition result on the display 104 via the processor 106 and stores the image recognition result in the memory 108.

[0041] remove Figure 1 Beyond the examples shown, the above steps can be performed independently by the user device or the server, or collaboratively by both. For instance, user device 102 can perform steps such as inputting the target image into the image recognition model and obtaining the image recognition result output by the model, thereby reducing the processing load on server 112. User device 102 includes, but is not limited to, handheld devices (such as mobile phones), laptops, tablets, desktop computers, in-vehicle devices, and smart TVs. This application does not limit the specific implementation of user device 102. Server 112 can be a single server, a server cluster consisting of multiple servers, or a cloud server.

[0042] Alternatively, as an alternative implementation method, such as Figure 2 As shown, image recognition methods can be performed by electronic devices, such as... Figure 1 The user equipment or server shown includes the following specific steps:

[0043] S202, Obtain the target image to be identified;

[0044] S204, Input the target image into the image recognition model, wherein the image recognition model is a neural network model for recognizing the main body of the image, which is trained using multiple original samples and multiple target samples. The target sample is a noise sample obtained by combining the image main body information corresponding to the first sample and the second sample. The multiple original samples include the first sample and the second sample. The image main body information is the information corresponding to the image main body in the first sample.

[0045] S206, Obtain the image recognition result output by the image recognition model, wherein the image recognition result is used to indicate the subject type to which the image subject in the target image belongs.

[0046] Optionally, in this embodiment, the above image recognition method can be applied to industrial defect inspection scenarios, but is not limited to industrial defect inspection. Industrial defect inspection can refer to quality inspection of industrial products in the production and manufacturing process. Traditional industrial quality inspection is generally carried out by quality inspection workers through manual visual inspection. In recent years, with the rise of AI technology, AI quality inspection based on machine vision can greatly improve the accuracy of quality inspection and save labor costs.

[0047] Specifically, this technology trains an image recognition model using manually labeled samples. The input to this trained model is an image of the surface of an industrial product, and the output is the confidence score of the defect, thus distinguishing between defective and non-defective images. However, this whole-image classification based on an image recognition model still has some problems. Training the model relies on a large amount of labeled training data. However, in industrial defect inspection scenarios, collecting a large number of defect images is often difficult because factory yields are generally high, resulting in a small number of defect samples. Therefore, under these conditions, training the image recognition model with a small amount of data may lead to overfitting, resulting in decreased generalization performance and poor test results.

[0048] To address the aforementioned technical issues, this embodiment utilizes a noise-adding operation on the original samples to obtain a large number of noisy samples for model training, thereby improving the training quality of the model. Furthermore, since the noise-adding operation in this embodiment combines the image subject information of different original samples, the difference between the resulting noisy samples and the original samples is reduced. This not only significantly increases the amount of noise data in the training samples but also makes the model less prone to overfitting and improves its generalization ability because the enhanced images are more difficult to train.

[0049] Optionally, in this embodiment, the original sample can be understood as, but is not limited to, a sample image labeled with various tags. However, the labeling of various tags may have labeling errors, resulting in the labeling information of the sample image being noisy. Therefore, the original sample can also be divided into clean sample or noisy sample. A clean sample can be understood as, but is not limited to, a sample image without noise labeling information, while a noisy sample can be understood as, but is not limited to, a sample image with noise labeling information.

[0050] It should be noted that, for model training, the inclusion of noisy samples can increase the training difficulty of the model, improve its generalization performance, and enhance the accuracy of image recognition output. However, in certain scenarios, the number of noisy samples is difficult to guarantee, thus failing to achieve the aforementioned improvement in model generalization performance. To increase the amount of noisy sample data, noise can be added to clean samples, but typical noise addition only generalizes noise elements to clean samples, such as adding Gaussian noise to clean samples to obtain noisy samples. However, the noise samples obtained in this way have a large difference from the original samples, and the model can easily identify the noise elements from the original samples, making it difficult to improve the aforementioned model generalization performance.

[0051] In this embodiment, a first sample and a second sample are selected from the original samples. The image subject information of the first sample is used as a specific noise element, and then the second sample is noise-added using the specific noise element to obtain a target sample with noise element and less difference from the original sample. Using the target sample for model training greatly increases the training difficulty of the model, thereby improving the generalization performance of the model and the output accuracy of image recognition, and truly solving the above-mentioned technical problems.

[0052] Optionally, in this embodiment, the image subject information is the information corresponding to the image subject in the first sample. The image subject can be understood as, but is not limited to, a valid identification subject. Taking the target image as an example, the image recognition model can, but is not limited to, first determine the valid identification subject from the target image, then distinguish the type of the valid identification subject, and then output the image recognition result. The image recognition result is used to indicate the subject type to which the image subject in the target image belongs.

[0053] Optionally, in this embodiment, the image recognition result is used to indicate the subject type to which the image subject in the target image belongs. For example, the image recognition result may be, but is not limited to, the confidence level of the subject type to which the image subject in the target image belongs. If the confidence level is greater than or equal to the confidence level threshold, the subject type to which the image subject in the target image belongs is a qualified subject type; conversely, if the confidence level is less than the aforementioned confidence level threshold, the subject type to which the image subject in the target image belongs is an unqualified subject type. In addition, to improve the diversity of image recognition, the subject type may also include, but is not limited to, multiple types, such as a first subject type, a second subject type, a third subject type, etc., and thus the image recognition result may be, but is not limited to, the confidence level of the image subject in the target image belonging to each type of subject type.

[0054] It should be noted that by adding noise to the original samples, a large number of noisy samples are obtained to participate in model training, thereby improving the training quality of the model. Furthermore, since the noise addition operation in this embodiment is performed by combining the image subject information of different original samples, the difference between the resulting noisy samples and the original samples is reduced. This not only significantly increases the amount of noise data in the training samples, but also makes the model less prone to overfitting and improves the model's generalization ability due to the greater difficulty of training the enhanced images. Thus, the technical effect of improving the accuracy of image recognition using image recognition models is achieved.

[0055] To further illustrate, optional examples include... Figure 3As shown, the target image 302 to be identified is obtained; the target image 302 is input into the image recognition model 304, wherein the image recognition model 304 is a neural network model for identifying the subject of the image, which is trained using multiple original samples and multiple target samples 304-4; the target samples 304-4 are noise samples obtained by combining the image subject information 304-3 corresponding to the first sample 304-1 and the second sample 304-2; the multiple original samples include the first sample 304-1 and the second sample 304-2; the image subject information 304-3 is the information 304-3 corresponding to the image subject in the first sample 304-1; the image recognition result 306 output by the image recognition model 304 is obtained, wherein the image recognition result 306 is used to indicate the subject type to which the image subject in the target image 302 belongs.

[0056] The embodiments provided in this application obtain a target image to be identified; the target image is input into an image recognition model, wherein the image recognition model is a neural network model trained using multiple original samples and multiple target samples for identifying the subject of an image, the target sample is a noise sample obtained by combining the subject information of the first sample with the second sample, the multiple original samples include the first sample and the second sample, and the subject information is the information corresponding to the subject of the image in the first sample; the image recognition result output by the image recognition model is obtained, wherein the image recognition result is used to indicate the subject type to which the subject of the image in the target image belongs. By adding noise to the original samples, a large number of noise samples are obtained to participate in model training, thereby improving the training quality of the model; furthermore, since the noise addition operation in this embodiment is performed by combining the subject information of different original samples, the difference between the obtained noise samples and the original samples is reduced, thereby not only achieving the goal of significantly increasing the amount of noise data in the training samples, but also making the model less prone to overfitting and improving the model's generalization ability due to the greater difficulty of training the enhanced images, thus achieving the technical effect of improving the accuracy of image recognition using the image recognition model.

[0057] As an optional approach, the method further includes the following steps before acquiring the target image to be identified:

[0058] S1-1, Obtain multiple original samples;

[0059] S1-2, input the training samples from multiple original samples into the initial first recognition model for training once, until the trained second recognition model is obtained;

[0060] S1-3, based on the model parameters of the second recognition model, information is extracted from the first sample among multiple original samples to obtain the image subject information;

[0061] S1-4 combines the image subject information with the second sample from multiple original samples to obtain multiple target samples;

[0062] S1-5, use multiple target samples for secondary training until a well-trained image recognition model is obtained.

[0063] Optionally, in this embodiment, the training phase of the image recognition model may be divided into a primary training phase and a secondary training phase. The primary training is the process of training using the original samples, while the secondary training is the process of continuing to train the image recognition model based on the primary training. The samples used in the secondary training include the original samples and the target samples obtained by combining the image subject information and the second samples from multiple original samples.

[0064] Optionally, in this embodiment, the original samples may be, but are not limited to, divided into first samples, second samples, and training samples. The training samples focus on primary training, while the first and second samples focus on secondary training, in order to improve the relevance of model training and thus improve the training quality of the model. However, in specific scenarios where the number of samples is scarce, the training samples may also include either the first or second samples to increase the number of samples and avoid the problem of poor model training quality due to a small number of samples.

[0065] To further illustrate, optional examples include... Figure 4 As shown, multiple original samples are obtained, such as sample A, sample B, and sample C; the training samples from the multiple original samples are input into the initial first recognition model for training until a trained second recognition model is obtained, such as an image recognition model with a model weight of 1; based on the model parameters of the second recognition model, information is extracted from the first sample (such as sample A) among the multiple original samples to obtain the image subject information; the image subject information and the second sample (such as sample B) among the multiple original samples are combined to obtain multiple target samples (such as sample B*); the multiple target samples are used for secondary training until a trained image recognition model is obtained, such as an image recognition model with a model weight of 2.

[0066] The embodiments provided in this application obtain multiple original samples; the training samples from the multiple original samples are input into an initial first recognition model for training until a trained second recognition model is obtained; based on the model parameters of the second recognition model, information is extracted from the first sample from the multiple original samples to obtain image subject information; the image subject information and the second sample from the multiple original samples are combined to obtain multiple target samples; the multiple target samples are used for secondary training until a trained image recognition model is obtained. This not only achieves the goal of significantly increasing the amount of noisy data in the training samples, but also makes the model less prone to overfitting and improves the model's generalization ability due to the greater difficulty of training the enhanced images, thereby achieving the technical effect of improving the accuracy of image recognition using the image recognition model.

[0067] As an optional approach, based on the model parameters of the second recognition model, information is extracted from the first sample among multiple original samples to obtain the image subject information, including:

[0068] S2-1, using the first model parameters of the second recognition model, feature extraction is performed on the first sample among multiple original samples to obtain multiple image feature points corresponding to the first sample. The first model parameters are the parameters corresponding to the feature processing structure in the second recognition model, and the image feature points are used to represent the image representation of each pixel on the first sample.

[0069] S2-2, using the second model parameters of the second recognition model, multiple image feature points are weighted to obtain the weight values ​​corresponding to each feature point in the multiple image features. The second model parameters are the parameters corresponding to the recognition output layer in the second recognition model. The weight values ​​are used to fuse each feature point in the multiple image feature points to obtain the recognition result output by the second recognition model. The correlation between each feature point in the multiple image feature points and the recognition result is positively correlated with the weight values.

[0070] S2-3, using weight values ​​to filter out multiple target features from each feature, wherein the weight value corresponding to the target feature is greater than or equal to a preset threshold;

[0071] S2-4: Obtain image subject information based on multiple target features.

[0072] Optionally, the image recognition model may, but is not limited to, have a multi-layer structure, such as an input layer, an intermediate layer, and an output layer. In this embodiment, the feature processing structure in the second recognition model may, but is not limited to, be understood as the relevant structure in the intermediate layer that processes features, such as convolutional layers, pooling layers, etc. Through one training, the model parameters of the feature processing structure for feature processing have been determined, and then the feature processing structure can be directly used to extract features from the first sample, thereby improving the feature extraction efficiency.

[0073] Furthermore, in this embodiment, the recognition output layer in the second recognition model can be understood as, but is not limited to, the output layer of a fully connected layer structure. Through one training, the allocation of output weights by the recognition output layer has been determined. Then, the recognition output layer can be directly used to determine the weight values ​​corresponding to each feature point in multiple image features, and multiple target features can be selected from each feature using the weight values, thereby improving the efficiency of target feature selection.

[0074] Through the embodiments provided in this application, the first model parameters of the second recognition model are used to extract features from the first sample among multiple original samples to obtain multiple image feature points corresponding to the first sample. The first model parameters are the parameters corresponding to the feature processing structure in the second recognition model, and the image feature points are used to represent the image representation of each pixel on the first sample. The second model parameters of the second recognition model are used to perform weighted processing on the multiple image feature points to obtain the weight values ​​corresponding to each feature point in the multiple image features. The second model parameters are the parameters corresponding to the recognition output layer in the second recognition model. Multiple target features are selected from each feature using the weight values, where the weight values ​​corresponding to the target features are greater than or equal to a preset threshold. Image subject information is obtained based on multiple target features, thereby achieving the goal of improving the efficiency of feature extraction and feature selection, and thus realizing the technical effect of improving the overall efficiency of image recognition.

[0075] As an optional approach, image subject information is obtained based on multiple target features, including:

[0076] S3-1, Obtain image masks corresponding to multiple target features;

[0077] S3-2, perform intersection processing on the image mask and the first sample to obtain the image part of the first sample that matches the image mask, and determine the image part as the main image of the first sample;

[0078] S3-3, Obtain the information corresponding to the image portion, wherein the image subject information includes the information corresponding to the image portion.

[0079] Optionally, in this embodiment, the image mask can be used, but is not limited to, a selected image, graphic, or object to occlude the image to be processed (fully or partially) to control the area or process of image processing. The specific image or object used for occlusion is called a mask or template in digital image processing. Specifically, the image mask can be used, but is not limited to, to extract the region of interest. A pre-made region of interest mask is multiplied with the image to be processed to obtain the region of interest image. The image values ​​within the region of interest remain unchanged, while the image values ​​outside the region are all 0.

[0080] To illustrate further, alternatively, the region containing the target features can be taken as the region of interest. That is, multiple image masks corresponding to the target features can be obtained, and then the image masks can be used to cover the first sample to obtain the region of interest image (the main image) in the first sample. This ensures that the image values ​​within the region of interest in the first sample remain unchanged, while the image values ​​outside the region are all 0.

[0081] Furthermore, in this embodiment, the image mask can also, but is not limited to, serve a shielding function, using the mask to shield certain areas on the image so that they do not participate in processing or the calculation of processing parameters, or only process or statistically analyze the shielded area; the image mask can also, but is not limited to, be used to extract structural features, using similarity variables or image matching methods to detect and extract structural features in the image that are similar to the mask; the image mask can also, but is not limited to, be used to create images with special shapes.

[0082] It should be noted that by using the method of intersecting the image mask with the first sample, the image subject is identified from the first sample. The image subject identified above can be understood, but is not limited to, the image subject that the image recognition model tends to identify after one training.

[0083] To further illustrate, optional examples include... Figure 5 As shown, the intersection of the image mask 504 and the first sample 502 is processed to obtain the image portion 506 that matches the first sample 502 and the image mask 504, and the image portion 506 is determined as the main image of the first sample.

[0084] The embodiments provided in this application obtain image masks corresponding to multiple target features; the intersection of the image mask and the first sample is processed to obtain the image portion of the first sample that matches the image mask, and the image portion is determined as the image subject of the first sample; information corresponding to the image portion is obtained, wherein the image subject information includes the information corresponding to the image portion, thereby achieving the technical effect of improving the efficiency of obtaining image subject information.

[0085] As an optional approach, the first model parameters of the second recognition model are used to extract features from the first sample among multiple original samples to obtain multiple image feature points corresponding to the first sample. This includes: using the first model parameters to extract features from the first sample among multiple original samples to obtain an initial feature map, wherein the initial feature map is used to represent image features.

[0086] Using the second model parameters of the second recognition model, multiple image feature points are weighted to obtain the weight values ​​corresponding to each feature point in the multiple image features. This includes: using the second model parameters to weight the initial feature map to obtain a feature activation map, wherein the initial feature map contains multiple image feature points and has N channels, where N is a natural number.

[0087] The parameters of the second model are related to the category of the first sample and include the parameters corresponding to the N channels of the recognition output layer in the second recognition model.

[0088] After using the second model parameters to weight the initial feature map and obtain the feature activation map, the method further includes: upsampling the feature activation map according to the image size of the first sample to obtain a feature response map with the same image size as the first sample;

[0089] Multiple target features are selected from various features using weight values, including: thresholding the feature response map using the value of each feature point in the feature activation map to obtain an image mask, wherein the multiple target features include each feature in the image mask.

[0090] Optionally, in this embodiment, it is assumed that the recognition output layer includes multiple fully connected layers, and the weight value of each channel of the last feature map contributing to the recognition result can be obtained through the last fully connected layer. Alternatively, a channel detects a certain feature in the feature map, and the weight value of the channel can be, but is not limited to, seen as a reflection of the degree of correlation between the feature and the recognition result by the recognition output layer.

[0091] Optionally, in this embodiment, a feature map approach is used to improve the efficiency of image mask acquisition, such as... Figure 6 As shown, the specific steps are as follows:

[0092] S602, using the first model parameters, perform feature extraction on the first sample among multiple original samples to obtain an initial feature map, wherein the initial feature map is used to represent image features;

[0093] S604, using the second model parameters, the initial feature map is weighted to obtain a feature activation map, wherein the feature activation map is used to indicate the weight values ​​corresponding to each channel in the initial feature map;

[0094] S606, Upsample the feature activation map according to the image size of the first sample to obtain a feature response map with the same image size as the first sample;

[0095] S608, using the value of each feature point in the feature activation map to perform threshold segmentation on the feature response map to obtain an image mask, wherein multiple target features include each feature in the image mask.

[0096] Through the embodiments provided in this application, features are extracted from a first sample among multiple original samples using first model parameters to obtain an initial feature map, wherein the initial feature map is used to represent image features; the initial feature map is weighted using second model parameters to obtain a feature activation map, wherein the feature activation map is used to indicate the weight values ​​corresponding to each channel in the initial feature map; the feature activation map is upsampled according to the image size of the first sample to obtain a feature response map with the same image size as the first sample; the feature response map is thresholded using the value of each feature point in the feature activation map to obtain an image mask, wherein multiple target features include each feature in the image mask, thereby achieving the technical effect of improving the acquisition efficiency of the image mask.

[0097] As an optional approach, a secondary training process is performed using multiple target samples until a well-trained image recognition model is obtained, including:

[0098] Multiple target samples are input into the second recognition model for training until a trained image recognition model is obtained; or, multiple target samples are input into the first recognition model for training until a trained image recognition model is obtained.

[0099] It should be noted that, in order to improve the diversity of image recognition models, the target samples can be used as the basis for training, but not limited to the recognition model obtained from a single training session; or, the target samples can be used as the basis for training, but not limited to the original recognition model.

[0100] Through the embodiments provided in this application, multiple target samples are input into a second recognition model for training until a trained image recognition model is obtained; or, multiple target samples are input into a first recognition model for training until a trained image recognition model is obtained, thereby achieving the technical effect of improving the diversity of image recognition models.

[0101] As an optional approach, before extracting information from the first sample among multiple original samples based on the model parameters of the second recognition model to obtain the image subject information, the method further includes:

[0102] S4-1, obtain the sample quality label carried by each original sample in multiple original samples, where the sample quality label is used to indicate whether the original sample is a clean sample or a noisy sample;

[0103] S4-2, select at least one clean original sample from multiple original samples according to the sample quality label, and determine the clean original sample as the first sample.

[0104] Optionally, in this embodiment, a clean original sample can be understood, but is not limited to, as an original sample whose sample quality meets the clean sample condition as indicated by the sample quality label. For example, a sample quality label is assigned to samples in the original sample whose noise content is less than or equal to a preset noise threshold to indicate that the sample quality meets the clean sample condition.

[0105] It should be noted that, in order to reduce the difference between noise and clean samples and further increase the difficulty of model training, clean samples will be selected as the material for creating noise, that is, the clean original samples will be determined as the first samples.

[0106] The embodiments provided in this application obtain sample quality labels carried by each of the multiple original samples. The sample quality labels are used to indicate whether the original sample is a clean sample or a noisy sample. At least one clean original sample is selected from the multiple original samples according to the sample quality labels, and the clean original sample is determined as the first sample. This achieves the purpose of reducing the difference between noise and clean samples, thereby realizing the technical effect of increasing the training difficulty of the model.

[0107] As an optional approach, before extracting information from the first sample among multiple original samples based on the model parameters of the second recognition model to obtain the image subject information, the method further includes:

[0108] S5-1, Obtain the sample type label carried by each original sample in the multiple original samples, wherein the sample type label is used to indicate that the image subject in the original sample belongs to any one of at least two subject types;

[0109] S5-2, select at least one target original sample from multiple original samples according to the sample type label, and determine the target original sample as the first sample, wherein the main image of the target original sample belongs to the target type, and the main image of the second sample belongs to the target type.

[0110] It should be noted that, in order to reduce the difference between noise and samples and further increase the difficulty of model training, samples of the same type will be selected as the noise generation material, that is, the target original sample will be determined as the first sample.

[0111] The embodiments provided in this application obtain sample type labels carried by each of multiple original samples, wherein the sample type labels are used to indicate that the image subject in the original sample belongs to any one of at least two subject types; at least one target original sample is selected from multiple original samples according to the sample type labels, and the target original sample is determined as the first sample, wherein the image subject in the target original sample belongs to the target type, and the image subject in the second sample belongs to the target type, thereby achieving the purpose of reducing the difference between noise and samples, thereby achieving the technical effect of increasing the training difficulty of the model.

[0112] As an alternative approach, for ease of understanding, the above image recognition method is applied to industrial defect inspection scenarios. Traditional machine vision-based industrial defect inspection algorithms extract manual features from the input image, including gradient features, texture features, etc. Then, based on the extracted manual features, an SVM classifier (or a tree-based classifier, such as random forest) is trained to perform binary classification to determine whether the current image is a defective image.

[0113] The above approach has two problems: first, the extracted hand-crafted features have poor generalization ability, often resulting in the extraction of harmful features that confuse subsequent classifiers for diverse video data; second, feature extraction and classifier training are performed independently, leading to high training costs. To address these issues, the recently developed Convolutional Neural Networks (CNNs) offer a good solution. CNNs directly feed the original image into a CNN network structure for feature extraction, followed by classification using fully connected layers. Model training is then performed end-to-end using the softmax loss function. This process eliminates the need for manual feature design; the model automatically learns the most suitable features for the current classification task during training, and the entire process is end-to-end, requiring no separate training for feature extraction and classification.

[0114] However, this CNN-based whole-image classification still has some problems. CNN training relies on a large amount of labeled training data. However, in industrial defect inspection scenarios, collecting a large number of defect images is often difficult because factory yields are generally high, resulting in a small number of defect samples. Therefore, under these conditions, training a CNN model with a small amount of data may lead to overfitting, resulting in decreased generalization performance and poor test results.

[0115] To address the aforementioned issues, this embodiment proposes an industrial defect detection algorithm based on weakly supervised mask extraction and robust training. The algorithm extracts the foreground region from the image using a weakly supervised mask extraction method (this foreground region extraction algorithm does not require pixel-level annotation of the foreground region; only image-level annotation information is needed, thus avoiding additional annotation costs). This extracted region is then randomly synthesized with other training images using a data augmentation strategy and subsequently incorporated into the training process. This approach significantly increases the amount of training samples, and because the augmented images are more difficult to train, the model is less prone to overfitting, thereby improving the model's generalization ability.

[0116] Optionally, in this embodiment, industrial defect detection is based on weakly supervised mask extraction and robust training. The initial stage is a conventional training phase, such as... Figure 7 As shown, the input training data is processed by a deep model f(·θ) for feature extraction and outputs a target probability prediction p, followed by loss calculation.

[0117] p = f(x; θ)

[0118] Where θ represents the weight parameters of the deep model, and the loss is calculated based on the predicted probability and the manually labeled results corresponding to the input data:

[0119] L1 = CE(p,y)

[0120] Here, CE() represents the cross-entropy loss function. Gradient descent iterations are then performed using the L1 loss function.

[0121] Update the model parameters. After training, the optimized deep model weights θ are obtained. * .

[0122] Furthermore, after the regular training phase, the weights of the deep model are obtained, and a mask of the weighted image is used. The specific steps are as follows: Figure 8 As shown, feature extraction is performed on the images (input data x) in the training set:

[0123] feat=f(x,θ1)

[0124] Where θ1 represents the fixed parameters before the fully connected layer in the deep model, and feat represents the feature map.

[0125] Then, the feature activation map is obtained.

[0126] The activation map is obtained by weighting the final feature map (the feature map before the last fully connected layer) using the weights of the last fully connected layer of the current category c (e.g., the category "dog" in the illustration), in order to obtain the activation map.

[0127]

[0128] In this scenario, assuming the feature map has k channels and the fully connected layer also contains k channels, the first training iteration will yield weights for each of the k channels across multiple classes. The activation map represents the weights of each channel in the feature map of the last fully connected layer. Here, c represents the class, and (i,j) represents the horizontal and vertical coordinates on the feature map. This represents the weight of the k-th channel of the feature map. Since the size of the activation map (i.e., the feature map size) differs from the size of the original input image, an upsampling operation is used to obtain a response map of the same size as the original image.

[0129] Mask c =UpSample(W c )

[0130] Finally, mask the response graph (activation graph). c Binarization (threshold segmentation) is performed to extract the mask.

[0131] Multiplying the mask by the corresponding input image yields the image subject information corresponding to the mask portion, such as... Figure 8 The foreground portion of the "dog" shown:

[0132] x′=Mask*x

[0133] It should be noted that the above process of obtaining the mask may or may not involve pixel-level annotation information, only image-level category annotation. Therefore, it is a weakly supervised mask extraction method, which can improve the efficiency of mask extraction.

[0134] Further in the robust training phase, after obtaining the foreground mask containing the object's foreground, these mask images are then used to assist in robust training, such as... Figure 9 As shown, for each training image, foreground mask information from other images can be used to augment its data. Specifically, the extracted foreground information (mask-enhanced image) is pasted into the input image. The pasting position and size can be manually specified or completely random. The pasted image and the original input image are then used together to train the deep learning model, outputting probabilities for loss calculation. This can be achieved by either using a model with completely randomly initialized weights for training from scratch, or by using the obtained deep learning model weights θ. * To perform weight fine-tuning.

[0135] In addition, optionally during the testing phase, an image is input, the confidence level of the target probability is output, and the output category is obtained based on the confidence level of each category.

[0136] The embodiments provided in this application, targeting industrial defect detection tasks, can accurately detect the degree of defects in input images. By designing an industrial defect detection algorithm based on weakly supervised mask extraction and robust training, the foreground region in the image is extracted using a weakly supervised mask extraction algorithm and then randomly synthesized with other training images using a data augmentation strategy before being incorporated into subsequent training. This approach significantly increases the amount of training samples, greatly alleviating the reliance on a large number of labeled training samples during deep model training, a problem often difficult to meet in industrial scenarios. This allows the trained CNN model to obtain more accurate and robust prediction results, thus providing reliable technical support for industrial AI defect quality detection.

[0137] It is understood that in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0138] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0139] According to another aspect of the embodiments of this application, an image recognition apparatus for implementing the above-described image recognition method is also provided. For example... Figure 10 As shown, the device includes:

[0140] The first acquisition unit 1002 is used to acquire the target image to be identified;

[0141] The input unit 1004 is used to input the target image into the image recognition model. The image recognition model is a neural network model for recognizing the main body of the image, which is trained using multiple original samples and multiple target samples. The target sample is a noise sample obtained by combining the image main body information corresponding to the first sample and the second sample. The multiple original samples include the first sample and the second sample. The image main body information is the information corresponding to the image main body in the first sample.

[0142] The second acquisition unit 1006 is used to acquire the image recognition result output by the image recognition model, wherein the image recognition result is used to indicate the subject type to which the image subject in the target image belongs.

[0143] For specific embodiments, please refer to the examples shown in the image recognition device above, which will not be repeated here.

[0144] As an optional solution, the device also includes:

[0145] The third acquisition unit is used to acquire multiple raw samples before acquiring the target image to be identified;

[0146] The first training unit is used to input the training samples from multiple original samples into the initial first recognition model for training once before acquiring the target image to be recognized, until the trained second recognition model is obtained.

[0147] The extraction unit is used to extract information from the first sample among multiple original samples based on the model parameters of the second recognition model before acquiring the target image to be identified, so as to obtain the image subject information.

[0148] The combining unit is used to combine the image subject information and the second sample from multiple original samples to obtain multiple target samples before acquiring the target image to be identified.

[0149] The second training unit is used to perform secondary training using multiple target samples before acquiring the target image to be recognized, until a trained image recognition model is obtained.

[0150] For specific implementation examples, please refer to the examples shown in the image recognition method above. These examples will not be repeated here.

[0151] As an optional solution, the extraction unit includes:

[0152] The extraction module is used to extract features from the first sample among multiple original samples using the first model parameters of the second recognition model, and obtain multiple image feature points corresponding to the first sample. The first model parameters are the parameters corresponding to the feature processing structure in the second recognition model, and the image feature points are used to represent the image representation of each pixel on the first sample.

[0153] The weighting module is used to perform weighting processing on multiple image feature points using the second model parameters of the second recognition model, so as to obtain the weight value corresponding to each feature point in the multiple image features. The second model parameters are the parameters corresponding to the recognition output layer in the second recognition model.

[0154] The filtering module is used to filter out multiple target features from various features using weight values, wherein the weight value corresponding to the target feature is greater than or equal to a preset threshold.

[0155] The acquisition module is used to acquire image subject information based on multiple target features.

[0156] For specific implementation examples, please refer to the examples shown in the image recognition method above. These examples will not be repeated here.

[0157] As an optional solution, the acquisition module includes:

[0158] The first acquisition submodule is used to acquire image masks corresponding to multiple target features;

[0159] The processing submodule is used to perform intersection processing on the image mask and the first sample to obtain the image part of the first sample that matches the image mask, and to determine the image part as the main image of the first sample.

[0160] The second acquisition submodule is used to acquire information corresponding to the image portion, wherein the image subject information includes information corresponding to the image portion.

[0161] For specific implementation examples, please refer to the examples shown in the image recognition method above. These examples will not be repeated here.

[0162] As an optional approach, the extraction module includes: an extraction submodule, used to extract features from a first sample among multiple original samples using first model parameters to obtain an initial feature map, wherein the initial feature map is used to represent image features;

[0163] The weighting module includes: a weighting submodule, used to perform weighting processing on the initial feature map using the second model parameters to obtain a feature activation map, wherein the feature activation map is used to indicate the weight values ​​corresponding to each channel in the initial feature map;

[0164] The device also includes a sampling submodule, which is used to perform weighted processing on the initial feature map using the second model parameters to obtain a feature activation map, and then upsample the feature activation map according to the image size of the first sample to obtain a feature response map with the same image size as the first sample.

[0165] The filtering module includes a segmentation submodule, which is used to perform threshold segmentation on the feature response map using the value of each feature point of the feature activation map to obtain an image mask, wherein multiple target features include each feature in the image mask.

[0166] For specific implementation examples, please refer to the examples shown in the image recognition method above. These examples will not be repeated here.

[0167] As an optional approach, the second training unit includes:

[0168] The first training module is used to input multiple target samples into the second recognition model for training until a well-trained image recognition model is obtained; or,

[0169] The second training module is used to input multiple target samples into the first recognition model for training until a trained image recognition model is obtained.

[0170] For specific implementation examples, please refer to the examples shown in the image recognition method above. These examples will not be repeated here.

[0171] As an optional solution, the device also includes:

[0172] The fourth acquisition unit is used to acquire the sample quality label carried by each original sample in the multiple original samples before extracting information from the first sample in the multiple original samples based on the model parameters of the second recognition model to obtain the image subject information. The sample quality label is used to indicate whether the original sample belongs to a clean sample or a noisy sample.

[0173] The first screening unit is used to extract information from the first sample among multiple original samples based on the model parameters of the second recognition model to obtain the image subject information, and to screen at least one clean original sample from multiple original samples according to the sample quality label, and to determine the clean original sample as the first sample.

[0174] For specific implementation examples, please refer to the examples shown in the image recognition method above. These examples will not be repeated here.

[0175] As an optional solution, the device also includes:

[0176] The fifth acquisition unit is used to acquire the sample type label carried by each original sample in the multiple original samples before extracting information from the first sample in the multiple original samples based on the model parameters of the second recognition model to obtain the image subject information. The sample type label is used to indicate that the image subject in the original sample belongs to any one of at least two subject types.

[0177] The second filtering unit is used to filter at least one target original sample from multiple original samples according to the sample type label before extracting information from the first sample among multiple original samples based on the model parameters of the second recognition model to obtain the image subject information, and to determine the target original sample as the first sample. The image subject in the target original sample belongs to the target type, and the image subject in the second sample also belongs to the target type.

[0178] For specific implementation examples, please refer to the examples shown in the image recognition method above. These examples will not be repeated here.

[0179] According to another aspect of the embodiments of this application, an electronic device for implementing the above-described image recognition method is also provided. This electronic device may, but is not limited to, […]. Figure 1The user equipment 102 or server 112 shown in the figure, in this embodiment, is taken as an example of an electronic device, namely user equipment 102. Further, as shown in the figure... Figure 11 As shown, the electronic device includes a memory 1102 and a processor 1104. The memory 1102 stores a computer program, and the processor 1104 is configured to execute the steps of any of the above method embodiments via the computer program.

[0180] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0181] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0182] S1, acquire the target image to be identified;

[0183] S2, input the target image into the image recognition model, wherein the image recognition model is a neural network model for recognizing the main body of the image, which is trained using multiple original samples and multiple target samples. The target sample is a noise sample obtained by combining the image main body information corresponding to the first sample and the second sample. The multiple original samples include the first sample and the second sample. The image main body information is the information corresponding to the image main body in the first sample.

[0184] S3, obtain the image recognition result output by the image recognition model, where the image recognition result is used to indicate the subject type to which the image subject in the target image belongs.

[0185] Alternatively, as those skilled in the art will understand, Figure 11 The structure shown is for illustrative purposes only. Figure 11 This does not limit the structure of the aforementioned electronic devices. For example, the electronic device may also include components that are more... Figure 11 The more or fewer components shown (such as network interfaces, etc.), or having the same Figure 11 The different configurations shown.

[0186] The memory 1102 can be used to store software programs and modules, such as the program instructions / modules corresponding to the image recognition method and apparatus in this embodiment. The processor 1104 executes various functional applications and data processing by running the software programs and modules stored in the memory 1102, thereby realizing the image recognition method described above. The memory 1102 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1102 may further include memory remotely located relative to the processor 1104, and these remote memories can be connected to user equipment via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 1102 may be used, but is not limited to, to store information such as target images, image recognition models, and image recognition results. As an example, such as... Figure 11 As shown, the memory 1102 may include, but is not limited to, the first acquisition unit 1002, the input unit 1004, and the second acquisition unit 1006 in the image recognition device. Furthermore, it may include, but is not limited to, other module units in the image recognition device, which will not be elaborated upon in this example.

[0187] Optionally, the transmission device 1106 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 1106 includes a Network Interface Controller (NIC), which can be connected to other network devices and a router via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 1106 is a radio frequency (RF) module, used for wireless communication with the Internet.

[0188] In addition, the above-mentioned electronic device also includes: a display 1108 for displaying the target image, image recognition model and image recognition results, etc.; and a connection bus 1110 for connecting the various module components in the above-mentioned electronic device.

[0189] In other embodiments, the aforementioned user equipment or server can be a node in a distributed system, wherein the distributed system can be a blockchain system, which is a distributed system formed by connecting multiple nodes through network communication. The nodes can form a peer-to-peer (P2P) network, and any form of computing device, such as a server, user equipment, or other electronic device, can become a node in the blockchain system by joining this peer-to-peer network.

[0190] According to one aspect of this application, a computer program product is provided, comprising a computer program containing program code for performing the methods shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit, it performs various functions provided in the embodiments of this application.

[0191] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0192] It should be noted that the computer system of the electronic device is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0193] A computer system includes a Central Processing Unit (CPU), which performs various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) or loaded from RAM. ROM also stores various programs and data required for system operation. The CPU, ROM, and RAM are interconnected via a bus. Input / output interfaces (I / O interfaces) are also connected to the bus.

[0194] The following components are connected to the input / output interface: input sections including keyboards, mice, etc.; output sections including cathode ray tubes (CRTs), liquid crystal displays (LCDs), and speakers; storage sections including hard drives; and communication sections including network interface cards such as LAN cards and modems. The communication section performs communication processing via a network such as the Internet. Drives are also connected to the input / output interface as needed. Removable media, such as disks, optical discs, magneto-optical discs, semiconductor memories, etc., are installed on the drive as needed so that computer programs read from them can be installed into the storage section as required.

[0195] Specifically, according to embodiments of this application, the processes described in the various method flowcharts can be implemented as computer programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit, it performs various functions defined in the system of this application.

[0196] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0197] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more electronic devices (which may be user equipment, servers, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0198] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0199] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0200] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0201] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0202] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An image recognition method, characterized in that, include: Acquire the target image to be identified; The target image is input into an image recognition model, wherein the image recognition model is a neural network model trained using multiple original samples and multiple target samples for recognizing the main body of an image. The target sample is a noise sample obtained by combining the image main body information corresponding to the first sample with the second sample. The multiple original samples include the first sample and the second sample. The image main body information is the information corresponding to the image main body in the first sample. The image main body information corresponding to the first sample is obtained by: using a second recognition model to obtain the weight values ​​corresponding to each feature point among multiple image feature points corresponding to the first sample, wherein the second recognition model is obtained by training the initial first recognition model once, and the image recognition model is obtained by training the second recognition model a second time. The weight values ​​are used to fuse each feature point among the multiple image feature points to obtain the recognition result of the second recognition model for the first sample. Multiple target feature points are selected from the multiple feature points using the weight values, wherein the weight values ​​corresponding to the target feature points are greater than or equal to a preset threshold. The image main body information is obtained based on the multiple target feature points. Obtain the image recognition result output by the image recognition model, wherein the image recognition result is used to indicate the subject type to which the image subject in the target image belongs.

2. The method according to claim 1, characterized in that, Prior to acquiring the target image to be identified, the method further includes: Obtain the multiple original samples; The training samples from the plurality of original samples are input into the initial first recognition model for training once, until the trained second recognition model is obtained; Based on the model parameters of the second recognition model, information is extracted from the first sample among the multiple original samples to obtain the image subject information; The image subject information and the second sample from the plurality of original samples are combined to obtain the plurality of target samples; The image recognition model is trained by using the multiple target samples until it is well trained.

3. The method according to claim 2, characterized in that, In the process of extracting information from the first sample among the plurality of original samples based on the model parameters of the second recognition model to obtain the image subject information, the method further includes: Using the first model parameters of the second recognition model, feature extraction is performed on the first sample among the plurality of original samples to obtain the plurality of image feature points, wherein the first model parameters are the parameters corresponding to the feature processing structure in the second recognition model, and the image feature points are used to represent the image representation of each pixel on the first sample; Using the second model parameters of the second recognition model, the plurality of image feature points are weighted to obtain the weight values. The second model parameters are the parameters corresponding to the recognition output layer in the second recognition model. The correlation between each feature point in the plurality of image feature points and the recognition result is positively correlated with the weight values.

4. The method according to claim 3, characterized in that, The step of obtaining the image subject information based on the plurality of target feature points includes: Obtain the image mask corresponding to the plurality of target feature points; The image mask and the first sample are intersected to obtain the image portion of the first sample that matches the image mask, and the image portion is determined as the main image of the first sample. Obtain information corresponding to the image portion, wherein the image subject information includes information corresponding to the image portion.

5. The method according to claim 3, characterized in that, The step of using the first model parameters of the second recognition model to extract features from the first sample among the plurality of original samples to obtain a plurality of image feature points corresponding to the first sample includes: using the first model parameters to extract features from the first sample among the plurality of original samples to obtain an initial feature map, wherein the initial feature map contains a plurality of image feature points and has N channels, where N is a natural number; The second model parameters are related to the category of the first sample and include the parameters corresponding to the N channels of the recognition output layer in the second recognition model; The step of using the second model parameters of the second recognition model to perform weighted processing on the plurality of image feature points to obtain the weight value corresponding to each feature point in the plurality of image feature points includes: using the parameters corresponding to the N channels of the recognition output layer to perform weighted processing on the initial feature map to obtain a feature activation map, wherein the value of each feature in the feature activation map is obtained by weighting the value of each feature in the initial feature map on the N channels according to the parameters corresponding to the N channels of the recognition output layer; After using the second model parameters to weight the initial feature map to obtain a feature activation map, the method further includes: upsampling the feature activation map according to the image size of the first sample to obtain a feature response map with the same image size as the first sample; The step of selecting multiple target feature points from the various feature points using the weight values ​​includes: performing threshold segmentation on the feature response map using the value of each feature of the feature activation map to obtain an image mask, wherein the multiple target feature points include each feature point in the image mask.

6. The method according to claim 2, characterized in that, The step of performing secondary training using the multiple target samples until a trained image recognition model is obtained includes: The multiple target samples are input into the second recognition model for training until a trained image recognition model is obtained; or, The multiple target samples are input into the first recognition model for training until the trained image recognition model is obtained.

7. The method according to any one of claims 2 to 6, characterized in that, Before extracting information from the first sample among the plurality of original samples based on the model parameters of the second recognition model to obtain the image subject information, the method further includes: Obtain the sample quality label carried by each of the plurality of original samples, wherein the sample quality label is used to indicate whether the original sample is a clean sample or a noisy sample; At least one clean original sample is selected from the plurality of original samples according to the sample quality label, and the clean original sample is identified as the first sample.

8. The method according to any one of claims 2 to 6, characterized in that, Before extracting information from the first sample among the plurality of original samples based on the model parameters of the second recognition model to obtain the image subject information, the method further includes: Obtain the sample type label carried by each of the plurality of original samples, wherein the sample type label is used to indicate that the image subject in the original sample belongs to any one of at least two subject types; At least one target original sample is selected from the plurality of original samples according to the sample type label, and the target original sample is determined as the first sample, wherein the image subject in the target original sample belongs to the target type, and the image subject in the second sample belongs to the target type.

9. An image recognition device, characterized in that, include: The first acquisition unit is used to acquire the target image to be identified; An input unit is used to input the target image into an image recognition model, wherein the image recognition model is a neural network model trained using multiple original samples and multiple target samples for recognizing the main body of an image. The target sample is a noise sample obtained by combining the image main body information corresponding to a first sample with a second sample. The multiple original samples include the first sample and the second sample. The image main body information is the information corresponding to the image main body in the first sample. The image main body information corresponding to the first sample is obtained by: using a second recognition model to obtain the weight values ​​corresponding to each feature point among multiple image feature points corresponding to the first sample, wherein the second recognition model is obtained by training the initial first recognition model once, and the image recognition model is obtained by training the second recognition model a second time. The weight values ​​are used to fuse each feature point among the multiple image feature points to obtain the recognition result of the second recognition model for the first sample. Multiple target feature points are selected from the multiple feature points using the weight values, wherein the weight values ​​corresponding to the target feature points are greater than or equal to a preset threshold. The image main body information is obtained based on the multiple target feature points. The second acquisition unit is used to acquire the image recognition result output by the image recognition model, wherein the image recognition result is used to indicate the subject type to which the image subject in the target image belongs.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a computer program, wherein the computer program is executed by an electronic device to perform the method according to any one of claims 1 to 8.

11. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1 to 8.

12. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method described in any one of claims 1 to 8 through the computer program.

Images