Image recognition method and device

Abstract

The present invention discloses an image recognition method and device, and relates to the field of computer technology. A specific implementation of the method includes: receiving an unlabeled image data set and a labeled image data set; using the labeled image data set to train a preset first multi-label classification network to obtain a trained first multi-label classification network; passing the unlabeled image data set through the trained first multi-label classification network to obtain a labeling result and a corresponding confidence value for each image; updating a preset second multi-label classification network according to the labeling result and the corresponding confidence value, and then labeling the image through the updated second multi-label classification network. This implementation can solve the problem that the prior art cannot process images quickly and effectively.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an image recognition method and device. Background Art

[0002] With the continuous development of science and technology, multimedia devices are appearing more and more in our lives. The multimedia experience they bring us is becoming more and more abundant, and the diverse image presentation forms have added more color and fun to our lives.

[0003] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:

[0004] However, multimedia data such as images has shown explosive growth. How to effectively manage massive image data has become a challenge. At the same time, how to quickly and effectively process image classification is also an urgent problem to be solved. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides an image recognition method and device, which can solve the problem that images cannot be processed quickly and effectively in the prior art.

[0006] To achieve the above-mentioned purpose, according to one aspect of an embodiment of the present invention, there is provided an image recognition method, comprising receiving an unlabeled image data set and a labeled image data set; using the labeled image data set to train a preset first multi-label classification network to obtain a trained first multi-label classification network; passing the unlabeled image data set through the trained first multi-label classification network to obtain a labeling result and a corresponding confidence value for each image; updating a preset second multi-label classification network according to the labeling result and the corresponding confidence value, and then labeling the image through the updated second multi-label classification network.

[0007] Optionally, using the labeled image data set to train a preset first multi-label classification network to obtain a trained first multi-label classification network includes:

[0008] The marked image is segmented into the marked area to obtain a set of candidate areas;

[0009] Inputting the candidate region into a preset first multi-label classification network for training to obtain a first multi-label classification network to be trained;

[0010] Based on the labeled image data set, a first multi-label classification network is trained using a weighted cross entropy loss function to obtain a trained first multi-label classification network.

[0011] Optionally, after obtaining the labeling result and the corresponding confidence value of each image, the following steps are included:

[0012] For the labeling results and the corresponding confidence values, the average score vector of the labeling results is calculated by Monte Carlo method;

[0013] By calculating the entropy of the average score vector, the overall confidence value for each tag is obtained.

[0014] Optionally, updating a preset second multi-label classification network according to the labeling result and the corresponding confidence value includes:

[0015] Convert the overall confidence value to the standardized posterior confidence to obtain the loss value of the unlabeled image;

[0016] Calculate the sum of the labeled image loss value and the unlabeled image loss value to update the preset second multi-label classification network according to the labeling result and the sum of the loss values; wherein the labeled image loss value is the classification cross entropy calculated based on the labeled image data set

[0017] Optionally, it also includes:

[0018] The first multi-label classification network is an 8-layer network model based on AlexNet, and the second multi-label classification network is a 5-layer convolution plus 3-layer fully connected model based on AlexNet.

[0019] In addition, according to one aspect of an embodiment of the present invention, an image recognition device is provided, including a receiving module for receiving an unlabeled image data set and a labeled image data set; a recognition module for training a preset first multi-label classification network using the labeled image data set to obtain a trained first multi-label classification network; the unlabeled image data set is passed through the trained first multi-label classification network to obtain a labeling result and a corresponding confidence value for each image; according to the labeling result and the corresponding confidence value, a preset second multi-label classification network is updated, and the image is labeled by the updated second multi-label classification network.

[0020] Optionally, the recognition module trains a preset first multi-label classification network using a labeled image data set to obtain a trained first multi-label classification network, including:

[0021] The marked image is segmented into the marked area to obtain a set of candidate areas;

[0022] Inputting the candidate region into a preset first multi-label classification network for training to obtain a first multi-label classification network to be trained;

[0023] Based on the labeled image data set, a first multi-label classification network is trained using a weighted cross entropy loss function to obtain a trained first multi-label classification network.

[0024] Optionally, after the recognition module obtains the labeling result and the corresponding confidence value of each image, it includes:

[0025] For the labeling results and the corresponding confidence values, the average score vector of the labeling results is calculated by Monte Carlo method;

[0026] By calculating the entropy of the average score vector, the overall confidence value for each tag is obtained.

[0027] According to another aspect of an embodiment of the present invention, there is further provided an electronic device, including:

[0028] one or more processors;

[0029] a storage device for storing one or more programs,

[0030] When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any of the above-mentioned image recognition embodiments.

[0031] According to another aspect of the embodiments of the present invention, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, any of the methods described above based on the image recognition embodiments is implemented.

[0032] One embodiment of the above invention has the following advantages or beneficial effects: the present invention receives an unlabeled image data set and a labeled image data set; uses the labeled image data set to train a preset first multi-label classification network to obtain a trained first multi-label classification network; passes the unlabeled image data set through the trained first multi-label classification network to obtain a labeling result and a corresponding confidence value for each image; updates the preset second multi-label classification network according to the labeling result and the corresponding confidence value, and then labels the image through the updated second multi-label classification network. Therefore, the image recognition method of the present invention can quickly and effectively recognize images, and at the same time, can increase a large amount of actually available unlabeled data within the scope of limited labeled data, so as to greatly improve the performance of image recognition.

[0033] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0035] Figure 1 is a schematic diagram of the main process of the image recognition method according to the first embodiment of the present invention;

[0036] Figure 2 is a schematic diagram of the main process of the image recognition method according to the second embodiment of the present invention;

[0037] Figure 3 is a schematic diagram of main modules of an image recognition device according to an embodiment of the present invention;

[0038] Figure 4 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0039] Figure 5 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0041] Figure 1 is a schematic diagram of the main process of an image recognition method according to an embodiment of the present invention. The image recognition method may include:

[0042] Step S101: receiving an unlabeled image dataset and a labeled image dataset.

[0043] Step S102: training a preset first multi-label classification network using the labeled image data set to obtain a trained first multi-label classification network.

[0044] Preferably, when training the first multi-label classification network, the labeled image is first segmented in the labeled area to obtain a set of candidate areas. Then, the candidate areas are input into a preset first multi-label classification network for training to obtain a first multi-label classification network to be trained. Finally, based on the labeled image data set, the first multi-label classification network is trained using a weighted cross entropy loss function to obtain a trained first multi-label classification network.

[0045] Step S103: The unlabeled image data set is passed through the trained first multi-label classification network to obtain a labeling result and a corresponding confidence value for each image.

[0046] Preferably, after obtaining the labeling result and the corresponding confidence value of each image, the labeling result and the corresponding confidence value can be calculated by Monte Carlo method to obtain the average score vector of the labeling result. Then, by calculating the entropy of the average score vector, the overall confidence value of each label is obtained.

[0047] Step S104: updating a preset second multi-label classification network according to the labeling result and the corresponding confidence value, and then labeling the image through the updated second multi-label classification network.

[0048] Preferably, the overall confidence value is converted into a standardized posterior confidence value, thereby obtaining a loss value of the unlabeled image. Then, the sum of the labeled image loss value and the unlabeled image loss value is calculated to train a preset second multi-label classification network according to the labeling result and the sum of the loss values ​​to update the parameters of the second multi-label classification network. The labeled image loss value is a classification cross entropy calculated based on the labeled image data set.

[0049] It is worth noting that the first multi-label classification network is an 8-layer network model based on AlexNet, and the second multi-label classification network is a 5-layer convolution plus 3-layer fully connected model based on AlexNet. Among them, AlexNet is a neural network that successfully applies tricks such as ReLU, Dropout and LRN in CNN, and uses GPU for computing acceleration.

[0050] According to the various embodiments described above, the image recognition method described in the present invention can continuously convert the to-be-labeled image samples with higher prediction credibility into training samples through the dual model mode of the first multi-label classification network and the second multi-label classification network (teacher-student dual model mode) in the case of fewer complete labeled images, and continuously train. In each iteration process, the first multi-label classification network (teacher multi-label classification network) can screen out more available to-be-labeled images (with higher confidence values) as training samples for the second multi-label classification network (student multi-label classification network), thereby expanding the number of training sample sets of the second multi-label classification network to a certain extent, and the second multi-label classification network finally obtained can be used for the prediction output of the results.

[0051] In addition, the image recognition method of the present invention is applicable to various forms of multi-label image labeling problems, and is particularly applicable to solving image labeling problems in the industrial sector. That is, within the scope of limited labeled data, a large amount of actually available unlabeled data is increased to improve performance.

[0052] Figure 2: is a schematic diagram of the main process of an image recognition method according to a reference embodiment of the present invention, and the image recognition method may include:

[0053] Step S201: receiving an unlabeled image dataset and a labeled image dataset.

[0054] In the embodiment, it is assumed that a large number of unlabeled images D are received. u =(x i ) and a small number of high-quality labeled images D l ={(x i ,y i )}, where x i is the image, y i is the corresponding tag set. At the same time, the first multi-label classification network and the second multi-label classification network are preset, for example: the teacher multi-label classification network F T (x) and the student multi-label classification network F S (x). The first multi-label classification network (teacher multi-label classification network) based on deep learning is trained using the labeled data set. The trained first multi-label classification network (teacher multi-label classification network) is then applied to each image in the unlabeled data set to obtain the labeling result and the corresponding confidence value, thereby generating a "pseudo" labeled sample. The confidence value represents the confidence value of the reliability of the label output by the first multi-label classification network (teacher multi-label classification network), and is used to guide the second multi-label classification network (student multi-label classification network) to perform learning training.

[0055] Step S202 , segmenting the marked area of ​​the marked image to obtain a set of candidate areas.

[0056] In the embodiment, the candidate regions of the image are first extracted based on BING to obtain a set of candidate object mark regions. The regions where the marks involved in the image are located are segmented out. Here, for the case where there are multiple marked entities in the multi-marked image, the BING model is selected for segmentation, so that the number of regions obtained by segmentation is as small as possible while covering all the marks. BING is a Binarized normed gradients model, which is used to extract candidate regions.

[0057] Step S203: input the candidate region into a preset first multi-label classification network for training to obtain a first multi-label classification network to be trained.

[0058] In an embodiment, these extracted candidate regions are input into a first multi-label classification network (teacher multi-label classification network) F with shared parameters. T(x) is used for training and modeling. Since there is no large amount of multi-label data for training the model, the present invention can use the currently available large-scale single-label data set as the training sample set of the model, and then use a small amount of complete and correct sample data related to the business scenario to fine-tune the parameters of the trained model. Among them, the commonly used single-label data can also be called multi-classification data, that is, the entire data set has many categories, but a picture or a sample has only one category. Commonly used large-scale multi-classification data include Cifa, ImageNet, etc.

[0059] Step S204: Based on the labeled image data set, a first multi-label classification network is trained using a weighted cross entropy loss function to obtain a trained first multi-label classification network.

[0060] Among them, the weighted cross entropy loss function is the cross entropy after softmax multiplied by the weight.

[0061] In the embodiment, after obtaining a preliminary first multi-label classification network (teacher multi-label classification network) F T After the (x) parameters are set, the model parameters are fine-tuned using multi-labeled data. Preferably, the labeled sample set D is trained using class-weighted classification cross entropy loss. l F T (x).

[0062] Step S205 , the unlabeled image data set is passed through the trained first multi-label classification network to obtain a labeling result and a corresponding confidence value for each image.

[0063] Step S206: For the labeling results and the corresponding confidence values, the average score vector of the labeling results is calculated by using the Monte Carlo method.

[0064] The Monte Carlo method is a type of numerical calculation method guided by probability statistics theory.

[0065] In the embodiment, the quantization processing of the marking results and the corresponding confidence values ​​is performed by averaging K samples through the Monte Carlo method to obtain the average score vector of the marking effect:

[0066]

[0067] Step S207, obtaining the overall confidence value of each mark by calculating the entropy of the average score vector.

[0068] In an embodiment, the average score vector includes the probability scores for each labeled class, i.e. By calculating the entropy of the average probability vector, we can get the overall tag confidence value for each tag:

[0069]

[0070] Step S208, converting the overall confidence value into a standardized posterior confidence value, thereby obtaining a loss value of the unlabeled image.

[0071] In the embodiment, F T (x) applied to unlabeled images x u ∈U, use equation 1 to get the average score vector (also called pseudo-label), and use equation 2 to get the corresponding overall label confidence value u. A higher value in the overall confidence value (i.e., the confidence value map) indicates that the generated pseudo-label may be incorrect and needs to be updated in F. S (x) when the weight is reduced.

[0072] Preferably, the overall confidence value u is converted into a standardized posterior confidence map as follows:

[0073] ω=exp[-αu] (3)

[0074] where α is a positive hyperparameter scalar and the confidence map ω∈[0,1] provides the confidence map given by F T (x) produces a pseudo-labeled confidence score, so that a higher confidence value produces a low confidence score, and vice versa. And ω∈[0,1] can represent the loss of the unlabeled image, denoted by L unlab .

[0075] Step S209, calculating the sum of the loss value of the labeled image and the loss value of the unlabeled image, and performing back propagation to update the preset second multi-label classification network according to the sum of the loss values, and then labeling the image through the updated second multi-label classification network.

[0076] In the embodiment, the semi-supervised loss is the sum of the labeled loss and the unlabeled loss:

[0077] L semi_sup =L lab +L unlab (4)

[0078] Among them, L lab It is calculated based on the labeled samples through the cross entropy loss function, with a weight of 1.

[0079] It can be seen that the proposed semi-supervised loss function encourages discarding T (x) produces inaccurate pseudo-labels. The hyperparameter α in Equation 3 controls the T (x) to F S(x) information flow. Smaller values ​​of α allow the second multi-label classification network to blindly follow the first multi-label classification network, while larger values ​​of α control learning by emphasizing the confidence of the first multi-label classification network. For example, setting α = 0 is equivalent to using all pseudo-labels, while setting α> 0 allows the probability of selecting more certain pseudo-labels.

[0080] In a preferred embodiment, the input unlabeled data set is set and labeled datasets First, use D l Training T (x). Then, the following process is repeated until the verification loss converges to obtain the updated F S (x). Further, the iterative process includes:

[0081] Step 1: From D u Small and medium sample sampling From D l Small and medium sample sampling

[0082] Step 2: Target The output pseudo-label z and the overall confidence value u are calculated according to Equations 1 and 2.

[0083] Step 3: According to Equation 3, the standardized posterior confidence map is calculated by the overall confidence value u.

[0084] Step 4: According to Equation 4, calculate the sum of the labeled and unlabeled losses.

[0085] Step 5: Update F using the backpropagation method S (x) parameter.

[0086] Among them, the back propagation method is a neural network learning algorithm.

[0087] In summary, the image recognition method described in the present invention can utilize a large amount of unlabeled noise data and improve the labeling of multi-label images in combination with uncertainty. Uncertainty is introduced in the first multi-label classification network (teacher multi-label classification network), so that the first multi-label classification network (teacher multi-label classification network) can not only classify unlabeled data, but also give uncertainty. Using uncertainty, the speed of model training and the effect of model training can be improved, and it can be directly applied to industrial environments. Because the industrial environment has such a scene: a large amount of unlabeled data, and there will be certain problems. Because there are a large amount of relevant or irrelevant unlabeled data, the first multi-label classification network (teacher multi-label classification network) can be trained based on a small amount of accurate data. In other words, the first multi-label classification network (teacher multi-label classification network) not only outputs the result, but also outputs the uncertainty of this result.

[0088] When the second multi-label classification network (student multi-label classification network) is trained, samples with low confidence values ​​(uncertainty) will have large weights, and samples with high confidence values ​​will have small weights. This can improve the accuracy of the second multi-label classification network (student multi-label classification network) and is more practical. That is, it pays more attention to samples with low confidence values ​​and does not pay too much attention to samples with high confidence values, thereby improving the robustness of the second multi-label classification network (student multi-label classification network).

[0089] In addition, in view of the large differences in data scenarios and data types in industrial environments, a public single-label large-scale dataset is used to pre-train the first multi-label classification network (teacher multi-label classification network), and then a small amount of multi-label scenario data is used to fine-tune the parameters of the first multi-label classification network (teacher multi-label classification network), so that it has better generalization ability. Therefore, the proposed uncertainty induction method can effectively transfer information from the first multi-label classification network to the second multi-label classification network for labeling tasks, and can generate expert-level labels using a limited number of labeled samples.

[0090] Figure 3 is an image recognition device according to an embodiment of the present invention, such as Figure 3As shown, the image recognition device includes a receiving module 301 and a recognition module 302. The receiving module 301 receives an unlabeled image data set and a labeled image data set. Then, the recognition module 302 uses the labeled image data set to train a preset first multi-label classification network to obtain a trained first multi-label classification network; the unlabeled image data set is passed through the trained first multi-label classification network to obtain a labeling result and a corresponding confidence value for each image; according to the labeling result and the corresponding confidence value, the preset second multi-label classification network is updated, and the image is labeled through the updated second multi-label classification network.

[0091] Preferably, the recognition module 302 uses the labeled image data set to train the preset first multi-label classification network, and when the trained first multi-label classification network is obtained, the labeled image can be segmented in the labeled area to obtain a candidate area set. The candidate area is input into the preset first multi-label classification network for training to obtain the first multi-label classification network to be trained. Based on the labeled image data set, the first multi-label classification network is trained using a weighted cross entropy loss function to obtain a trained first multi-label classification network.

[0092] In addition, preferably, after obtaining the labeling result and the corresponding confidence value of each image, the recognition module 302 can calculate the average score vector of the labeling result by Monte Carlo method. By calculating the entropy of the average score vector, the overall confidence value of each label is obtained.

[0093] Further, the recognition module 302 converts the overall confidence value into a standardized posterior confidence value, thereby obtaining a loss value of the unlabeled image. The sum of the labeled image loss value and the unlabeled image loss value is calculated to update the preset second multi-label classification network according to the labeling result and the sum of the loss values. The labeled image loss value is a classification cross entropy calculated based on the labeled image data set.

[0094] It is also worth mentioning that the first multi-label classification network is an 8-layer network model based on AlexNet, and the second multi-label classification network is a 5-layer convolution plus 3-layer fully connected model based on AlexNet.

[0095] It should be noted that the specific implementation content of the image recognition device of the present invention has been described in detail in the above-mentioned image recognition method, so the repeated content will not be described again here.

[0096] Figure 4 An exemplary system architecture 400 to which the image recognition method or image recognition device according to the embodiment of the present invention can be applied is shown.

[0097] like Figure 4 As shown, system architecture 400 may include terminal devices 401, 402, 403, network 404 and server 405. Network 404 is used to provide a medium for communication links between terminal devices 401, 402, 403 and server 405. Network 404 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0098] Users can use terminal devices 401, 402, 403 to interact with server 405 through network 404 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 401, 402, 403, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0099] The terminal devices 401 , 402 , and 403 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0100] The server 405 may be a server that provides various services, such as a backend management server (only an example) that provides support for shopping websites browsed by users using the terminal devices 401, 402, and 403. The backend management server may analyze and process the received data such as product information query requests, and feed back the processing results (such as target push information, product information - only an example) to the terminal device.

[0101] It should be noted that the image recognition method provided in the embodiment of the present invention is generally executed by the server 405 , and accordingly, the image recognition device is generally disposed in the server 405 .

[0102] It should be understood that Figure 4 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0103] Reference below Figure 5 , which shows a schematic diagram of the structure of a computer system 500 of a terminal device suitable for implementing an embodiment of the present invention. Figure 5 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0104] like Figure 5As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the system 500 are also stored. The CPU 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0105] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage section 508 as needed.

[0106] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the above-mentioned functions defined in the system of the present invention are executed.

[0107] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0108] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0109] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor. For example, it may be described as follows: a processor includes a receiving module and an identification module. The names of these modules do not, in some cases, limit the modules themselves.

[0110] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: receiving an unlabeled image data set and a labeled image data set; using the labeled image data set, training a preset first multi-label classification network to obtain a trained first multi-label classification network; passing the unlabeled image data set through the trained first multi-label classification network to obtain a labeling result and a corresponding confidence value for each image; updating a preset second multi-label classification network according to the labeling result and the corresponding confidence value, and then labeling the image through the updated second multi-label classification network.

[0111] The technical solution according to the embodiment of the present invention can solve the problem that the prior art cannot process images quickly and effectively.

[0112] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An image recognition method, characterized in that: include: Receiving an unlabeled image dataset and a labeled image dataset; Using the labeled image data set, training a preset first multi-label classification network to obtain a trained first multi-label classification network; Using the labeled image data set, training the preset first multi-label classification network to obtain a trained first multi-label classification network, including: segmenting the labeled area of ​​the labeled image to obtain a set of candidate areas; inputting the candidate areas into the preset first multi-label classification network for training to obtain a first multi-label classification network to be trained; based on the labeled image data set, training the first multi-label classification network using a weighted cross entropy loss function to obtain a trained first multi-label classification network; The unlabeled image dataset is passed through the trained first multi-label classification network to obtain the labeling result and corresponding confidence value of each image; According to the labeling result and the corresponding confidence value, the preset second multi-label classification network is updated, and then the image is labeled by the updated second multi-label classification network.

2. The method according to claim 1, characterized in that After obtaining the labeling results and corresponding confidence values ​​of each image, including: For the labeling results and the corresponding confidence values, the average score vector of the labeling results is calculated by the Monte Carlo method; By calculating the entropy of the average score vector, the overall confidence value for each tag is obtained.

3. The method according to claim 2, characterized in that According to the labeling results and the corresponding confidence values, updating a preset second multi-label classification network includes: Convert the overall confidence value to the standardized posterior confidence to obtain the loss value of the unlabeled image; The sum of the labeled image loss value and the unlabeled image loss value is calculated to update the preset second multi-label classification network according to the labeling result and the sum of the loss values; wherein the labeled image loss value is the classification cross entropy calculated based on the labeled image data set.

4. The method according to any one of claims 1 to 3, characterized in that: Also includes: The first multi-label classification network is an 8-layer network model based on AlexNet, and the second multi-label classification network is a 5-layer convolution plus 3-layer fully connected model based on AlexNet.

5. An image recognition device, characterized in that: include: A receiving module, used for receiving an unlabeled image dataset and a labeled image dataset; A recognition module, used to train a preset first multi-label classification network using a labeled image data set to obtain a trained first multi-label classification network; The recognition module uses the labeled image data set to train the preset first multi-label classification network to obtain the trained first multi-label classification network, including: dividing the labeled image into the labeled area to obtain a set of candidate areas; inputting the candidate areas into the preset first multi-label classification network for training to obtain the first multi-label classification network to be trained; based on the labeled image data set, the first multi-label classification network is trained using a weighted cross entropy loss function to obtain the trained first multi-label classification network; the unlabeled image data set is passed through the trained first multi-label classification network to obtain the labeling result and corresponding confidence value of each image; according to the labeling result and the corresponding confidence value, the preset second multi-label classification network is updated, and the image is labeled through the updated second multi-label classification network.

6. The device according to claim 5, characterized in that After the recognition module obtains the labeling result and the corresponding confidence value of each image, it includes: For the labeling results and the corresponding confidence values, the average score vector of the labeling results is calculated by the Monte Carlo method; By calculating the entropy of the average score vector, the overall confidence value for each tag is obtained.

7. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.

8. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images