Image recognition model processing method, skin image recognition method and skin image recognition device

By extracting and utilizing global and local features in the image recognition model, the problem of image recognition schemes neglecting the detailed features of the lesions in medical image diagnosis is solved, and the reliability and accuracy of the recognition are improved.

CN118968207APending Publication Date: 2024-11-15BEIJING JINGDONG TUOXIAN TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202310545449.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-15
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

In the prior art, image recognition schemes tend to ignore the detailed characteristics of the lesions in medical image diagnosis, resulting in poor recognition reliability and accuracy.

Method used

By collecting image samples, extracting global features and local features, and training and parameter adjustment of the image recognition model based on these features, a target image recognition model is generated to improve the recognition reliability and accuracy of local features.

Benefits of technology

The reliability and accuracy of image recognition model for local features is improved, and the reliability and accuracy of image recognition is comprehensively improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118968207A_ABST
    Figure CN118968207A_ABST
Patent Text Reader

Abstract

The invention provides an image recognition model processing method and a skin image recognition method and device.The image recognition model processing method comprises the steps that an image sample is collected, and the image sample comprises local features of a specified target object; performing feature extraction on the image sample to obtain global features and local features; training an image recognition model according to the global features and the local features; determining a loss function corresponding to the global feature and a loss function corresponding to the local feature according to a training result; and performing parameter adjustment on the image recognition model according to the loss function corresponding to the global feature and the loss function corresponding to the local feature to generate a target image recognition model. According to the embodiment of the invention, the reliability and accuracy of the image recognition model are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0002] At present, in scenarios such as meteorological data processing, geological surveys, and medical image diagnosis, there are usually massive amounts of collected images that require image recognition and classification.

[0003] In related technologies, for example, in medical image diagnosis scenarios, an end-to-end convolutional neural network is used to classify skin diseases. This is mainly reflected in the fact that after the patient inputs an image, the system classifies the image based on the convolutional neural network.

[0004] However, when classifying skin diseases based on only one skin image during image recognition, detailed features of the lesion area are often overlooked, resulting in poor reliability and accuracy of the image recognition solution.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] The purpose of the present disclosure is to provide a processing method, device, electronic device and readable medium for an image recognition model, which are used to overcome, at least to a certain extent, the problems of poor reliability and accuracy of image recognition solutions caused by limitations and defects of related technologies.

[0007] According to a first aspect of an embodiment of the present disclosure, a method for processing an image recognition model is provided, comprising: collecting image samples, the image samples comprising local features of a specified target object; performing feature extraction on the image samples to obtain global features and the local features; training the image recognition model based on the global features and the local features; determining a loss function corresponding to the global features and a loss function corresponding to the local features based on the training results; and adjusting parameters of the image recognition model based on the loss function corresponding to the global features and the loss function corresponding to the local features to generate a target image recognition model.

[0008] In an exemplary embodiment of the present disclosure, training the image recognition model according to the global features and the local features includes:

[0009] Converting the global feature into a one-dimensional global feature vector through a fully connected layer of the image recognition model;

[0010] Converting the local features into a one-dimensional local feature vector through the fully connected layer;

[0011] The image recognition model is trained according to the one-dimensional global feature vector and the one-dimensional local feature vector.

[0012] In an exemplary embodiment of the present disclosure, determining the loss function corresponding to the global feature and the loss function corresponding to the local feature according to the training result includes:

[0013] Determining classification training results of the image recognition model on the one-dimensional global feature vector and the one-dimensional local feature vector;

[0014] A loss function corresponding to the global feature and a loss function corresponding to the local feature are determined according to the one-dimensional global feature vector, the one-dimensional local feature vector and the classification training result.

[0015] In an exemplary embodiment of the present disclosure, determining the loss function corresponding to the global feature and the loss function corresponding to the local feature according to the one-dimensional global feature vector, the one-dimensional local feature vector and the classification training result includes:

[0016] Calculate the cross entropy loss between the one-dimensional global feature vector and the classification training result, recorded as a first loss function;

[0017] The cross entropy loss between the one-dimensional local feature vector and the classification training result is calculated and recorded as the second loss function.

[0018] In an exemplary embodiment of the present disclosure, adjusting parameters of the image recognition model according to the loss function corresponding to the global feature and the loss function corresponding to the local feature to generate a target image recognition model includes:

[0019] Determine the loss function corresponding to the global feature according to the first loss function;

[0020] Determine the loss function and weight corresponding to the local feature according to the second loss function;

[0021] Determine the loss function of the image recognition model according to the loss function and weight corresponding to the local feature and the loss function corresponding to the global feature;

[0022] The parameters of the image recognition model are adjusted according to the loss function.

[0023] In an exemplary embodiment of the present disclosure, it further includes:

[0024] Perform reverse gradient update on the loss function of the image recognition model.

[0025] In an exemplary embodiment of the present disclosure, it further includes:

[0026] Inputting the image to be recognized into the trained image recognition model to obtain a global feature image and a local feature image;

[0027] Sending the global feature image and the local feature image to a user interaction interface;

[0028] Obtaining revision feedback of the user interaction interface on the global feature image and the local feature image;

[0029] Determining, according to the revision feedback, a classification result of the image to be identified and a matching degree corresponding to the classification result;

[0030] The classification results are sorted in descending order of the matching degree and displayed on a user interaction interface associated with the image recognition model.

[0031] According to a second aspect of an embodiment of the present disclosure, a method for processing an image recognition model is provided, comprising:

[0032] Acquire skin images containing local features;

[0033] The skin image is input into an image recognition model formed by the image recognition model processing method described in any of the above technical solutions to determine the recognition result of the skin image.

[0034] According to a third aspect of an embodiment of the present disclosure, there is provided a processing device for an image recognition model, comprising:

[0035] An acquisition module, configured to acquire image samples, wherein the image samples include local features of a specified target object;

[0036] An extraction module, configured to perform feature extraction on the image sample to obtain the global feature and the local feature;

[0037] A training module, configured to train the image recognition model according to the global features and the local features;

[0038] A determination module, configured to determine a loss function corresponding to the global feature and a loss function corresponding to the local feature according to a training result;

[0039] An adjustment module is configured to adjust parameters of the image recognition model according to a loss function corresponding to the global feature and a loss function corresponding to the local feature to generate a target image recognition model.

[0040] In an exemplary embodiment of the present disclosure, the training module is further configured to:

[0041] Converting the global feature into a one-dimensional global feature vector through a fully connected layer of the image recognition model;

[0042] Converting the local features into a one-dimensional local feature vector through the fully connected layer;

[0043] The image recognition model is trained according to the one-dimensional global feature vector and the one-dimensional local feature vector.

[0044] In an exemplary embodiment of the present disclosure, the determination module is further configured to:

[0045] Determining classification training results of the image recognition model on the one-dimensional global feature vector and the one-dimensional local feature vector;

[0046] A loss function corresponding to the global feature and a loss function corresponding to the local feature are determined according to the one-dimensional global feature vector, the one-dimensional local feature vector and the classification training result.

[0047] In an exemplary embodiment of the present disclosure, the determination module is further configured to:

[0048] Calculate the cross entropy loss between the one-dimensional global feature vector and the classification training result, recorded as a first loss function;

[0049] The cross entropy loss between the one-dimensional local feature vector and the classification training result is calculated and recorded as the second loss function.

[0050] In an exemplary embodiment of the present disclosure, the adjustment module is further configured to:

[0051] Determine the loss function corresponding to the global feature according to the first loss function;

[0052] Determine the loss function and weight corresponding to the local feature according to the second loss function;

[0053] Determine the loss function of the image recognition model according to the loss function and weight corresponding to the local feature and the loss function corresponding to the global feature;

[0054] The parameters of the image recognition model are adjusted according to the loss function.

[0055] In an exemplary embodiment of the present disclosure, the processing device of the image recognition model is also used for:

[0056] Perform reverse gradient update on the loss function of the image recognition model.

[0057] In an exemplary embodiment of the present disclosure, the processing device of the image recognition model is also used for:

[0058] Inputting the image to be recognized into the trained image recognition model to obtain a global feature image and a local feature image;

[0059] Sending the global feature image and the local feature image to a user interaction interface;

[0060] Obtaining revision feedback of the user interaction interface on the global feature image and the local feature image;

[0061] Determining, according to the revision feedback, a classification result of the image to be identified and a matching degree corresponding to the classification result;

[0062] The classification results are sorted in descending order of the matching degree and displayed on a user interaction interface associated with the image recognition model.

[0063] According to a fourth aspect of the present disclosure, a skin image recognition device is provided, comprising:

[0064] An acquisition module configured to acquire skin images containing local features;

[0065] The recognition module is configured to input the skin image into an image recognition model formed by the image recognition model processing method described in any of the above technical solutions to determine the recognition result of the skin image.

[0066] According to a fifth aspect of the present disclosure, an electronic device is provided, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute any one of the methods described above based on instructions stored in the memory.

[0067] According to a sixth aspect of the present disclosure, a computer-readable storage medium is provided, on which a program is stored, and when the program is executed by a processor, the method for processing an image recognition model as described in any one of the above items is implemented.

[0068] In the disclosed embodiment, by collecting image samples, the image samples include local features of the specified target object, and performing feature extraction on the image samples to obtain global features and the local features, and then training the image recognition model based on the global features and the local features, the reliability and accuracy of the image recognition model for local feature recognition can be improved. In addition, the loss function corresponding to the global features and the loss function corresponding to the local features are determined based on the training results, and then the parameters of the image recognition model are adjusted based on the loss function corresponding to the global features and the loss function corresponding to the local features to generate a target image recognition model, thereby comprehensively improving the reliability and accuracy of the image recognition model.

[0069] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0071] Figure 1 A schematic diagram showing an exemplary system architecture to which a processing solution of an image recognition model according to an embodiment of the present invention can be applied;

[0072] Figure 2 is a flowchart of a processing method of an image recognition model in an exemplary embodiment of the present disclosure;

[0073] Figure 3 is a flowchart of another processing method of an image recognition model in an exemplary embodiment of the present disclosure;

[0074] Figure 4 is a flowchart of another processing method of an image recognition model in an exemplary embodiment of the present disclosure;

[0075] Figure 5 is a flowchart of another processing method of an image recognition model in an exemplary embodiment of the present disclosure;

[0076] Figure 6 is a flowchart of another processing method of an image recognition model in an exemplary embodiment of the present disclosure;

[0077] Figure 7 is a flowchart of another processing method of an image recognition model in an exemplary embodiment of the present disclosure;

[0078] Figure 8 is a flowchart of another processing method of an image recognition model in an exemplary embodiment of the present disclosure;

[0079] Fig. 9 is an architectural processing diagram of another processing scheme of an image recognition model in an exemplary embodiment of the present disclosure;

[0080] Fig.10 is an interface diagram of another processing solution of an image recognition model in an exemplary embodiment of the present disclosure;

[0081] Fig.11 is an interface diagram of another processing solution of an image recognition model in an exemplary embodiment of the present disclosure;

[0082] Fig.12 is a block diagram of a processing device for an image recognition model in an exemplary embodiment of the present disclosure;

[0083] Fig.13is a block diagram of an electronic device in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0084] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0085] In addition, the accompanying drawings are only schematic diagrams of the present disclosure, and the same reference numerals in the drawings represent the same or similar parts, so their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities, which do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0086] Figure 1 A schematic diagram showing an exemplary system architecture to which a processing solution of an image recognition model according to an embodiment of the present invention can be applied.

[0087] like Figure 1 As shown, the system architecture 100 may include one or more of terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0088] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. According to the implementation requirements, there may be any number of terminal devices, networks and servers. For example, the server 105 may be a server cluster composed of multiple servers.

[0089] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Terminal devices 101, 102, 103 can be various electronic devices with display screens, including but not limited to smart phones, tablet computers, portable computers, desktop computers, etc.

[0090] In some embodiments, the processing method of the image recognition model provided in the embodiments of the present invention is generally executed by the server 105. Accordingly, the processing device of the image recognition model is generally set in the terminal device 103 (or the terminal device 101 or 102). In other embodiments, some terminals may have functions similar to those of the server device to execute the method.

[0091] The exemplary embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0092] Figure 2 is a flowchart of a processing method of an image recognition model in an exemplary embodiment of the present disclosure.

[0093] refer to Figure 2 , the processing methods of the image recognition model may include:

[0094] Step S202: collecting image samples, where the image samples include local features of the specified target object.

[0095] Step S204: extracting features from the image samples to obtain global features and the local features.

[0096] Step S206: training the image recognition model according to the global features and the local features.

[0097] Step S208, determining the loss function corresponding to the global feature and the loss function corresponding to the local feature according to the training result.

[0098] Step S210, adjusting parameters of the image recognition model according to the loss function corresponding to the global feature and the loss function corresponding to the local feature to generate a target image recognition model.

[0099] In the disclosed embodiment, by collecting image samples, the image samples include local features of the specified target object, and performing feature extraction on the image samples to obtain global features and the local features, and then training the image recognition model based on the global features and the local features, the reliability and accuracy of the image recognition model for local feature recognition can be improved. In addition, the loss function corresponding to the global features and the loss function corresponding to the local features are determined based on the training results, and then the parameters of the image recognition model are adjusted based on the loss function corresponding to the global features and the loss function corresponding to the local features to generate a target image recognition model, thereby comprehensively improving the reliability and accuracy of the image recognition model.

[0100] Below, each step of the processing method of the image recognition model is described in detail.

[0101] In an exemplary embodiment of the present disclosure, Figure 3 As shown, training the image recognition model according to the global features and the local features includes:

[0102] Step S302: convert the global feature into a one-dimensional global feature vector through the fully connected layer of the image recognition model.

[0103] Step S304: convert the local features into a one-dimensional local feature vector through the fully connected layer.

[0104] Step S306: training the image recognition model according to the one-dimensional global feature vector and the one-dimensional local feature vector.

[0105] In an exemplary embodiment of the present disclosure, Figure 4 As shown, determining the loss function corresponding to the global feature and the loss function corresponding to the local feature according to the training results includes:

[0106] Step S402: determining the classification training result of the image recognition model on the one-dimensional global feature vector and the one-dimensional local feature vector.

[0107] Step S404: determining a loss function corresponding to the global feature and a loss function corresponding to the local feature according to the one-dimensional global feature vector, the one-dimensional local feature vector and the classification training result.

[0108] In an exemplary embodiment of the present disclosure, Figure 5 As shown, determining the loss function corresponding to the global feature and the loss function corresponding to the local feature according to the one-dimensional global feature vector, the one-dimensional local feature vector and the classification training result includes:

[0109] Step S502, calculating the cross entropy loss between the one-dimensional global feature vector and the classification training result, recorded as the first loss function.

[0110] Step S504, calculating the cross entropy loss between the one-dimensional local feature vector and the classification training result, recorded as a second loss function.

[0111] In an exemplary embodiment of the present disclosure, Figure 6 As shown, adjusting the parameters of the image recognition model according to the loss function corresponding to the global feature and the loss function corresponding to the local feature to generate a target image recognition model includes:

[0112] Step S602: determine a loss function corresponding to the global feature according to the first loss function.

[0113] Step S604: determine the loss function and weight corresponding to the local feature according to the second loss function.

[0114] Step S606, determining the loss function of the image recognition model according to the loss function and weight corresponding to the local feature and the loss function corresponding to the global feature.

[0115] Step S608: adjusting parameters of the image recognition model according to the loss function.

[0116] In the above embodiment, since the same global feature may include multiple local features, and the importance of multiple local features to image recognition classification is usually different, the distinction between local features is improved by increasing the weight of local features, which is beneficial to further improve the reliability and accuracy of image recognition.

[0117] In an exemplary embodiment of the present disclosure, Figure 7 As shown, the processing method of the image recognition model also includes:

[0118] Step S702, performing reverse gradient update on the loss function of the image recognition model.

[0119] In the above embodiment, by performing reverse gradient update on the loss function, the image recognition model can learn the latest global features and local features in a timely manner, which not only improves the reliability of the image recognition model, but also improves the timeliness of the image recognition model.

[0120] In an exemplary embodiment of the present disclosure, Figure 8 As shown, the processing method of the image recognition model also includes:

[0121] Step S802: input the image to be recognized into the trained image recognition model to obtain a global feature image and a local feature image.

[0122] Step S804: sending the global feature image and the local feature image to a user interaction interface.

[0123] Step S806: obtaining revision feedback of the user interaction interface on the global feature image and the local feature image.

[0124] Step S808: determining the classification result of the image to be identified and the matching degree corresponding to the classification result according to the revised feedback.

[0125] Step S810, sorting the classification results in descending order of the matching degree, and displaying them on a user interaction interface associated with the image recognition model.

[0126] In an exemplary embodiment of the present disclosure, a method for processing an image recognition model is also provided, comprising:

[0127] Acquire skin images containing local features;

[0128] The skin image is input into an image recognition model formed by the image recognition model processing method described in any of the above technical solutions to determine the recognition result of the skin image.

[0129] Combine the following Figures 9 to 11 The processing steps of the image recognition model of the embodiment of the present disclosure are described in detail.

[0130] The image recognition model of the embodiment of the present disclosure is a dual-branch network model, which can be recorded as a skin lesion area target detection model in the medical image identification scenario.

[0131] Step 1: After a user or image identification expert uploads a skin image via a mobile phone, the potential skin lesion area will be identified based on the skin lesion area target detection model.

[0132] More generally, the skin lesion area target detection model may select a single-stage Yolo series model, or a more accurate dual-stage detection Faster RCNN model, etc., but is not limited thereto.

[0133] Step 2. In order to introduce the knowledge of image identification experts or users, the embodiment of this scheme allows image identification experts or users to manually modify the skin lesion area results identified in step 1. It supports both modification on existing detection results and reselection of skin lesion areas, which can reduce the error rate of subsequent skin disease identification results caused by the detection error of the model in step 1.

[0134] Step 3. After steps 1 and 2, the embodiment of this scheme obtains a total of 2 images, one of which is a complete skin image, and the other is an image containing only the skin lesion area. The embodiment of this scheme inputs these two images into a dual-branch network at the same time, which can obtain more accurate skin disease recognition results and solve the problem that the model needs to pay attention to both global and local features of the skin image at the same time.

[0135] First, if Fig. 9 As shown, the embodiment of the present solution collects a large number of skin disease images to construct a data set.

[0136] Secondly, the skin lesion target detection model in step 1 is used to perform target detection on the image, so as to obtain a large number of image pairs of the original image 902 and the skin lesion image 904. In the training process and the recognition process, Fig.10 The order image recognition interface 1000 shown introduces the doctor's knowledge by manually modifying the recognition result 1002.

[0137] Third, the backbone network Shared Backbone 906 of the implementation model of the present solution can select either a convolutional neural network or a Transformer network structure, but is not limited thereto.

[0138] Fourth, the above two images can obtain the global feature representation and local feature representation of the image respectively through the image feature extraction network, such as Fig. 9 Feature map 908 is shown.

[0139] Fifth, the two feature maps are converted into one-dimensional feature vectors through the fully connected layer FC 910, and the cross entropy loss is calculated with the final classification result. The loss obtained by the global feature representation is called Global Loss 912, and the loss obtained by the local feature representation is called Local Loss 914.

[0140] Sixth, the final loss function 916 of the network is obtained by adding the above two loss functions, that is, Loss = GlobalLoss + Local Loss, and reverse gradient update is performed on this basis, so that the model can learn the global features and local features of the skin image at the same time.

[0141] At this point, the embodiment of this solution has completed the training of the dual-branch network model.

[0142] Step 4: After obtaining the possible skin disease classification results, the embodiment of this scheme passes Fig.11The image recognition result interface 1100 in the image recognition system displays the final result to the image identification expert or user. The recognition results are sorted in descending order according to the final matching degree, providing the image identification expert or user with more information for auxiliary reference.

[0143] Corresponding to the above method embodiments, the present disclosure also provides a processing device for an image recognition model, which can be used to execute the above method embodiments.

[0144] Fig.12 It is a block diagram of a processing device of an image recognition model in an exemplary embodiment of the present disclosure.

[0145] refer to Fig.12 , the processing device 1200 of the image recognition model may include:

[0146] The acquisition module 1202 is configured to acquire image samples, where the image samples include local features of a specified target object.

[0147] The extraction module 1204 is configured to perform feature extraction on the image sample to obtain the global feature and the local feature.

[0148] The training module 1206 is configured to train the image recognition model according to the global features and the local features.

[0149] The determination module 1208 is configured to determine the loss function corresponding to the global feature and the loss function corresponding to the local feature according to the training results.

[0150] The adjustment module 1210 is configured to adjust the parameters of the image recognition model according to the loss function corresponding to the global feature and the loss function corresponding to the local feature to generate a target image recognition model.

[0151] In an exemplary embodiment of the present disclosure, the training module 1206 is further configured to:

[0152] Converting the global feature into a one-dimensional global feature vector through a fully connected layer of the image recognition model;

[0153] Converting the local features into a one-dimensional local feature vector through the fully connected layer;

[0154] The image recognition model is trained according to the one-dimensional global feature vector and the one-dimensional local feature vector.

[0155] In an exemplary embodiment of the present disclosure, the determination module 1208 is further configured to:

[0156] Determining classification training results of the image recognition model on the one-dimensional global feature vector and the one-dimensional local feature vector;

[0157] A loss function corresponding to the global feature and a loss function corresponding to the local feature are determined according to the one-dimensional global feature vector, the one-dimensional local feature vector and the classification training result.

[0158] In an exemplary embodiment of the present disclosure, the determination module 1208 is further configured to:

[0159] Calculate the cross entropy loss between the one-dimensional global feature vector and the classification training result, recorded as a first loss function;

[0160] The cross entropy loss between the one-dimensional local feature vector and the classification training result is calculated and recorded as the second loss function.

[0161] In an exemplary embodiment of the present disclosure, the adjustment module 1210 is further configured to:

[0162] Determine the loss function corresponding to the global feature according to the first loss function;

[0163] Determine the loss function and weight corresponding to the local feature according to the second loss function;

[0164] Determine the loss function of the image recognition model according to the loss function and weight corresponding to the local feature and the loss function corresponding to the global feature;

[0165] The parameters of the image recognition model are adjusted according to the loss function.

[0166] In an exemplary embodiment of the present disclosure, the image recognition model processing device 1200 is further used for:

[0167] Perform reverse gradient update on the loss function of the image recognition model.

[0168] In an exemplary embodiment of the present disclosure, the image recognition model processing device 1200 is further used for:

[0169] Inputting the image to be recognized into the trained image recognition model to obtain a global feature image and a local feature image;

[0170] Sending the global feature image and the local feature image to a user interaction interface;

[0171] Obtaining revision feedback of the user interaction interface on the global feature image and the local feature image;

[0172] Determining, according to the revision feedback, a classification result of the image to be identified and a matching degree corresponding to the classification result;

[0173] The classification results are sorted in descending order of the matching degree and displayed on a user interaction interface associated with the image recognition model.

[0174] Since the functions of the image recognition model processing device 1200 have been described in detail in the corresponding method embodiments, the present disclosure will not elaborate on them here.

[0175] In an exemplary embodiment of the present disclosure, a skin image recognition device is provided, comprising:

[0176] An acquisition module configured to acquire skin images containing local features;

[0177] The recognition module is configured to input the skin image into an image recognition model formed by the image recognition model processing method described in any of the above technical solutions to determine the recognition result of the skin image.

[0178] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0179] In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.

[0180] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as a system, method or program product. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as a "circuit", "module" or "system".

[0181] Refer to the following Fig.13 An electronic device 1300 according to this embodiment of the present invention is described. Fig.13 The electronic device 1300 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0182] like Fig.13 As shown, the electronic device 1300 is in the form of a general computing device. The components of the electronic device 1300 may include but are not limited to: at least one processing unit 1310, at least one storage unit 1320, and a bus 1330 connecting different system components (including the storage unit 1320 and the processing unit 1310).

[0183] The storage unit stores program codes, which can be executed by the processing unit 1310, so that the processing unit 1310 performs the steps according to various exemplary embodiments of the present invention described in the above “Exemplary Method” section of this specification. For example, the processing unit 1310 can perform the method shown in the embodiment of the present disclosure.

[0184] The storage unit 1320 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 13201 and / or a cache storage unit 13202 , and may further include a read-only storage unit (ROM) 13203 .

[0185] The storage unit 1320 may also include a program / utility 13204 having a set (at least one) of program modules 13205, such program modules 13205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0186] Bus 1330 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0187] The electronic device 1300 may also communicate with one or more external devices 1340 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 1300, and / or communicate with any device that enables the electronic device 1300 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1350. Furthermore, the electronic device 1300 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 1360. As shown, the network adapter 1360 communicates with other modules of the electronic device 1300 via a bus 1330. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 1300, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0188] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0189] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present invention can also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present invention described in the above "Exemplary Method" section of the present specification.

[0190] The program product for implementing the above method according to an embodiment of the present invention can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto, and in this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0191] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0192] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0193] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0194] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0195] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0196] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are to be considered exemplary only, and the true scope and concept of the present disclosure are indicated by the claims.

Claims

1. A processing method for an image recognition model, characterized in that: include: Collecting an image sample, wherein the image sample includes local features of a specified target object; Performing feature extraction on the image sample to obtain global features and the local features; Training the image recognition model according to the global features and the local features; Determine the loss function corresponding to the global feature and the loss function corresponding to the local feature according to the training result; The parameters of the image recognition model are adjusted according to the loss function corresponding to the global feature and the loss function corresponding to the local feature to generate a target image recognition model.

2. The method for processing an image recognition model according to claim 1, wherein: Training the image recognition model according to the global features and the local features includes: Converting the global feature into a one-dimensional global feature vector through a fully connected layer of the image recognition model; Converting the local features into a one-dimensional local feature vector through the fully connected layer; The image recognition model is trained according to the one-dimensional global feature vector and the one-dimensional local feature vector.

3. The processing method of the image recognition model according to claim 2, characterized in that: Determining the loss function corresponding to the global feature and the loss function corresponding to the local feature according to the training result includes: Determining classification training results of the image recognition model on the one-dimensional global feature vector and the one-dimensional local feature vector; A loss function corresponding to the global feature and a loss function corresponding to the local feature are determined according to the one-dimensional global feature vector, the one-dimensional local feature vector and the classification training result.

4. The method for processing an image recognition model according to claim 3, wherein: Determining the loss function corresponding to the global feature and the loss function corresponding to the local feature according to the one-dimensional global feature vector, the one-dimensional local feature vector and the classification training result comprises: Calculate the cross entropy loss between the one-dimensional global feature vector and the classification training result, recorded as a first loss function; The cross entropy loss between the one-dimensional local feature vector and the classification training result is calculated and recorded as the second loss function.

5. The method for processing an image recognition model according to claim 4, characterized in that: Adjusting parameters of the image recognition model according to the loss function corresponding to the global feature and the loss function corresponding to the local feature to generate a target image recognition model includes: Determine the loss function corresponding to the global feature according to the first loss function; Determine the loss function and weight corresponding to the local feature according to the second loss function; Determine the loss function of the image recognition model according to the loss function and weight corresponding to the local feature and the loss function corresponding to the global feature; The parameters of the image recognition model are adjusted according to the loss function.

6. The method for processing an image recognition model according to any one of claims 1 to 5, characterized in that: Also includes: Perform reverse gradient update on the loss function of the image recognition model.

7. The method for processing an image recognition model according to any one of claims 1 to 5, characterized in that: Also includes: Inputting the image to be recognized into the trained image recognition model to obtain a global feature image and a local feature image; Sending the global feature image and the local feature image to a user interaction interface; Obtaining revision feedback of the user interaction interface on the global feature image and the local feature image; Determining, according to the revision feedback, a classification result of the image to be identified and a matching degree corresponding to the classification result; The classification results are sorted in descending order of the matching degree and displayed on a user interaction interface associated with the image recognition model.

8. A skin image recognition method, characterized in that: include: Collect skin images containing local features; The skin image is input into an image recognition model formed by the image recognition model processing method according to any one of claims 1 to 7 to determine a recognition result of the skin image.

9. A processing device for an image recognition model, characterized in that: include: An acquisition module, configured to acquire image samples, wherein the image samples include local features of a specified target object; An extraction module, configured to perform feature extraction on the image sample to obtain the global feature and the local feature; A training module, configured to train the image recognition model according to the global features and the local features; A determination module, configured to determine a loss function corresponding to the global feature and a loss function corresponding to the local feature according to a training result; An adjustment module is configured to adjust parameters of the image recognition model according to a loss function corresponding to the global feature and a loss function corresponding to the local feature to generate a target image recognition model.

10. The image recognition model processing device according to claim 9, characterized in that: The training module is also set up as: Converting the global feature into a one-dimensional global feature vector through a fully connected layer of the image recognition model; Converting the local features into a one-dimensional local feature vector through the fully connected layer; The image recognition model is trained according to the one-dimensional global feature vector and the one-dimensional local feature vector.

11. The image recognition model processing device according to claim 10, characterized in that: Make sure the module is also set to: Determining classification training results of the image recognition model on the one-dimensional global feature vector and the one-dimensional local feature vector; A loss function corresponding to the global feature and a loss function corresponding to the local feature are determined according to the one-dimensional global feature vector, the one-dimensional local feature vector and the classification training result.

12. The image recognition model processing device according to claim 11, characterized in that: Make sure the module is also set to: Calculate the cross entropy loss between the one-dimensional global feature vector and the classification training result, recorded as a first loss function; The cross entropy loss between the one-dimensional local feature vector and the classification training result is calculated and recorded as the second loss function.

13. The image recognition model processing device according to claim 12, characterized in that: The tuning module is also set to: Determine the loss function corresponding to the global feature according to the first loss function; Determine the loss function and weight corresponding to the local feature according to the second loss function; Determine the loss function of the image recognition model according to the loss function and weight corresponding to the local feature and the loss function corresponding to the global feature; The parameters of the image recognition model are adjusted according to the loss function.

14. The processing device of the image recognition model according to any one of claims 9 to 13, characterized in that: The processing device of the image recognition model is also used to: Perform a reverse gradient update on the loss function of the image recognition model.

15. The processing device of the image recognition model according to any one of claims 9 to 13, characterized in that: The processing device of the image recognition model is also used to: Inputting the image to be recognized into the trained image recognition model to obtain a global feature image and a local feature image; Sending the global feature image and the local feature image to a user interaction interface; Obtaining revision feedback of the user interaction interface on the global feature image and the local feature image; Determining, according to the revision feedback, a classification result of the image to be identified and a matching degree corresponding to the classification result; The classification results are sorted in descending order of the matching degree and displayed on a user interaction interface associated with the image recognition model.

16. A skin image recognition device, characterized in that: include: An acquisition module configured to acquire skin images containing local features; The recognition module is configured to input the skin image into an image recognition model formed by the processing method of the image recognition model according to any one of claims 1 to 7 to determine a recognition result of the skin image.

17. An electronic device, characterized in that: include: Memory; as well as A processor coupled to the memory, the processor being configured to execute the method for processing the image recognition model according to any one of claims 1 to 8 based on instructions stored in the memory.

18. A computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the processing method of the image recognition model as described in any one of claims 1 to 8.

Citation Information

Patent Citations

  • Pedestrian re-identification method and device, electronic equipment and storage medium

    CN111488833A

  • Cross-modal pedestrian re-identification method based on character attribute assistance

    CN115909407A