Target object recognition method, device, apparatus and storage medium

Abstract

The present application relates to artificial intelligence technology in the field of digital medicine, and discloses a target object recognition method, comprising: performing image pair enhancement processing on images in a training image set to obtain a training image pair set, performing image classification on the images in the training image pair set by using a preset contrast classification network to obtain an image classification result, performing contrast learning on the images in the training image pair set by using the contrast classification network to obtain a contrast learning result, jointly training the contrast classification network based on the image classification result and the contrast learning result to obtain a target object recognition model, and performing target object recognition on a to-be-recognized image by using the target object recognition model to obtain a recognition result. The present application also relates to blockchain technology, and the recognition result can be stored in a node of a blockchain. The present application also provides a target object recognition device, an electronic device and a readable storage medium. The present application can improve the accuracy of target object recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital medical treatment and artificial intelligence, and in particular to a target object recognition method and device, electronic equipment and a readable storage medium. BACKGROUND

[0002] With the development of artificial intelligence, target object recognition is widely used in different fields, which can improve the efficiency of data recognition. For example, in medical business, specific drug information can be determined by recognizing medicine box pictures.

[0003] In the prior art, there are mainly two medicine box recognition methods: one is based on ocr recognition technology to recognize the drug name on the medicine box. However, the ocr technology can only recognize the generic name (such as metformin hydrochloride sustained-release tablets) on the medicine box, and cannot correspond to more specific drug information (such as specific manufacturers), and the drug name is relatively isolated, which leads to inaccurate drug recognition; the second is based on a classification model of pictures, which extracts picture features and inputs them into a classification network to give the category of the drug. However, since the same manufacturer's drugs may have consistent outer packaging, but the drugs are completely different, relying solely on the outer packaging classification model will result in low drug recognition accuracy. SUMMARY

[0004] The present application provides a target object recognition method, device, electronic equipment and readable storage medium, which can improve the accuracy of target object recognition.

[0005] To achieve the above purpose, the present application provides a target object recognition method, which comprises:

[0006] Obtaining a training image set, performing image pair enhancement processing on the images in the training image set to obtain a training image pair set;

[0007] Using a preset contrast classification network to classify the images in the training image pair set to obtain an image classification result;

[0008] Using the contrast classification network to perform contrast learning on the images in the training image pair set to obtain a contrast learning result;

[0009] Jointly training the contrast classification network based on the image classification result and the contrast learning result to obtain a target object recognition model;

[0010] Using the target object recognition model to recognize the target object of a to-be-recognized image to obtain a recognition result.

[0011] Optionally, the image pair enhancement processing on the images in the training image set to obtain the training image pair set comprises:

[0012] sequentially selecting images in the training image set as target images;

[0013] performing twice different image enhancement processing on the target image to obtain two different enhanced images, and taking the two different enhanced images as an enhanced image pair of the target image;

[0014] after all images in the training image set are selected as target images, all enhanced image pairs are summarized to obtain the training image pair set.

[0015] Optionally, before the image classification of the images in the training image pair set by using the preset contrast classification network, the method further comprises:

[0016] parallel processing of the two preset feature extraction networks to obtain a feature extraction layer;

[0017] connecting a classification layer after the feature extraction layer, and connecting a contrast calculation layer after the feature extraction layer to obtain the contrast classification network.

[0018] Optionally, the image classification of the images in the training image pair set by using the preset contrast classification network to obtain an image classification result comprises:

[0019] feature extraction of the enhanced image pair in the training image pair set by using the feature extraction layer in the contrast classification network to obtain an image feature pair;

[0020] image classification of the image feature pair by using the classification layer in the contrast classification network to obtain an image classification result.

[0021] Optionally, the joint training of the contrast classification network based on the image classification result and the contrast learning result to obtain a target object recognition model comprises:

[0022] calculating a classification loss by using the image classification result, and calculating a contrast loss by using the contrast learning result;

[0023] calculating a total loss by using the classification loss and the contrast loss, adjusting network parameters in the contrast classification network when the total loss does not meet a preset loss threshold, and returning to the step of performing image pair enhancement processing on the images in the training image set until the total loss meets the preset loss threshold, and taking the feature extraction network in the contrast classification network as the target object recognition model.

[0024] Optionally, before the target object recognition of the to-be-recognized image by using the target object recognition model, the method further comprises:

[0025] extracting features of the images in the training image set by using the target object recognition model to obtain image feature vectors;

[0026] mapping the image feature vectors with preset product information, and collecting all the associated image feature vectors to obtain an image vector library.

[0027] Optionally, the target object recognition of the to-be-recognized image by using the target object recognition model comprises:

[0028] extracting features of the to-be-recognized image by using the target object recognition model to obtain a to-be-recognized feature vector;

[0029] retrieving the image feature vectors in the image vector library by using the to-be-recognized feature vector, and determining product information corresponding to the retrieved image feature vector as the recognition result.

[0030] To solve the above problems, the application further provides a target object recognition device, which comprises:

[0031] an image enhancement module, configured to obtain a training image set, perform image pair enhancement processing on images in the training image set, and obtain a training image pair set;

[0032] a model training module, configured to perform image classification on images in the training image pair set by using a preset contrast classification network, obtain an image classification result, perform contrast learning on the images in the training image pair set by using the contrast classification network, obtain a contrast learning result, jointly train the contrast classification network based on the image classification result and the contrast learning result, and obtain a target object recognition model;

[0033] a target object recognition module, configured to perform target object recognition on a to-be-recognized image by using the target object recognition model, and obtain a recognition result.

[0034] To solve the above problems, the application further provides an electronic device, which comprises:

[0035] a memory, configured to store at least one computer program; and

[0036] a processor, configured to execute the computer program stored in the memory to implement the target object recognition method.

[0037] To solve the above problems, the application further provides a computer readable storage medium, which stores at least one computer program, and the at least one computer program is executed by a processor in an electronic device to implement the target object recognition method.

[0038] The application can improve the data quality by image pair enhancement processing on the images in the training image set to obtain a training image pair set. During model training, the contrast classification network is jointly trained in combination with the image classification result and the contrast learning result, so that a target object recognition model with higher recognition accuracy can be obtained. For example, in medicine box image recognition, the joint training not only focuses on the classification of medicines, but also focuses on the contrast and similarity results of medicine boxes, so that the accuracy of medicine box image recognition can be improved. At the same time, the target object recognition model can recognize the target object in the image to be recognized, and the recognition result can contain more associated information (such as medicine manufacturer information), so as to further improve the accuracy of target object recognition. Therefore, the target object recognition method, device, electronic equipment and computer readable storage medium provided by the application can improve the accuracy of target object recognition. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 A flowchart of a target object recognition method provided by an embodiment of the application is shown.

[0040] Figure 2 A function module diagram of a target object recognition device provided by an embodiment of the application is shown.

[0041] Figure 3 A structure diagram of an electronic device for implementing the target object recognition method provided by an embodiment of the application is shown.

[0042] The implementation, functional features and advantages of the application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0043] It should be understood that the specific embodiments described herein are only used to explain the application, and are not used to limit the application.

[0044] An embodiment of the application provides a target object recognition method. The execution subject of the target object recognition method includes but is not limited to at least one of the electronic devices that can be configured to execute the method provided by the embodiment of the application, such as a server and a terminal. In other words, the target object recognition method can be executed by software or hardware installed in a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be a stand-alone server, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (CDN), and big data and artificial intelligence platforms, etc. basic cloud computing services.

[0045] REFERENCE Figure 1Fig. 1 shows a flowchart of a target object recognition method according to an embodiment of the present application.

[0046] In this embodiment, the target object recognition method comprises the following steps S1-S5:

[0047] S1, obtain a training image set, and perform image pair enhancement processing on images in the training image set to obtain a training image pair set.

[0048] In this embodiment, the image pair enhancement processing refers to performing two different image processing on one image, and taking the two images obtained by image enhancement as an image pair. The training image pair set is a training set composed of image pairs obtained by image enhancement. For example, in the medical field, the training image set can be a medicine corresponding medicine box image.

[0049] In detail, the image pair enhancement processing on the images in the training image set to obtain the training image pair set comprises:

[0050] select an image in the training image set as a target image in turn;

[0051] perform two different image enhancement processing on the target image to obtain two different enhanced images, and take the two different enhanced images as an enhanced image pair of the target image;

[0052] After all the images in the training image set are selected as target images, all the enhanced image pairs are summarized to obtain the training image pair set.

[0053] In this embodiment, performing two different image processing on the same image and taking the two images as an image pair can increase the number of training images, and the image pair is enhanced from the same image, so that there is no large difference between the enhanced images, avoiding the large difference between the enhanced images and the original image, and improving the quality of the training images. In addition, the enhanced image pair of the same target image has the same label.

[0054] For example, for a medicine box image in the medical field, for any medicine box picture, perform two image processing (including random flipping, random cropping, random noise adding, random brightness adjusting, etc.), obtain two processed enhanced pictures, and form an enhanced image pair. The two enhanced images correspond to the same generic name of medicine (i.e. label).

[0055] S2, use a preset contrast classification network to perform image classification on the images in the training image pair set to obtain an image classification result.

[0056] In the embodiment of the present application, the preset contrast classification network can include: an image feature extraction layer, a classification layer, a contrast calculation layer, etc., wherein the image feature extraction layer is used to extract image features, and can be a resnet50 network or the like, the classification layer is used to classify the extracted image features, and the contrast calculation layer is used to perform similarity contrast calculation on the extracted image features.

[0057] Optionally, before the image classification of the images in the set of training image pairs is performed by using the preset contrast classification network, the method further includes:

[0058] The two preset feature extraction networks are processed in parallel to obtain a feature extraction layer.

[0059] A classification layer is connected in series after the feature extraction layer, and a contrast calculation layer is connected in series after the feature extraction layer to obtain the contrast classification network.

[0060] In an optional embodiment of the present application, the contrast classification network is constructed based on a Siamese Network structure. The Siamese Network structure is composed of two neural networks (which can be CNN, resnet50 network or the like feature extraction network) sharing weights, and is used to measure the similarity of two inputs. Among them, the Siamese Network has two inputs (Input1 and Input2), and the two outputs are sent to two neural networks (Network1 and Network2). The two neural networks respectively map the inputs to a new space to form the representation of the inputs in the new space. Through the calculation of Loss, the similarity of the two inputs is evaluated.

[0061] Compared with the Siamese Network, the present application adds a classification layer connected in series after the feature extraction layer, which increases the classification ability of the model for images and can improve the accuracy of model recognition.

[0062] In detail, the image classification of the images in the set of training image pairs by using the preset contrast classification network obtains an image classification result, including:

[0063] The feature extraction layer in the contrast classification network is used to extract features of the enhanced image pairs in the set of training image pairs to obtain an image feature pair.

[0064] The classification layer in the contrast classification network is used to classify the image feature pair to obtain an image classification result.

[0065] In an optional embodiment of the present application, taking the medical field medicine box image as an example, the resnet50 network is used to perform feature extraction on the enhanced medicine box enhanced image pair after enhancement, to obtain a medicine box image feature vector pair, the classification layer is used to classify the medicine box image feature vector, to obtain the medicine box image classification result of each enhanced image, for example, the original medicine box image a is labeled as medicine A, and the classification layer is used to determine whether the enhanced medicine box image a1 and the medicine box image a2 are correctly classified as medicine A.

[0066] S3, performing contrast learning on the images in the training image pair set by using the contrast classification network, to obtain a contrast learning result.

[0067] In detail, the contrast learning on the images in the training image pair set by using the contrast classification network to obtain a contrast learning result comprises:

[0068] The similarity of the image feature pair is calculated by using the contrast calculation layer in the contrast classification network, and the similarity is taken as the contrast learning result.

[0069] In the embodiment of the present application, the similarity of the image feature pair can be calculated by using the Euclidean distance formula, the cosine distance formula, etc.

[0070] In an optional embodiment of the present application, in the medicine box image recognition in the medical field, the enhanced image pair after data enhancement is obtained by transforming one image, and during medicine box recognition, the images should be similar, so the similarity of the image feature pair can be determined by calculating the Euclidean distance in the contrast calculation layer.

[0071] S4, jointly training the contrast classification network based on the image classification result and the contrast learning result, to obtain a target object recognition model.

[0072] In the embodiment of the present application, the joint training of the contrast classification network means that two different losses are calculated by using the image classification result and the contrast learning result to perform iterative training.

[0073] In detail, the joint training of the contrast classification network based on the image classification result and the contrast learning result to obtain a target object recognition model comprises:

[0074] The classification loss is calculated by using the image classification result, and the contrast loss is calculated by using the contrast learning result.

[0075] A total loss is calculated using the classification loss and the contrast loss, and when the total loss does not satisfy a preset loss threshold, network parameters in the contrast classification network are adjusted, and the step of performing image pair enhancement processing on the images in the set of training images is returned until the total loss satisfies the preset loss threshold, and a feature extraction network in the contrast classification network is taken as the target object recognition model.

[0076] In an optional embodiment of the present application, the classification loss loss1 can be calculated by a cross-entropy loss function, the contrast loss loss2 can be calculated by a contrastive loss function, and a total loss loss = loss1 + loss2 is taken as a model total loss function, and the contrast classification network is iteratively trained until the loss value satisfies a preset loss threshold (for example, less than the preset loss threshold), and a feature extraction network (for example, a resnet50 network) in the contrast classification network is taken as the target object recognition model. By combining the classification loss and the contrast learning loss, the accuracy and stability of model recognition can be greatly improved.

[0077] S5, performing target object recognition on a to-be-recognized image using the target object recognition model to obtain a recognition result.

[0078] In an optional embodiment of the present application, before performing target object recognition on a to-be-recognized image using the target object recognition model, the following steps are further included:

[0079] Performing feature extraction on the images in the set of training images using the target object recognition model to obtain image feature vectors;

[0080] Mapping and associating the image feature vectors with preset product information, and collecting all the associated image feature vectors to obtain an image vector library.

[0081] In an optional embodiment of the present application, taking a medical kit image in the medical field as an example, after the target object recognition model is trained, the kit vector of each kit image is recalculated, and is mapped and associated with original product information, the product information including: drug name, manufacturer, taking time, dosage, and drug category, etc. The kit vector library is formed by storing the associated kit vectors, and in kit recognition, not only the drug generic name can be recognized, but also more associated product information can be recognized.

[0082] In detail, performing target object recognition on a to-be-recognized image using the target object recognition model to obtain a recognition result includes:

[0083] Performing feature extraction on the to-be-recognized image using the target object recognition model to obtain a to-be-recognized feature vector;

[0084] Retrieving an image feature vector in the image vector library by using the to-be-identified feature vector, and determining product information corresponding to the retrieved image feature vector as the identification result.

[0085] In the embodiment of the present application, for a to-be-identified medicine box image, a feature is directly extracted by using a target object recognition model, a to-be-identified medicine box vector obtained by the feature extraction is used to retrieve a medicine box vector library, a most similar medicine box vector is found out, and corresponding medicine information is the final identification result. Meanwhile, if a medicine box image needs to be added, a model does not need to be retrained, and only a vector of the medicine box image needs to be calculated, and a retrieved medicine box vector library needs to be updated.

[0086] The present application can improve the data quality by performing image pair enhancement processing on the images in the training image set to obtain the training image pair set. During model training, the contrast classification network is jointly trained in combination with the image classification result and the contrast learning result, so that a target object recognition model with higher recognition accuracy can be obtained. For example, in medicine box image recognition, the joint training not only focuses on the classification of medicines, but also focuses on the contrast and similarity results of medicine boxes, so that the accuracy of medicine box image recognition can be improved. Meanwhile, the target object recognition model is used to recognize the to-be-identified image, and more associated information (such as medicine manufacturer information) can be included in the recognition result, so that the accuracy of target object recognition can be further improved. Therefore, the target object recognition method provided by the present application can improve the accuracy of target object recognition.

[0087] As shown in Figure 2 Fig. 1 is a functional module diagram of a target object recognition device provided by an embodiment of the present application.

[0088] The target object recognition device 100 described in the present application can be installed in an electronic device. According to the functions to be implemented, the target object recognition device 100 can include an image enhancement module 101, a model training module 102, and a target object recognition module 103. The modules described in the present application can also be referred to as units, which refer to a series of computer program segments that can be executed by an electronic device processor and can complete a fixed function, and are stored in the memory of the electronic device.

[0089] In the present embodiment, the functions of each module / unit are as follows:

[0090] The image enhancement module 101 is configured to obtain a training image set, perform image pair enhancement processing on the images in the training image set, and obtain a training image pair set.

[0091] The model training module 102 is configured to perform image classification on images in the set of training image pairs by using a preset contrast classification network, to obtain image classification results, perform contrast learning on the images in the set of training image pairs by using the contrast classification network, to obtain contrast learning results, and jointly train the contrast classification network based on the image classification results and the contrast learning results, to obtain a target object recognition model.

[0092] The target object recognition module 103 is configured to perform target object recognition on a to-be-recognized image by using the target object recognition model, to obtain a recognition result.

[0093] In detail, the specific implementation of each module of the target object recognition apparatus 100 is as follows.

[0094] Step one, obtaining a set of training images, performing image pair enhancement processing on images in the set of training images, to obtain a set of training image pairs.

[0095] In the embodiment of the present application, the image pair enhancement processing refers to performing two different image processing on one image, and taking the two images obtained by image enhancement as an image pair. The set of training image pairs is a training set composed of image pairs obtained by image enhancement. For example, in the medical field, the set of training images can be images of medicine corresponding to medicine boxes.

[0096] In detail, the image pair enhancement processing on the images in the set of training images to obtain the set of training image pairs includes:

[0097] Selecting images in the set of training images as target images in turn;

[0098] Performing two different image enhancement processing on the target images, to obtain two different enhanced images, and taking the two different enhanced images as an enhanced image pair of the target image;

[0099] After the images in the set of training images are all selected as target images, all enhanced image pairs are collected to obtain the set of training image pairs.

[0100] In the embodiment of the present application, performing two different image processing on the same image and taking the image as an image pair can increase the number of training images, and the image pair is enhanced from the same image, so that there are no enhanced images with large differences, avoiding the difference between the enhanced image and the original image being too large, and improving the quality of the training image. In addition, the enhanced image pair of the same target image has the same label.

[0101] For example, for the medical field of kit images, for any one kit picture, two times of image processing (including random flipping, random cropping, random noise adding, random brightness adjusting, etc.) are performed to obtain two processed enhanced pictures, and an enhanced image pair is formed, and the two enhanced pictures correspond to the same drug generic name (i.e. annotation).

[0102] Step two, using a preset contrast classification network to perform image classification on the images in the set of training image pairs to obtain image classification results.

[0103] In the embodiment of the application, the preset contrast classification network can include an image feature extraction layer, a classification layer, a contrast calculation layer, etc., wherein the image feature extraction layer is used to extract image features and can be a resnet50 network or the like, the classification layer is used to classify the extracted image features, and the contrast calculation layer is used to perform similarity contrast calculation on the extracted image features.

[0104] Optionally, before the image classification on the images in the set of training image pairs using the preset contrast classification network, the following steps are further included:

[0105] The two preset feature extraction networks are processed in parallel to obtain a feature extraction layer;

[0106] A classification layer is connected in series after the feature extraction layer, and a contrast calculation layer is connected in series after the feature extraction layer to obtain the contrast classification network.

[0107] In an optional embodiment of the application, the contrast classification network is constructed based on a Siamese Network (twin network / connected network) structure, and the Siamese Network structure is composed of two neural networks (which can be CNN, resnet50 network or the like feature extraction network) sharing weights and is used to measure the similarity of two inputs. Among them, the Siamese Network has two inputs (Input1 and Input2), and the two outputs are sent to two neural networks (Network1 and Network2), which respectively map the inputs to a new space to form the representation of the inputs in the new space, and then evaluate the similarity of the two inputs through the calculation of Loss.

[0108] Compared with the Siamese Network, the application adds the classification ability of the model to the image by connecting a classification layer in series after the feature extraction layer, which can improve the accuracy of model recognition.

[0109] In detail, the image classification on the images in the set of training image pairs using the preset contrast classification network to obtain image classification results includes:

[0110] extracting features of the enhanced image pairs in the training image pair set by using a feature extraction layer in the contrast classification network to obtain image feature pairs;

[0111] classifying the image feature pairs by using a classification layer in the contrast classification network to obtain image classification results.

[0112] In an optional embodiment of the present application, taking the medical field of kit images as an example, the resnet50 network is used to extract features of the enhanced kit enhanced image pairs to obtain kit image feature vector pairs, and the classification layer is used to classify the kit image features to obtain the kit image classification results of each enhanced image, for example, the original kit image a is labeled as drug A, and whether the enhanced kit image a1 and the kit image a2 are correctly classified as drug A is determined by the classification layer.

[0113] Step three, contrastively learning the images in the training image pair set by using the contrast classification network to obtain contrast learning results.

[0114] In detail, the contrastively learning the images in the training image pair set by using the contrast classification network to obtain contrast learning results comprises:

[0115] calculating the similarity of the image feature pairs by using a contrast calculation layer in the contrast classification network, and taking the similarity as the contrast learning results.

[0116] In the embodiment of the present application, the similarity of the image feature pairs can be calculated by using the Euclidean distance formula, the cosine distance formula, etc.

[0117] In an optional embodiment of the present application, in the medical field of kit image recognition, the enhanced image pairs after data enhancement are obtained by transforming one image, and during kit recognition, they should be similar, so the similarity of the image feature pairs can be determined by calculating the Euclidean distance in the contrast calculation layer.

[0118] Step four, jointly training the contrast classification network based on the image classification results and the contrast learning results to obtain a target object recognition model.

[0119] In the embodiment of the present application, the joint training of the contrast classification network means that two different losses are calculated by using the image classification results and the contrast learning results to perform iterative training.

[0120] In detail, the joint training of the contrast classification network based on the image classification results and the contrast learning results to obtain a target object recognition model comprises:

[0121] calculating a classification loss by using the image classification results, and calculating a contrast loss by using the contrast learning results.

[0122] The total loss is calculated using the classification loss and the contrast loss, and when the total loss does not satisfy a preset loss threshold, the network parameters in the contrast classification network are adjusted, and the step of performing image pair enhancement processing on the images in the set of training images is returned until the total loss satisfies the preset loss threshold, and the feature extraction network in the contrast classification network is taken as the target object recognition model.

[0123] In an optional embodiment of the present application, the classification loss loss1 can be calculated by a cross-entropy loss function, the contrast loss loss2 can be calculated by a contrast loss function, and the total loss loss = loss1 + loss2 is taken as the total loss function of the model, and the contrast classification network is iteratively trained until the loss value satisfies the preset loss threshold (for example, less than the preset loss threshold), and the feature extraction network (for example, a resnet50 network) in the contrast classification network is taken as the target object recognition model. By combining the classification loss and the contrast learning loss, the accuracy and stability of model recognition can be greatly improved.

[0124] Step five, performing target object recognition on a to-be-recognized image using the target object recognition model to obtain a recognition result.

[0125] In an optional embodiment of the present application, before performing target object recognition on a to-be-recognized image using the target object recognition model, the following steps are further included:

[0126] Performing feature extraction on the images in the set of training images using the target object recognition model to obtain image feature vectors;

[0127] Mapping and associating the image feature vectors with preset product information, and collecting all the associated image feature vectors to obtain an image vector library.

[0128] In an optional embodiment of the present application, taking a medical kit image in the medical field as an example, after the target object recognition model is trained, the kit vector of each kit image is recalculated, and is mapped and associated with the original product information, the product information including: drug name, manufacturer, taking time, dosage, and drug category, etc. The kit vector library is formed by storing the associated kit vectors, and in kit recognition, not only the drug generic name can be recognized, but also more associated product information can be recognized.

[0129] In detail, the target object recognition on the to-be-recognized image using the target object recognition model to obtain a recognition result includes:

[0130] Performing feature extraction on the to-be-recognized image using the target object recognition model to obtain a to-be-recognized feature vector;

[0131] The image feature vector in the image vector library is retrieved by using the to-be-identified feature vector, and product information corresponding to the retrieved image feature vector is determined as the identification result.

[0132] In the embodiment of the present application, for the to-be-identified medicine box image, the target object recognition model is directly used for feature extraction, the to-be-identified medicine box vector obtained based on the feature extraction is used to retrieve the medicine box vector library, the most similar medicine box vector is found out, and the corresponding medicine information is the final identification result. At the same time, if a new medicine box image is to be added, the model does not need to be retrained, and only the vector of the new medicine box image needs to be calculated, and the medicine box vector library for retrieval needs to be updated.

[0133] The present application can improve the data quality by performing image pair enhancement processing on the images in the training image set to obtain a training image pair set. During model training, the contrast classification network is jointly trained in combination with the image classification result and the contrast learning result, so that a target object recognition model with higher recognition accuracy can be obtained. For example, in medicine box image recognition, joint training not only focuses on the classification of medicines, but also focuses on the contrast and similarity results of medicine boxes, so that the accuracy of medicine box image recognition can be improved. At the same time, the target object recognition model is used to recognize the to-be-identified image, and more associated information (such as medicine manufacturer information) can be included in the recognition result, further improving the accuracy of target object recognition. Therefore, the target object recognition device provided by the present application can improve the accuracy of target object recognition.

[0134] As shown in Figure 3 Fig. 1 is a structural schematic diagram of an electronic device for implementing the target object recognition method according to an embodiment of the present application.

[0135] The electronic device can include a processor 10, a memory 11, a communication interface 12 and a bus 13, and can further include a computer program stored in the memory 11 and executable on the processor 10, such as a target object recognition program.

[0136] The memory 11 includes at least one type of readable storage medium, such as a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 can also be an external storage device of the electronic device, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 11 can include both an internal storage unit and an external storage device of the electronic device. The memory 11 can be used to store application software installed in the electronic device and various data, such as codes of an object recognition program, etc., and can also be used to temporarily store data that has been output or will be output.

[0137] The processor 10 can be composed of an integrated circuit in some embodiments, such as a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more combinations of a central processing unit (CPU), a microprocessor, a digital processing chip, a graphics processor, and various control chips, etc. The processor 10 is a control unit of the electronic device, which connects various components of the electronic device through various interfaces and lines, and executes various functions and processes data of the electronic device by running or executing programs or modules stored in the memory 11 (such as an object recognition program, etc.) and calling data stored in the memory 11.

[0138] The communication interface 12 is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface can include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is usually used to establish a communication connection between the electronic device and other electronic devices. The user interface can be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface can also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch screen, etc. The display can also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visualized user interface.

[0139] The bus 13 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus 13 can be divided into an address bus, a data bus, a control bus, etc. The bus 13 is configured to enable connection and communication between the memory 11, the at least one processor 10, etc.

[0140] Figure 3 Only the electronic device with components is shown, and those skilled in the art can understand that, Figure 3 The structure shown does not constitute a limitation on the electronic device, and can include fewer or more components than shown, or combine certain components, or different component arrangements.

[0141] For example, although not shown, the electronic device can also include a power supply (such as a battery) to power each component. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, so that the power management device can implement functions such as charge management, discharge management, and power consumption management. The power supply can also include one or more direct current or alternating current power supplies, recharging devices, power supply fault detection circuits, power supply converters or inverters, power supply status indicators, etc. The electronic device can also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which are not described here.

[0142] Further, the electronic device can also include a network interface, which can optionally include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), and is typically used to establish a communication connection between the electronic device and other electronic devices.

[0143] Optionally, the electronic device can also include a user interface, which can be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally a standard wired interface, a wireless interface. Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, etc. The display can also be appropriately referred to as a display screen or a display unit, and is used to display information processed in the electronic device and to display a visualized user interface.

[0144] It should be understood that the embodiments are only for illustration and are not limited in the scope of the patent application by this structure.

[0145] The target object recognition program stored in the memory 11 in the electronic device is a combination of a plurality of instructions, which, when executed in the processor 10, can achieve:

[0146] Obtain a training image set, perform image pair enhancement processing on images in the training image set to obtain a training image pair set;

[0147] Classify the images in the training image pair set using a preset contrast classification network to obtain image classification results;

[0148] Perform contrast learning on the images in the training image pair set using the contrast classification network to obtain contrast learning results;

[0149] Jointly train the contrast classification network based on the image classification results and the contrast learning results to obtain a target object recognition model;

[0150] Perform target object recognition on a to-be-recognized image using the target object recognition model to obtain a recognition result.

[0151] Specifically, the specific implementation method of the processor 10 on the above instructions can refer to the description of the related steps in the corresponding embodiments of the drawings, which will not be repeated here.

[0152] Further, the modules / units integrated in the electronic device, if realized in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. The computer readable storage medium can be volatile or non-volatile. For example, the computer readable medium can include any entity or device capable of carrying the computer program code, a recording medium, a U disk, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM).

[0153] The application also provides a computer readable storage medium, which stores a computer program, and the computer program, when executed by a processor of an electronic device, can achieve:

[0154] Obtain a training image set, perform image pair enhancement processing on images in the training image set to obtain a training image pair set;

[0155] Classify the images in the training image pair set using a preset contrast classification network to obtain image classification results;

[0156] Perform contrast learning on the images in the training image pair set using the contrast classification network to obtain contrast learning results;

[0157] Based on the image classification result and the contrast learning result, the contrast classification network is jointly trained to obtain a target object recognition model.

[0158] The target object recognition model is used for target object recognition on a to-be-recognized image to obtain a recognition result.

[0159] In several embodiments provided in the present application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and the division of the modules is merely a logical function division, and there can be another division manner in actual implementation.

[0160] The modules illustrated as separated components can or can not be physically separated, and the components illustrated as modules can or can not be physical units, that is, can be located in one place, or can be distributed on a plurality of network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments.

[0161] In addition, each function module in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of hardware plus software function modules.

[0162] It is obvious for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application.

[0163] Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present application is defined by the appended claims rather than the above description, and therefore all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application. Any additional reference signs in the claims should not be considered as limiting the claims involved.

[0164] The embodiments of the present application can acquire and process related data based on artificial intelligence technology. Among them, artificial intelligence (Artificial Intelligence, AI) is to use digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. Theory, method, technology and application system.

[0165] The basic technology of artificial intelligence generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. The software technology of artificial intelligence mainly includes computer vision technology, robot technology, biometric identification technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0166] The blockchain referred to in the present application is a new application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm. The blockchain is essentially a decentralized database, which is a series of data blocks associated using cryptographic methods, each data block containing information of a batch of network transactions, for verifying the validity (anti-fake) of the information and generating the next block. The blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer, etc.

[0167] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The plurality of units or devices stated in the system claims can also be implemented by one unit or device through software or hardware. The second and the like are used to indicate the name, and do not indicate any specific order.

[0168] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit it, although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A target object recognition method characterized by comprising: The method comprises: obtaining a training image set, performing image pair enhancement processing on images in the training image set to obtain a training image pair set, comprising: sequentially selecting images in the training image set as target images, performing twice different image enhancement processing on the target images to obtain two different enhanced images, taking the two different enhanced images as an enhanced image pair of the target image, after the images in the training image set are all selected as target images, all enhanced image pairs are summarized to obtain the training image pair set; using a feature extraction layer in a preset contrast classification network to perform feature extraction on the enhanced image pairs in the training image pair set to obtain image feature pairs, using a classification layer in the contrast classification network to perform image classification on the image feature pairs to obtain image classification results; using a contrast calculation layer in the contrast classification network to calculate the similarity of the image feature pairs, taking the similarity as a contrast learning result; using the image classification results to calculate a classification loss, and using the contrast learning result to calculate a contrast loss; using the classification loss and the contrast loss to calculate a total loss, when the total loss does not satisfy a preset loss threshold, adjusting network parameters in the contrast classification network, and returning to the step of performing image pair enhancement processing on the images in the training image set until the total loss satisfies the preset loss threshold, taking the feature extraction network in the contrast classification network as a target object recognition model; using the target object recognition model to perform target object recognition on a to-be-recognized image to obtain a recognition result.

2. The object recognition method of claim 1, wherein The method further comprises: parallel processing of two preset feature extraction networks to obtain a feature extraction layer; a classification layer is connected in series after the feature extraction layer, and a contrast calculation layer is connected in series after the feature extraction layer to obtain the contrast classification network.

3. The object recognition method as claimed in claim 1, characterized in that, Before using the target object recognition model to perform target object recognition on a to-be-recognized image, the method further comprises: using the target object recognition model to perform feature extraction on images in the training image set to obtain image feature vectors; mapping and associating the image feature vectors with preset product information, and summarizing all associated image feature vectors to obtain an image vector library.

4. The object recognition method of claim 3, wherein Using the target object recognition model to perform target object recognition on a to-be-recognized image to obtain a recognition result, comprising: using the target object recognition model to perform feature extraction on a to-be-recognized image to obtain a to-be-recognized feature vector; using the to-be-recognized feature vector to retrieve image feature vectors in the image vector library, and determining product information corresponding to the retrieved image feature vectors as the recognition result.

5. An object recognition apparatus characterized by comprising: The device comprises: The image enhancement module is configured to obtain a training image set, perform image pair enhancement processing on images in the training image set, and obtain a training image pair set. The image pair enhancement processing includes: selecting an image in the training image set as a target image, performing twice different image enhancement processing on the target image to obtain two different enhanced images, taking the two different enhanced images as an enhanced image pair of the target image, and after all images in the training image set are selected as target images, collecting all enhanced image pairs to obtain the training image pair set. The model training module is configured to: extract features of the enhanced image pair in the training image pair set by using a feature extraction layer in a preset contrast classification network, perform image classification on the image feature pair by using a classification layer in the contrast classification network to obtain an image classification result, calculate a similarity of the image feature pair by using a contrast calculation layer in the contrast classification network, take the similarity as a contrast learning result, calculate a classification loss by using the image classification result, calculate a contrast loss by using the contrast learning result, calculate a total loss by using the classification loss and the contrast loss, adjust network parameters in the contrast classification network when the total loss does not satisfy a preset loss threshold, and return to the step of performing image pair enhancement processing on the images in the training image set until the total loss satisfies the preset loss threshold, and take the feature extraction network in the contrast classification network as a target object recognition model. The target object recognition module is configured to perform target object recognition on a to-be-recognized image by using the target object recognition model to obtain a recognition result. The electronic device includes: at least one processor; and a memory connected with the at least one processor in communication; wherein 6. An electronic device, comprising: the memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the target object recognition method in any one of claims 1 to 4. The computer program is executed by the processor to implement the target object recognition method in any one of claims 1 to 4. ​ ​ 7. A computer readable storage medium storing a computer program, characterized in that, ​

Images