Artificial intelligence-based image recognition method, device, equipment and medium

Abstract

The embodiment of the application relates to the technical field of artificial intelligence and intelligent medical treatment, and discloses an image recognition method, device and equipment based on artificial intelligence and a medium, the method comprising: acquiring image information and source information of an image to be recognized, determining a target image category of the image to be recognized based on the source information, and determining a target label set of the image to be recognized according to the target image category and preset recognition information; inputting the image information and the target label set into a first preset model, determining a similarity value of the image information and each label in the target label set, and obtaining a similarity set; inputting the image information into a second preset model to obtain a category number corresponding to the image to be recognized; and selecting a target similarity value according to the category number to complete the recognition of the image to be recognized. The category number of a target to be recognized in the image to be recognized is determined, and the number of target labels is determined according to the category number, so that each image to be recognized is targeted, the recognition process is more intelligent, and the effect is better.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence and intelligent medical treatment, and in particular to an image recognition method and device based on artificial intelligence, an equipment and a medium. BACKGROUND

[0002] With the development of artificial intelligence, the application of using a picture recognition model to recognize a picture is more and more extensive, which can not only be applied in life, but also can be completed by artificial intelligence for more and more work requiring image recognition. For example, when it is required to identify the food contained in a picture containing one or more foods (such as corn and rice), it can be completed by artificial intelligence; for example, in the field of intelligent medical treatment, when it is required to identify the medical instrument contained in a picture containing one or more medical instruments (such as a surgical knife and a tourniquet), it can be completed by artificial intelligence; for example, in the field of equipment maintenance, when it is required to identify the maintenance instrument contained in a picture containing one or more maintenance instruments (such as an electric pen and a screwdriver), it can be completed by artificial intelligence; and the current idea is to regard the process of recognizing the image as a target detection process.

[0003] However, in the current general target detection framework, it is required to first generate a bounding box to produce a region of interest (ROI) that may exist at present, then to select and recombine a series of candidate boxes containing targets so that each target is framed by a single box, and finally to extract features of the region of interest and then to perform subsequent classification or regression through various neural networks. However, in the prior art, since the features in the region need to be calculated explicitly, the resolution of the picture for target detection often has a high requirement. Meanwhile, the process of generating the candidate region is often very time-consuming, and the speed of training and detection is also slow, which requires high computing resources. Meanwhile, the process of generating the bounding box and selecting and recombining a series of candidate boxes often brings a series of errors, and the ROI features to represent the image feature data can be regarded as region granularity features, which may bring some noise loss. SUMMARY

[0004] Therefore, the present application provides an image recognition method and device based on artificial intelligence, an equipment and a medium, which are used to solve the problems of complicated recognition process, large amount of calculation and inaccurate recognition in the prior art.

[0005] To achieve one or part or all of the above purposes or other purposes, the present application provides an image recognition method based on artificial intelligence, comprising: acquiring image information and source information of a to-be-recognized image, determining a target image category of the to-be-recognized image based on the source information, and determining a target label set of the to-be-recognized image according to the target image category and preset recognition information, wherein the preset recognition information comprises different image categories and preset label sets corresponding to each image category;

[0006] inputting the image information and the target label set into a first preset model to determine a similarity value between the image information and each label in the target label set, obtaining a similarity set, wherein the first preset model comprises a preset encoding algorithm, an image encoding algorithm and a similarity calculation algorithm;

[0007] inputting the image information into a second preset model to obtain a category number of a to-be-recognized target in the to-be-recognized image, wherein the second preset model comprises a first basic model for acquiring deep semantic information of the to-be-recognized image and a second basic model for classification;

[0008] selecting, from the similarity set, a similarity value with the maximum value and the category number, taking the selected similarity value as a target similarity value, taking a label corresponding to the target similarity value as a target label, and completing recognition of the to-be-recognized image based on the target label.

[0009] On the other hand, the present application provides an image recognition device based on artificial intelligence, comprising:

[0010] a data acquisition module, configured to acquire image information and source information of a to-be-recognized image, determine a target image category of the to-be-recognized image based on the source information, and determine a target label set of the to-be-recognized image according to the target image category and preset recognition information, wherein the preset recognition information comprises different image categories and preset label sets corresponding to each image category;

[0011] a first calculation module, configured to input the image information and the target label set into a first preset model to determine a similarity value between the image information and each label in the target label set, and obtain a similarity set, wherein the first preset model comprises a preset encoding algorithm, an image encoding algorithm and a similarity calculation algorithm;

[0012] a second calculation module, configured to input the image information into a second preset model to obtain a category number of a to-be-recognized target in the to-be-recognized image, wherein the second preset model comprises a first basic model for acquiring deep semantic information of the to-be-recognized image and a second basic model for classification;

[0013] The identification module is configured to select, from the similarity set, similarity values with the maximum values and the number of the category number, take the selected similarity values as target similarity values, take labels corresponding to the target similarity values as target labels, and complete identification of the to-be-identified image based on the target labels.

[0014] In another aspect, the present application provides an electronic device, comprising a processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor communicates with the memory through the bus, and the machine readable instructions are executed by the processor to perform the following steps: obtaining image information and source information of a to-be-identified image, determining a target image category of the to-be-identified image based on the source information, and determining a target label set of the to-be-identified image according to the target image category and preset identification information, wherein the preset identification information comprises different image categories and preset label sets corresponding to each image category; inputting the image information and the target label set into a first preset model to determine similarity values between the image information and each label in the target label set, obtaining a similarity set, wherein the first preset model comprises a preset encoding algorithm, an image encoding algorithm and a similarity calculation algorithm; inputting the image information into a second preset model to obtain a category number of a to-be-identified target in the to-be-identified image, wherein the second preset model comprises a first base model for obtaining deep semantic information of the to-be-identified image and a second base model for classification; selecting, from the similarity set, similarity values with the maximum values and the number of the category number, taking the selected similarity values as target similarity values, taking labels corresponding to the target similarity values as target labels, and completing identification of the to-be-identified image based on the target labels.

[0015] In another aspect, the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to perform the following steps: obtaining image information and source information of a to-be-identified image, determining a target image category of the to-be-identified image based on the source information, and determining a target label set of the to-be-identified image according to the target image category and preset identification information, wherein the preset identification information comprises different image categories and a preset label set corresponding to each image category; inputting the image information and the target label set into a first preset model to determine a similarity value between the image information and each label in the target label set, to obtain a similarity set, wherein the first preset model comprises a preset encoding algorithm, an image encoding algorithm and a similarity calculation algorithm; inputting the image information into a second preset model to obtain a category number of a to-be-identified target in the to-be-identified image, wherein the second preset model comprises a first base model for obtaining deep semantic information of the to-be-identified image and a second base model for classification; selecting, from the similarity set, similarity values with the maximum similarity value and the number of the category number, taking the selected similarity values as target similarity values, taking labels corresponding to the target similarity values as target labels, and completing identification of the to-be-identified image based on the target labels.

[0016] By implementing the embodiments of the present application, the following beneficial effects can be achieved:

[0017] By acquiring image information and source information of a to-be-recognized image, determining a target image category of the to-be-recognized image based on the source information, and determining a target label set of the to-be-recognized image according to the target image category and preset recognition information, the preset recognition information including different image categories and preset label sets corresponding to each image category, the image information and the target label set are input into a first preset model to determine a similarity value of the image information and each label in the target label set, to obtain a similarity set, the first preset model including a preset encoding algorithm, an image encoding algorithm and a similarity calculation algorithm; the image information is input into a second preset model to obtain a category number of a to-be-recognized target in the to-be-recognized image, the second preset model including a first basic model for acquiring deep-level semantic information of the to-be-recognized image and a second basic model for classification; similarity values with the maximum value and the number of the category number are selected from the similarity set, the selected similarity values are taken as target similarity values, labels corresponding to the target similarity values are taken as target labels, and recognition of the to-be-recognized image is completed based on the target labels. The similarity values of the to-be-recognized image and the preset labels are obtained by encoding, avoiding complex interaction between the image and the preset labels, determining the category number of the to-be-recognized target in the to-be-recognized image, and determining the number of the target labels according to the category number. Compared with the method of directly selecting target labels according to a threshold value or uniformly cutting a certain number of target labels, the method can recognize each to-be-recognized image specifically, avoid errors caused by the method of selecting target labels according to a threshold value or uniformly cutting a certain number of target labels, for example, if the threshold value is greater than the actual category number, the target labels that do not exist in the to-be-recognized image are selected, and the method of determining the number of the target labels according to the category number is more intelligent and has better recognition effect. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, brief introductions will be given to the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0019] Among them:

[0020] Figure 1 is an application scenario of an image recognition method based on artificial intelligence provided by an embodiment of the present application;

[0021] Figure 2 is a flowchart of an image recognition method based on artificial intelligence provided by an embodiment of the present application;

[0022] Figure 3is a structural schematic diagram of an image recognition device based on artificial intelligence provided by an embodiment of the present application;

[0023] Figure 4 is a structural schematic diagram of a computer device in an embodiment of the present application;

[0024] Figure 5 is another structural schematic diagram of a computer device in an embodiment of the present application.

[0025] Figure 6 is a structural schematic diagram of an electronic device provided by an embodiment of the present application;

[0026] Figure 7 is a structural schematic diagram of a storage medium provided by an embodiment of the present application. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0028] The image recognition method based on artificial intelligence provided by the embodiments of the present application can be applied to, for example, Figure 1In an application environment, a client communicates with a server through a network. The server can obtain image information and source information of a to-be-identified image, determine a target image category of the to-be-identified image based on the source information, and determine a target label set of the to-be-identified image according to the target image category and preset identification information, wherein the preset identification information includes different image categories and a preset label set corresponding to each image category. The image information and the target label set are input into a first preset model to determine a similarity value between the image information and each label in the target label set, and a similarity set is obtained, wherein the first preset model includes a preset encoding algorithm, an image encoding algorithm, and a similarity calculation algorithm. The image information is input into a second preset model to obtain a category number of a to-be-identified target in the to-be-identified image, wherein the second preset model includes a first base model for obtaining deep semantic information of the to-be-identified image and a second base model for classification. Similarity values with the maximum value and the category number from the similarity set are selected, the selected similarity values are taken as target similarity values, a label corresponding to the target similarity values is taken as a target label, and identification of the to-be-identified image is completed based on the target label. In the present application, similarity values between a to-be-identified image and a preset label are obtained through encoding, complex interaction between an image and a preset label is avoided, a category number of a to-be-identified target in a to-be-identified image is determined, and the number of target labels is determined according to the category number. Compared with a method of directly selecting a threshold value or uniformly selecting a plurality of target labels, the present application can identify each to-be-identified image, avoid errors caused by the method of selecting a threshold value or uniformly selecting a plurality of target labels, and improve the identification effect. The client can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices. The server can be implemented by an independent server or a server cluster composed of multiple servers. The present application will be described in detail through specific embodiments.

[0029] To reduce the computing pressure of the server, the image recognition method based on artificial intelligence provided in the embodiments of the present application can also be applied in Figure 1the client in the server, that is, obtaining image information and source information of the image to be identified, determining a target image category of the image to be identified based on the source information, and determining a target label set of the image to be identified according to the target image category and preset identification information, wherein the preset identification information includes different image categories and a preset label set corresponding to each image category; inputting the image information and the target label set into a first preset model to determine a similarity value of the image information and each label in the target label set, to obtain a similarity set, wherein the first preset model includes a preset encoding algorithm, an image encoding algorithm and a similarity calculation algorithm; inputting the image information into a second preset model to obtain a category number of a target to be identified in the image to be identified, wherein the second preset model includes a first basic model for obtaining deep semantic information of the image to be identified and a second basic model for classification; selecting, from the similarity set, a similarity value with the maximum value and the number of the category number, taking the selected similarity value as a target similarity value, taking a label corresponding to the target similarity value as a target label, and completing identification of the image to be identified based on the target label.

[0030] As shown in Figure 2 The embodiment of the present application provides an image recognition method based on artificial intelligence, which comprises the following steps:

[0031] S101, obtaining image information and source information of an image to be identified, determining a target image category of the image to be identified based on the source information, and determining a target label set of the image to be identified according to the target image category and preset identification information, wherein the preset identification information includes different image categories and a preset label set corresponding to each image category;

[0032] The image recognition method provided by the embodiment of the present application can be applied to an image recognition device or an image recognition engine in various scenes. The process of image recognition is usually realized through a server. The server for image recognition can realize real-time data transmission with the client of the user. For example, when the server receives image recognition request information of the client, the image information of the image to be identified is obtained according to the image recognition request information, and the source information of the image to be identified is recorded. When the image to be identified is identified, the label corresponding to the image to be identified needs to be combined. The labels corresponding to different image categories of the image to be identified are not the same. Therefore, the image category of the image to be identified is determined according to the source information of the image to be identified. For example, when the source information of the image to be identified is the medical field, that is, the image to be identified is obtained through a camera arranged in a region belonging to the medical field or from a database of the medical field, a target label set corresponding to the medical field is selected.

[0033] Exemplarily, the preset recognition information is constructed according to different businesses and scenes corresponding to the businesses. For example, for a business of food material recognition, a first label set is constructed according to food material categories; for a business of medical instrument recognition, a second label set is constructed according to medical instrument categories; for a business of maintenance tool recognition, a third label set is constructed according to maintenance tool categories; an initial recognition information is constructed according to the first label set, the second label set and the third label set; and the initial recognition information is associated with scenes corresponding to the businesses to obtain the preset recognition information.

[0034] The source information includes identification information of an image acquisition device for acquiring the to-be-recognized image, identification information of a database for storing the to-be-recognized image, and the like. When the source information is the identification information of the database for storing the to-be-recognized image, the image acquisition device for acquiring the to-be-recognized image can be determined according to a storage rule of the database.

[0035] The target image category of the to-be-recognized image includes but is not limited to food material categories and instrument categories. The preset information includes a label set of the food material categories, i.e., a first label set, and a label set of the instrument categories, i.e., a second label set.

[0036] S102, input the image information and the target label set into a first preset model to determine a similarity value between the image information and each label in the target label set, to obtain a similarity set. The first preset model includes a preset encoding algorithm, an image encoding algorithm and a similarity calculation algorithm.

[0037] Exemplarily, the image information of the to-be-recognized image and the target label set corresponding to the to-be-recognized image are input into a first preset model. The first preset model characterizes the image information as first dimension data (embedding), and characterizes each label in the target label set as second dimension data (embedding). The similarity between the first dimension data and the second dimension data is calculated. The similarity represents the degree of association between the first dimension data and the second dimension data, i.e., the similarity represents the corresponding degree between the image information of the to-be-recognized image and each label in the target label set.

[0038] The image information of the to-be-recognized image is an actual picture of the to-be-recognized image, for example, a picture containing one or more kinds of food, a picture containing one or more kinds of medical instruments, and the like. The preset encoding algorithm includes a text encoding algorithm and an image encoding algorithm. The preset encoding algorithm and the image encoding algorithm are implemented by a preset encoder and an image encoder, respectively.

[0039] The similarity set includes a similarity value between the image information and each label in the target label set.

[0040] S103, input the image information into a second preset model to obtain a number of categories of the to-be-identified target in the to-be-identified image, the second preset model comprising a first base model for obtaining deep semantic information of the to-be-identified image and a second base model for classification;

[0041] For example, the deep semantic information of the to-be-identified image is obtained by the first base model in the second preset model, and the to-be-identified target in the to-be-identified image is classified according to the deep semantic information of the to-be-identified image by the second base model in the second preset model, thereby obtaining the number of categories of the to-be-identified target in the to-be-identified image.

[0042] Taking a picture containing food as an example, the number of categories represents the number of categories of food contained in the to-be-identified image.

[0043] The deep semantic information is the texture and color of the image content in the to-be-identified image. In this application, the deep semantic information of the to-be-identified image is the category information of the image content.

[0044] S104, selecting a target similarity value with the maximum similarity value and the number of categories from the similarity set, taking the target similarity value corresponding to the target label as the target label, and completing the identification of the to-be-identified image based on the target label.

[0045] For example, the number of categories of the to-be-identified target in the to-be-identified image determined according to the second preset model is used to select a corresponding number of target labels in the similarity set.

[0046] For example, when selecting a corresponding number of target labels in the similarity set, the similarity values in the similarity set can be arranged in descending order, and the top number of categories of similarity values are selected, for example, the number of categories is 4, and the top 4 similarity values in the similarity set are selected. The corresponding number of target labels can also be selected from the similarity set by the maximum value, for example, the target similarity with the maximum similarity value in the similarity set is selected, and after the selection is completed, the selected target similarity is removed from the similarity set, and the maximum target similarity value is selected again in the similarity set after the target similarity is removed. After the selection is completed, the selected target similarity is removed from the similarity set, and the above selection process is repeated for the number of categories, thereby obtaining the number of target similarity values.

[0047] By encoding the similarity value of the to-be-identified image and the preset label, the image and the preset label are prevented from complex interaction, the number of categories of the to-be-identified target in the to-be-identified image is determined, and the number of target labels is determined according to the number of categories. Compared with the method of directly according to a threshold value or uniformly intercepting a plurality of target labels, the method can be targeted for each to-be-identified image, and the error caused by the method of directly according to a threshold value or uniformly intercepting a plurality of target labels is avoided. For example, if the threshold value is greater than the actual number of categories, the target label that does not exist in the to-be-identified image will be selected, and the method of determining the number of target labels according to the number of categories is more intelligent, and the recognition effect is better.

[0048] In a possible implementation, the step of determining the image category of the to-be-identified image based on the source information comprises:

[0049] Determining the identification information of the target data acquisition device that acquires the to-be-identified image based on the source information;

[0050] Determining the data acquisition region corresponding to the target data acquisition device based on the identification information and preset data acquisition device distribution data, and determining the region category of the data acquisition region according to the functional department to which the data acquisition region belongs;

[0051] Taking the region category as the image category of the to-be-identified image.

[0052] For example, the to-be-identified image can be an image directly sent to the server by the data acquisition device, or an image extracted from the database. When the to-be-identified image is an image directly sent to the server by the data acquisition device, the data acquisition device is the target data acquisition device, the identification information of the target data acquisition device is obtained. When the to-be-identified image is an image extracted from the database, the attribute information of the to-be-identified image is obtained, and the identification information of the target data acquisition device that acquires the to-be-identified image is determined according to the attribute information.

[0053] For example, the position of the target data acquisition device is determined based on the identification information and preset data acquisition device distribution data, and the data acquisition region corresponding to the target data acquisition device is determined according to the position of the target data acquisition device. The preset data acquisition device distribution data includes a distribution map of the data acquisition device, an installation planning map of the data acquisition device, and the like.

[0054] For example, the area type of the data collection area is determined according to the functional department to which the data collection area belongs. For example, if the functional department to which the data collection area belongs is a department responsible for medical instrument management, the area type of the data collection area is a medical instrument type, and then the image type of the to-be-identified image is determined as a medical instrument type. If the functional department to which the data collection area belongs is a department responsible for food material management, the area type of the data collection area is a food material type, and then the image type of the to-be-identified image is determined as a food material type.

[0055] In a possible implementation, the step of determining the area type of the data collection area according to the functional department to which the data collection area belongs includes:

[0056] When there are at least two functional departments to which the data collection area belongs, target request information is obtained, and the target request information is used to initiate the step of obtaining the image information and the source information of the to-be-identified image.

[0057] According to the target request information and log data of an image recognition process, account information corresponding to the target request information is determined.

[0058] According to the account information, a target functional department to which the target request information belongs is determined.

[0059] According to the target functional department, the area type of the data collection area is determined.

[0060] For example, when there are at least two functional departments to which the data collection area belongs, for example, when the data collection area, such as a warehouse, simultaneously stores medical instruments and maintenance tools, and the functional departments to which the data collection area belongs are a medical department and a maintenance department, target request information for initiating a current image recognition process is obtained. When the target request information is initiated by account information of the medical department, the image type of the to-be-identified image generated based on the data collection area is determined as a medical instrument type. This avoids mutual influence of different departments on the same to-be-identified image, and causes inaccurate identification.

[0061] The log data is process event record data generated by a recording system. In this example, the log data is an image recognition event record generated by a system recording an image recognition process, and records an identification initiation to an identification end.

[0062] In a possible implementation, before the step of inputting the image information and the target label set into the first preset model to obtain the similarity value of the image information and each label in the target label set, the method further includes:

[0063] The data form of the target label set is obtained.

[0064] The preset encoding algorithm in the first preset model is determined according to the data form of the target label set.

[0065] Exemplarily, the data form of the label set constructed in real work can be a text form, an image form, etc., and therefore the preset encoding algorithm in the first preset model is determined according to the data form of the label set, for example, when the data form of the label set constructed in real work is a text form, a text encoding algorithm is selected as the preset encoding algorithm in the first preset model.

[0066] By selecting the preset encoding algorithm in the first preset model to adapt to the label set of different data forms, the applicability of the image recognition method described in the present application is enhanced.

[0067] In a possible implementation, the step of inputting the image information and the target label set into the first preset model to determine the similarity value of the image information and each label in the target label set to obtain a similarity set, the first preset model including a preset encoding algorithm, an image encoding algorithm and a similarity calculation algorithm, comprises:

[0068] The target label set is input into the preset encoding algorithm in the first preset model, so that the preset encoding algorithm encodes and represents each label in the target label set to obtain a first low-dimensional vector;

[0069] The image information is input into the image encoding algorithm in the first preset model, so that the image encoding algorithm encodes and represents the image information to obtain a second low-dimensional vector;

[0070] The first low-dimensional vector and the second low-dimensional vector are input into the similarity calculation algorithm in the first preset model, so that the similarity calculation algorithm calculates the cosine similarity of the first low-dimensional vector and the second low-dimensional vector;

[0071] The cosine similarity is taken as the similarity value of the image information and the label in the target label set.

[0072] Exemplarily, the first preset model adopts a large-scale image-text pre-training model (CLIP) based on contrastive learning, and the text of the target label set (labels) is encoded and represented by a text encoder into a first low-dimensional vector;

[0073] The image information, i.e., the actual picture, is encoded and represented by an image encoder into a second low-dimensional vector.

[0074] The cosine similarity of the second low-dimensional vector and the first low-dimensional vector is calculated.

[0075] For example, the first low-dimensional vector A is a low-dimensional vector with a dimension of 512, and the second low-dimensional vector B is also a low-dimensional vector with a dimension of 512. The inner product of the first low-dimensional vector and the second low-dimensional vector is calculated to obtain a value with a dimension of 1, that is, the cosine similarity, specifically:

[0076] A = [0.3, 0.4, 0.5], B = [0.2, 0.1, 0.6]

[0077] The cosine similarity of A and B is = 0.3*0.2 + 0.4*0.1 + 0.5*0.6 = 0.4.

[0078] For example, the large-scale image-text pre-training model based on contrast learning adopts an open-source Chinese-based Taiyi multi-modal model, and the preset encoding algorithm is implemented by using a text encoder with a model number of Taiyi-326M, and the image encoder used for implementing the image encoding algorithm has a model number of clip-vit-large-patch14.

[0079] In a possible implementation, before the step of inputting the image information into the second preset model to obtain the number of categories of the to-be-identified target in the to-be-identified image, the second preset model includes a first base model for obtaining deep semantic information of the to-be-identified image and a second base model for classification, the step further includes:

[0080] constructing the second base model for classification based on a full connection layer and a preset classifier;

[0081] connecting the full connection layer of the second base model with the first base model for obtaining deep semantic information of the to-be-identified image to obtain an initial model;

[0082] training the initial model according to a preset data set to obtain the second preset model.

[0083] For example, two layers of full connection layers (MLP) are used, and a classifier (softmax) is connected after the full connection layers to realize the classification function, wherein the two layers of full connection layers use an activation function (relu), and the second base model is obtained, specifically:

[0084]

[0085] wherein, W represents the weight, T represents the current round, x represents the input data, b represents the bias, and 1 and 2 represent the first layer of full connection layer and the second layer of full connection layer.

[0086] The first base model adopts a model (ViT, Vision Transformer) with an encoder, which can be trained in parallel and master global information.

[0087] In a possible implementation, the step of training the initial model according to a preset data set to obtain the second preset model comprises:

[0088] obtaining a first initial parameter of the first base model and a second initial parameter of the second base model;

[0089] obtaining data pairs obtained by different labeling methods according to a preset proportion, constructing a training set according to the data pairs, and training the second base model in the initial model based on the training set to obtain a second target parameter of the second base model;

[0090] updating the second base model according to the second target parameter, and keeping the first initial parameter of the first base model unchanged;

[0091] obtaining the second preset model based on the updated second base model and the first base model.

[0092] For example, in the training of the initial model, food material recognition is taken as an example, nearly 20,000 pictures containing food are collected from the network, the labels in the 20,000 pictures are labeled by named entity recognition (NER), and the data pairs of image-labels are obtained. The number of labels of each picture is counted. In addition, 5,000 pictures are manually labeled to obtain the number of labels, that is, a total of 25,000 pictures and the number of labels of each image are used as a training set to train the initial model.

[0093] For example, in the training process, the vision transformer part is frozen, that is, the first initial parameter of the first base model is kept unchanged, and only the second base model is trained and the parameter is updated.

[0094] For example, after two layers of neural language program learning (NLP), the final score of each label is obtained by a normalization exponential (softmax) function.

[0095] In a possible implementation, as shown in Figure 3 The application provides an image recognition device based on artificial intelligence, which comprises:

[0096] The data acquisition module 201 is configured to acquire image information and source information of a to-be-recognized image, determine a target image category of the to-be-recognized image based on the source information, and determine a target label set of the to-be-recognized image according to the target image category and preset recognition information, wherein the preset recognition information comprises different image categories and preset label sets corresponding to each image category.

[0097] The first computing module 202 is configured to input the image information and the target label set into a first preset model to determine a similarity value between the image information and each label in the target label set, and obtain a similarity set, wherein the first preset model comprises a preset encoding algorithm, an image encoding algorithm, and a similarity calculation algorithm.

[0098] The second computing module 203 is configured to input the image information into a second preset model to obtain a category number of the to-be-recognized target in the to-be-recognized image, wherein the second preset model comprises a first basic model for obtaining deep-level semantic information of the to-be-recognized image and a second basic model for classification.

[0099] The recognition module 204 is configured to select, from the similarity set, similarity values with the maximum similarity value and the category number, take the selected similarity values as target similarity values, take labels corresponding to the target similarity values as target labels, and complete recognition of the to-be-recognized image based on the target labels.

[0100] In a possible implementation, the data acquisition module 201 is configured to:

[0101] determine, based on the source information, identification information of a target data acquisition device that acquires the to-be-recognized image;

[0102] determine, based on the identification information and preset data acquisition device distribution data, a data acquisition region corresponding to the target data acquisition device, and determine a region category of the data acquisition region according to a functional department to which the data acquisition region belongs;

[0103] take the region category as an image category of the to-be-recognized image.

[0104] In a possible implementation, the data acquisition module 201 is configured to:

[0105] when the functional department to which the data acquisition region belongs has at least two, obtain target request information, wherein the target request information is used to initiate the steps of obtaining the image information and the source information of the to-be-recognized image;

[0106] determine, according to the target request information and log data of an image recognition process, account information corresponding to the target request information;

[0107] determine, based on the account information, a target functional department to which the target request information belongs;

[0108] determine, according to the target functional department, a region category of the data acquisition region.

[0109] In a possible implementation, the first computing module 202 is configured to:

[0110] obtaining a data form of the target label set;

[0111] determining a preset encoding algorithm in the first preset model for the data form of the target label set.

[0112] In a possible implementation, the first calculation module 202 is configured to:

[0113] input the target label set into a preset encoding algorithm in the first preset model, so that the preset encoding algorithm encodes and represents each label in the target label set, to obtain a first low-dimensional vector;

[0114] input the image information into an image encoding algorithm in the first preset model, so that the image encoding algorithm encodes and represents the image information, to obtain a second low-dimensional vector;

[0115] input the first low-dimensional vector and the second low-dimensional vector into a similarity calculation algorithm in the first preset model, to calculate a cosine similarity between the first low-dimensional vector and the second low-dimensional vector by using the similarity calculation algorithm;

[0116] use the cosine similarity as a similarity value between the image information and a label in the target label set.

[0117] In a possible implementation, the second calculation module 203 is configured to:

[0118] construct a second base model for classification based on a full connection layer and a preset classifier;

[0119] connect the full connection layer of the second base model with the first base model for obtaining deep-level semantic information of the image to be recognized, to obtain an initial model;

[0120] train the initial model according to a preset data set, to obtain the second preset model.

[0121] In a possible implementation, the second calculation module 203 is configured to:

[0122] obtain a first initial parameter of the first base model and a second initial parameter of the second base model;

[0123] obtain data pairs obtained by different labeling manners according to a preset proportion, construct a training set according to the data pairs, and train the second base model in the initial model based on the training set, to obtain a second target parameter of the second base model;

[0124] updating the second base model according to the second target parameter, and keeping the first initial parameter of the first base model unchanged;

[0125] based on the updated second base model and the first base model.

[0126] The application provides an image recognition device. The similarity value of a to-be-recognized image and a preset label is obtained through coding, so that the to-be-recognized image and the preset label are prevented from being complicatedly interacted, the number of categories of a to-be-recognized target in the to-be-recognized image is determined, and the number of target labels is determined according to the number of categories. Compared with a method of directly selecting target labels according to a threshold value or uniformly selecting a plurality of target labels, the method can recognize each to-be-recognized image, and avoids errors caused by the method of selecting target labels according to a threshold value or uniformly selecting a plurality of target labels. For example, if the threshold value is greater than the actual number of categories, the target labels that do not exist in the to-be-recognized image are selected, and then the method of determining the number of target labels according to the number of categories is more intelligent, and the recognition effect is better.

[0127] The specific limitations of the image recognition device can be referred to the limitations of the image recognition method in the foregoing, and will not be described herein. Each module in the image recognition device can be realized by software, hardware and a combination thereof in whole or in part. Each module can be embedded in or independent of a processor in a computer device in a hardware form, or can be stored in a memory in a computer device in a software form, so as to be called and executed by a processor to perform the operations corresponding to each module.

[0128] In an embodiment, a computer device is provided, which can be a server, and an internal structure diagram thereof can be as shown in Figure 4 The computer device includes a processor, a memory, a network interface and a database connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is configured to communicate with an external client through a network connection. The computer program is executed by the processor to implement a function or a step of a server side of an image recognition method based on artificial intelligence.

[0129] In an embodiment, a computer device is provided, which can be a client, and an internal structure diagram thereof can be as shown in Figure 5As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the client-side functions or steps of an artificial intelligence-based image recognition method.

[0130] In one possible implementation, such as Figure 6 As shown, this application embodiment provides an electronic device 300, including: a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor 320. When the processor 320 executes the computer program 311, it performs the following: acquiring image information and source information of an image to be identified; determining the target image type of the image to be identified based on the source information; and determining a target tag set of the image to be identified based on the target image type and preset identification information, wherein the preset identification information includes different image types and preset tag sets corresponding to each image type; and inputting the image information and the target tag set into a first preset model to determine the image information and the target tag set. The similarity value of each tag in the target tag set is used to obtain a similarity set. The first preset model includes a preset encoding algorithm, an image encoding algorithm, and a similarity calculation algorithm. The image information is input into a second preset model to obtain the number of categories of the target to be identified in the image to be identified. The second preset model includes a first basic model for obtaining deep semantic information of the image to be identified and a second basic model for classification. The similarity value with the largest similarity value and the number of categories is selected from the similarity set. The selected similarity value is used as the target similarity value, and the tag corresponding to the target similarity value is used as the target tag. The identification of the image to be identified is completed based on the target tag.

[0131] By encoding the similarity value between the image to be identified and the preset label, complex interactions between the image and the preset label are avoided. At the same time, the number of target categories in the image to be identified is determined, and the number of target labels is determined based on the number of categories. Compared with methods that directly use a threshold or uniformly select a number of target labels, this method can perform targeted identification for each image to be identified, avoiding the errors caused by methods that use a threshold or uniformly select a number of target labels. For example, if the threshold is greater than the actual number of categories, more target labels that do not exist in the image to be identified will be selected. Therefore, the method of determining the number of target labels based on the number of categories is more intelligent and has a better recognition effect.

[0132] In one possible implementation, such as Figure 7 As shown, this application embodiment provides a computer-readable storage medium 400 storing a computer program 411. When executed by a processor, the computer program 411 performs the following: acquiring image information and source information of an image to be identified; determining the target image type of the image to be identified based on the source information; and determining a target tag set of the image to be identified based on the target image type and preset identification information, wherein the preset identification information includes different image types and preset tag sets corresponding to each image type; and inputting the image information and the target tag set into a first preset model to determine the similarity value between the image information and each tag in the target tag set. A similarity set is obtained. The first preset model includes a preset encoding algorithm, an image encoding algorithm, and a similarity calculation algorithm. The image information is input into a second preset model to obtain the number of categories of the target to be identified in the image to be identified. The second preset model includes a first basic model for obtaining deep semantic information of the image to be identified and a second basic model for classification. The similarity value with the largest similarity value and the number of categories is selected from the similarity set. The selected similarity value is used as the target similarity value, and the label corresponding to the target similarity value is used as the target label. The identification of the image to be identified is completed based on the target label.

[0133] By encoding the similarity value between the image to be identified and the preset label, complex interactions between the image and the preset label are avoided. At the same time, the number of target categories in the image to be identified is determined, and the number of target labels is determined based on the number of categories. Compared with methods that directly use a threshold or uniformly select a number of target labels, this method can perform targeted identification for each image to be identified, avoiding the errors caused by methods that use a threshold or uniformly select a number of target labels. For example, if the threshold is greater than the actual number of categories, more target labels that do not exist in the image to be identified will be selected. Therefore, the method of determining the number of target labels based on the number of categories is more intelligent and has a better recognition effect.

[0134] The computer storage medium of the embodiments of the present application can adopt any combination of one or more computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. More specific examples (non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device.

[0135] The computer readable signal medium can include a data signal propagated in baseband or propagated as a carrier wave, in which computer readable program code is embodied. Such propagated data signals can take a wide variety of forms, including but not limited to electro-magnetic signals, optical signals, or any suitable combination thereof. Computer readable signal medium can also be any computer readable medium that is not a storage medium, that is capable of storing the program for use by or in connection with the instruction execution system, apparatus or device.

[0136] The program code embodied on the computer readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination of the above.

[0137] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, application specific circuitry, or field programmable gate array (FPGA) circuitry can execute the computer program code.

[0138] Those skilled in the art should understand that the modules or steps of the present application described above can be realized by general computing devices, which can be centralized on a single computing device or distributed on a network composed of multiple computing devices, and optionally, they can be realized by program codes executable by the computing devices, so that they can be stored in storage devices and executed by the computing devices, or they can be respectively manufactured into individual integrated circuit modules, or multiple modules or steps among them can be manufactured into a single integrated circuit module to realize. Thus, the present application is not limited to any specific combination of hardware and software.

[0139] It is noted that the above merely describes the preferred embodiments of the present application and the principles of the applied technology. Those skilled in the art will understand that the present application is not limited to the specific embodiments described above, and various obvious changes, re-adjustments and substitutions can be made by those skilled in the art without departing from the scope of the present application. Therefore, although the present application has been described in detail through the above embodiments, the present application is not limited to the above embodiments, and more other equivalent embodiments can be included without departing from the concept of the present application, and the scope of the present application is determined by the appended claims.

[0140] The above only discloses the preferred embodiments of the present application, and of course cannot limit the scope of the rights of the present application, so equivalent changes made according to the claims of the present application are still within the scope of the present application.

Claims

1. An artificial intelligence-based image recognition method, characterized by, The method comprises the following steps: obtaining image information and source information of a to-be-identified image, determining a target image category of the to-be-identified image based on the source information, and determining a target label set of the to-be-identified image according to the target image category and preset identification information, wherein the preset identification information comprises different image categories and preset label sets corresponding to each image category; inputting the image information and the target label set into a first preset model to determine a similarity value between the image information and each label in the target label set, and obtaining a similarity set, wherein the first preset model comprises a preset encoding algorithm, an image encoding algorithm and a similarity calculation algorithm; inputting the image information into a second preset model to obtain a category number of a to-be-identified target in the to-be-identified image, wherein the second preset model comprises a first basic model for obtaining deep semantic information of the to-be-identified image and a second basic model for classification; selecting, from the similarity set, similarity values with the maximum value and the number of the category number, taking the selected similarity values as target similarity values, taking labels corresponding to the target similarity values as target labels, and completing identification of the to-be-identified image based on the target labels. 2.The artificial intelligence-based image recognition method of claim 1, wherein, The step of determining the image category of the to-be-identified image based on the source information comprises the following steps: determining identification information of a target data acquisition device for acquiring the to-be-identified image based on the source information; determining a data acquisition region corresponding to the target data acquisition device based on the identification information and preset data acquisition device distribution data, and determining a region category of the data acquisition region according to a functional department to which the data acquisition region belongs; taking the region category as the image category of the to-be-identified image. 3.The AI-based image recognition method of claim 2, wherein, The step of determining the region category of the data acquisition region according to the functional department to which the data acquisition region belongs comprises the following steps: when there are at least two functional departments to which the data acquisition region belongs, obtaining target request information, wherein the target request information is used to initiate the step of obtaining the image information and the source information of the to-be-identified image; determining account information corresponding to the target request information according to the target request information and log data of an image recognition process; determining a target functional department to which the target request information belongs based on the account information; determining the region category of the data acquisition region according to the target functional department. 4.The AI-based image recognition method of claim 1, wherein Before the step of inputting the image information and the target label set into the first preset model to obtain the similarity value between the image information and each label in the target label set, the method further comprises the following steps: obtaining a data form of the target label set; determining the preset encoding algorithm in the first preset model according to the data form of the target label set. 5.The AI-based image recognition method of claim 1, wherein The step of inputting the image information and the target label set into the first preset model to determine the similarity value between the image information and each label in the target label set, and obtaining a similarity set, wherein the first preset model comprises a preset encoding algorithm, an image encoding algorithm and a similarity calculation algorithm, comprises the following steps: inputting the target label set into a preset encoding algorithm in the first preset model, so that the preset encoding algorithm encodes and represents each label in the target label set, to obtain a first low-dimensional vector; inputting the image information into an image encoding algorithm in the first preset model, so that the image encoding algorithm encodes and represents the image information, to obtain a second low-dimensional vector; inputting the first low-dimensional vector and the second low-dimensional vector into a similarity calculation algorithm in the first preset model, so that the similarity calculation algorithm calculates the cosine similarity of the first low-dimensional vector and the second low-dimensional vector; taking the cosine similarity as the similarity value of the image information and the label in the target label set. 6.The AI-based image recognition method of claim 1, wherein, Before the step of inputting the image information into the second preset model to obtain the number of categories of the to-be-identified target in the to-be-identified image, the second preset model includes a first base model for obtaining deep semantic information of the to-be-identified image and a second base model for classification, the step further includes: constructing the second base model for classification based on a full connection layer and a preset classifier; connecting the full connection layer of the second base model with the first base model for obtaining deep semantic information of the to-be-identified image to obtain an initial model; training the initial model according to a preset data set to obtain the second preset model. 7.The AI-based image recognition method of claim 6, wherein, The step of training the initial model according to a preset data set to obtain the second preset model includes: obtaining first initial parameters of the first base model and second initial parameters of the second base model; obtaining data pairs obtained by different labeling methods according to a preset proportion, constructing a training set according to the data pairs, and training the second base model in the initial model based on the training set to obtain second target parameters of the second base model; updating the second base model according to the second target parameters, and keeping the first initial parameters of the first base model unchanged; obtaining the second preset model based on the updated second base model and the first base model.

8. An artificial intelligence-based image recognition device, characterized by, The device includes: a data acquisition module configured to acquire image information and source information of a to-be-identified image, determine a target image category of the to-be-identified image based on the source information, and determine a target label set of the to-be-identified image according to the target image category and preset identification information, wherein the preset identification information includes different image categories and preset label sets corresponding to each image category; a first calculation module configured to input the image information and the target label set into a first preset model to determine similarity values of the image information and each label in the target label set, to obtain a similarity set, wherein the first preset model includes a preset encoding algorithm, an image encoding algorithm, and a similarity calculation algorithm; a second calculation module configured to input the image information into a second preset model to obtain the number of categories of a to-be-identified target in the to-be-identified image, wherein the second preset model includes a first base model for obtaining deep semantic information of the to-be-identified image and a second base model for classification. The recognition module is used to select the similarity value with the largest similarity value and the number of similarity values ​​equal to the number of categories from the similarity set, take the selected similarity value as the target similarity value, take the label corresponding to the target similarity value as the target label, and complete the recognition of the image to be recognized based on the target label.

9. An electronic device, comprising: include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the artificial intelligence-based image recognition method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the image recognition method based on artificial intelligence as described in any one of claims 1 to 7.

Images