Image recognition method

Abstract

Embodiments of the present specification provide an image recognition method, which comprises: receiving an image to be recognized, inputting the image to be recognized into a target image recognition model, and obtaining an image recognition result of the image to be recognized, wherein the target image recognition model is obtained by training an initial image recognition model according to current sample image features of a current sample image and historical sample image features of historical sample images, and the initial image recognition model is obtained by training according to the historical sample image features. Through the target image recognition model, the problem that the recognition ability of a neural network model for old types of objects will continuously decrease with the continuous increase of new types of objects, and the problem of reduced image recognition ability of the model are avoided, and the accuracy of the image recognition result is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present specification relate to the technical field of computer, in particular to an image recognition method. BACKGROUND

[0002] With the continuous development and application of image recognition technology, specific categories of objects can be accurately recognized through image recognition technology, thereby providing convenience for people's daily life. However, the image recognition technology in the prior art is mostly realized through a neural network model. In order to adapt to the increasing new category objects, the neural network model also needs to be continuously trained, so as to recognize new type objects. However, in the process of continuous learning and training of the model, as the number of new type objects increases, the recognition ability of the neural network model for old type objects will decrease, thereby causing the image recognition ability of the model to decrease, and the image recognition result to be inaccurate. SUMMARY

[0003] Therefore, the embodiments of the present specification provide an image recognition method. One or more embodiments of the present specification also relate to an image recognition model training method, another image recognition method, two image recognition devices, an image recognition model training device, a computing device, a computer readable storage medium, and a computer program, to solve the technical defects in the prior art.

[0004] According to a first aspect of the embodiments of the present specification, an image recognition method is provided, comprising:

[0005] receiving a to-be-recognized image, and inputting the to-be-recognized image into a target image recognition model to obtain an image recognition result of the to-be-recognized image, wherein the target image recognition model is obtained by training an initial image recognition model according to current sample image features of a current sample image and historical sample image features of historical sample images, and the initial image recognition model is obtained by training according to the historical sample image features.

[0006] According to a second aspect of the embodiments of the present specification, an image recognition model training method is provided, comprising:

[0007] determining a current sample image of an initial image recognition model, and performing feature extraction on the current sample image to obtain current sample image features;

[0008] obtaining historical sample image features of the initial image recognition model, wherein the historical sample image features are image features generated in the process of training the initial image recognition model according to historical sample images;

[0009] processing the historical sample image features and the current sample image features to obtain target sample image features;

[0010] training the initial image recognition model based on the target sample image features to obtain a target image recognition model.

[0011] According to a third aspect of the embodiments of the present specification, an image recognition device is provided, comprising:

[0012] An image recognition module is configured to receive an image to be recognized, and input the image to be recognized into a target image recognition model to obtain an image recognition result of the image to be recognized, wherein the target image recognition model is obtained by training an initial image recognition model based on current sample image features of a current sample image and historical sample image features of historical sample images, and the initial image recognition model is obtained by training based on the historical sample image features.

[0013] According to a fourth aspect of the embodiments of the present specification, an image recognition model training device is provided, comprising:

[0014] A first feature acquisition module is configured to determine a current sample image of an initial image recognition model, and perform feature extraction on the current sample image to obtain current sample image features;

[0015] A second feature acquisition module is configured to obtain historical sample image features of the initial image recognition model, wherein the historical sample image features are image features generated in the process of training the initial image recognition model based on historical sample images;

[0016] A feature processing module is configured to process the historical sample image features and the current sample image features to obtain target sample image features;

[0017] A model training module is configured to train the initial image recognition model based on the target sample image features to obtain a target image recognition model.

[0018] According to a fifth aspect of the embodiments of the present specification, an image recognition method is provided, applied to a cloud-side device, comprising:

[0019] Obtaining an image to be recognized sent by an edge-side device;

[0020] Inputting the image to be recognized into a target image recognition model to obtain an image recognition result of the image to be recognized, wherein the target image recognition model is obtained by training an initial image recognition model based on current sample image features of a current sample image and historical sample image features of historical sample images, and the initial image recognition model is obtained by training based on the historical sample image features.

[0021] returning the image recognition result to the terminal-side device.

[0022] According to a sixth aspect of the embodiments of the present disclosure, an image recognition device is provided, applied to a cloud-side device, comprising:

[0023] an obtaining module configured to obtain an image to be recognized sent by a terminal-side device;

[0024] an identifying module configured to input the image to be recognized into a target image recognition model to obtain an image recognition result of the image to be recognized, wherein the target image recognition model is obtained by training an initial image recognition model according to current sample image features of a current sample image and historical sample image features of historical sample images, and the initial image recognition model is obtained by training according to the historical sample image features;

[0025] a returning module configured to return the image recognition result to the terminal-side device.

[0026] According to a seventh aspect of the embodiments of the present disclosure, a computing device is provided, comprising:

[0027] a memory and a processor;

[0028] the memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions, and the computer executable instructions, when executed by the processor, implement the steps of the two image recognition methods and the image recognition model training method.

[0029] According to an eighth aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, which stores computer executable instructions, and the computer executable instructions, when executed by a processor, implement the steps of the two image recognition methods and the image recognition model training method.

[0030] According to a ninth aspect of the embodiments of the present disclosure, a computer program is provided, and when the computer program is executed in a computer, the computer program causes the computer to execute the steps of the two image recognition methods and the image recognition model training method.

[0031] The image recognition method provided by the present disclosure comprises: receiving an image to be recognized, and inputting the image to be recognized into a target image recognition model to obtain an image recognition result of the image to be recognized, wherein the target image recognition model is obtained by training an initial image recognition model according to current sample image features of a current sample image and historical sample image features of historical sample images, and the initial image recognition model is obtained by training according to the historical sample image features.

[0032] Specifically, the image recognition method provided in the specification can use the current sample image features of the current sample image and the historical sample image features of the historical sample image to process the target image recognition model obtained by training the initial image recognition model on the initial image recognition model after receiving the image to be recognized, so as to obtain an accurate image recognition result. In addition, the target image recognition model is obtained by training the initial image recognition model based on the current sample image features of the current sample image and the historical sample image features of the historical sample image, and the initial image recognition model is obtained by training based on the historical sample image features. Therefore, by using the target image recognition model, the problem that the recognition ability of the neural network model for the old type of object will decrease with the continuous increase of the new type of object, and the problem of reduced image recognition ability of the model are avoided, and the accuracy of the image recognition result is improved. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 is an application scenario diagram of an image recognition method provided by an embodiment of the specification;

[0034] Figure 2 is a flowchart of an image recognition method provided by an embodiment of the specification;

[0035] Figure 3 is a schematic diagram of model training in an image recognition method provided by an embodiment of the specification;

[0036] Figure 4 is a flowchart of an image recognition model training method provided by an embodiment of the specification;

[0037] Figure 5 is a schematic diagram of feature storage in an image recognition model training method provided by an embodiment of the specification;

[0038] Figure 6 is a schematic diagram of a fine-tuning module in an image recognition model training method provided by an embodiment of the specification;

[0039] Figure 7 is a schematic diagram of classification layer weight initialization in an image recognition model training method provided by an embodiment of the specification;

[0040] Figure 8 is a processing process flowchart of an image recognition model training method provided by an embodiment of the specification;

[0041] Figure 9 is a flowchart of another image recognition method provided by an embodiment of the specification;

[0042] Figure 10is a structural block diagram of a computing device provided by one embodiment of the present specification. DETAILED DESCRIPTION

[0043] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present specification. However, the present specification can be practiced without the specific details, other than in the examples described herein, and it is understood that the scope of the present specification is not limited to the details below.

[0044] The terminology used in one or more embodiments of the present specification is for the purpose of describing particular embodiments only and is not intended to be limiting of one or more embodiments of the present specification. As used in one or more embodiments of the present specification and the accompanying claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in one or more embodiments of the present specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0045] It will be understood that, although the terms first, second, etc. can be used herein to describe various information, these terms are not intended to denote a temporal or chronological order. Rather, these terms are used solely to distinguish one from another only. For example, without departing from the scope of one or more embodiments of the present specification, first can be termed second, and similarly, second can be termed first. Depending on the context, the word "if' as used herein can be interpreted to mean "when" or "in response to determining" or "in response to a determination."

[0046] First, the noun terms related to one or more embodiments of the present specification are explained.

[0047] Pre-trained model: A model produced after large-scale training and tuning on a representative dataset. It is usually used as an initialization model for other downstream task training to speed up training or achieve better results.

[0048] Foundation Model: A type of pre-trained model. In different fields, a model trained by a large amount of data, powerful computing power, and carefully designed structure can adapt to a series of downstream tasks.

[0049] Transformer: A network architecture, usually composed of several layers of encoders and decoders.

[0050] Vision Transformer: A network architecture, which is a migration application of the Transformer architecture in the visual field.

[0051] Transformer Block: A sub-module of a Transformer, usually composed of multi-head attention and feed-forward network.

[0052] Multi-head Attention (MHA): A sub-module in a Transformer, usually a module that performs correlation vector calculation on Query, Key, Value.

[0053] Lifelong Learning: An advanced machine learning paradigm that requires the model to learn new knowledge from successive learning tasks while not forgetting old knowledge.

[0054] Class-incremental Learning: A special form of lifelong learning, which assumes that different tasks contain different classes.

[0055] Few-shot Class-incremental Learning: Except for the first learning task, each class in the subsequent task contains very few training samples.

[0056] Catastrophic Forgetting: As the incremental process proceeds, the model's performance on old classes drops significantly.

[0057] Overfitting: The model overfits the training samples, and the classification performance on test samples is very low.

[0058] Image Recognition: Assigning corresponding class labels to different image contents.

[0059] Prototype Feature: Usually refers to the mean of all sample features of each class.

[0060] Domain: The domain formed by the sample features and their marginal probability distribution.

[0061] Old Domain: The domain formed by the old class samples.

[0062] New Domain: The domain formed by the new class samples.

[0063] Incremental Domain: The domain formed by the new and old class samples.

[0064] Incremental Domain Adaptation: Refers to the transfer from the old domain to the incremental domain.

[0065] CLIP (Contrastive Language-Image Pre-training): A pre-training model based on contrastive text-image pairs.

[0066] With the continuous development and application of image recognition technology, specific objects can be accurately recognized through image recognition technology, thereby providing convenience for people's daily life. In recent years, under the support of massive data and powerful computing power, deep models have achieved great success in the field of image recognition. However, the performance of these models is limited to the categories contained in the training set. In actual image recognition scenarios, new categories often appear in succession, which poses a great challenge to existing image recognition model methods. This challenge is often referred to as class-incremental learning, which requires the model to recognize new class objects while preventing forgetting of old class objects. However, due to privacy, device storage limitations, and high storage costs, data of old class objects are often unavailable or only a small number of samples can be retained, which makes the model prone to catastrophic forgetting problem - the performance of old class recognition drops significantly. Moreover, existing class-incremental learning methods usually assume that new class data is sufficient, but in some scenarios such as rare bird recognition and automobile part defect recognition, new bird or automobile part defect samples are often scarce, which makes some existing class-incremental learning methods perform poorly in such scenarios. Therefore, small sample class-incremental learning has emerged. In small sample class-incremental learning, training the model with a small number of samples provided by the new class can easily cause the model to overfit on the training data, resulting in a very serious catastrophic forgetting problem. To solve this problem, existing small sample class-incremental learning methods often freeze the model during the incremental stage. Moreover, to adapt the model to new class objects, these methods tend to improve the representation ability of the model as much as possible with the support of sufficient training samples provided by the first learning task. However, small sample class-incremental learning essentially defines an incremental domain adaptation problem, i.e., the arrival of new tasks causes the domain to change constantly, which requires the model to dynamically adjust the output to adapt to the incremental domain.

[0067] At the same time, at the present stage, the basic pre-training model provides strong representation ability under the support of massive data. Therefore, many scholars have designed some downstream task migration based on some existing basic pre-training models, such as CLIP, etc. In small sample incremental learning, due to the small number of new class samples, it is difficult for the model to learn a good representation in the new task learning stage so that the model can recognize the new class well. With the help of the basic pre-training model, this problem can be alleviated to some extent. However, most existing works mainly focus on the research of small sample class-incremental learning of the original model.

[0068] Based on this, in the present specification, an image recognition method is provided, the present specification also relates to an image recognition model training method, another image recognition method, two image recognition devices, an image recognition model training device, a computing device, a computer readable storage medium and a computer program, which are described in detail one by one in the following embodiments.

[0069] Referring to Figure 1 , Figure 1 A specific scenario diagram of an image recognition method according to one embodiment of the present specification is shown. Figure 1 The user 102, the front end 104, and the back end 106; wherein the user 102 can be understood as a user of a target project (such as a shopping website, a social website, etc.), an operator, a developer, etc.; the front end 104 and the back end 106 can both be understood as a server, such as a physical server or a cloud server, etc., the difference is only that the front end 104 and the back end 106 are responsible for different processing matters, and in actual application, since the front end 104 faces the user, the following is also called the front end 104 as the client.

[0070] In actual application, the front end 104 and the back end 106 constitute an image recognition system, the image recognition method is applied in the image recognition system, and all links in the process from the user side inputting the recognition image to the last execution engine returning the recognition result can be fully connected in an end-to-end manner, so that the high-frequency demand of image recognition is highly available in actual project scenarios, such as the large-scale image recognition capability provided by the image recognition method, which can be fully applied to image recognition and analysis in large social networks, shopping networks, etc., greatly improving the computing efficiency.

[0071] The specific implementation process is as follows:

[0072] In one embodiment provided by the present specification, the user 102 can collect the image of the bird through a camera or the like, and send the collected bird image to the front end 104, the front end 104 forwards the received bird image to the back end 106 for processing, the back end 106 inputs the bird image into a target image recognition model, and uses the target image recognition model to recognize the bird image to obtain the image recognition result of whether the bird is a protected bird. Then the back end 106 sends the image recognition result to the front end 104, and the front end 104 provides the image recognition result (such as a non-protected bird) to the user 102. Based on this, the image recognition method provided by the present specification can recognize the bird image through the target image recognition model, thereby obtaining an accurate image recognition result; and since the target image recognition model is obtained by training the initial image recognition model based on the current sample image features of the current sample image and the historical sample image features of the historical sample image, and the initial image recognition model is obtained according to the historical sample image features, therefore, through the target image recognition model, the problem that the recognition ability of the neural network model for the old type object will continuously decrease with the continuous increase of the new type object, and the problem of reduced image recognition ability of the model are avoided, and the accuracy of the image recognition result is improved.

[0073] Figure 2 A flowchart of an image recognition method according to one embodiment of the present specification is shown, which specifically includes the following steps.

[0074] Step 202: receiving an image to be identified, and inputting the image to be identified into a target image recognition model to obtain an image recognition result of the image to be identified. Wherein, the target image recognition model is obtained by training an initial image recognition model according to a current sample image feature of a current sample image and a historical sample image feature of a historical sample image, and the initial image recognition model is obtained by training according to the historical sample image feature.

[0075] Wherein, the image to be identified refers to an image that needs to be identified. For example, bird images, automobile parts images, etc. The image recognition method provided in the present specification can accurately identify protected birds in an e-commerce scenario through image recognition technology, avoiding the sale of protected animals; or accurately identify automobile fault parts through image recognition technology. The target image recognition model can be understood as a neural network model capable of image recognition. The image recognition result can be understood as the recognition result of the image to be identified, and the image recognition result can be a classification label. For example, a label representing whether it is a protected bird or a label representing whether it is an automobile defect part.

[0076] The current sample image can be understood as an image used as a sample to train the target image recognition model, for example, a protected bird image, an automobile defect part image, etc. It should be noted that the present solution can be applied to lifelong learning field, and provides a small sample class incremental learning method based on a pre-trained base model. Therefore, the target image recognition model is trained based on the initial image recognition model. The historical sample image can be understood as a model used as a sample to train the initial image recognition model before the current sample image. For example, a protected bird image or an automobile defect part image different from the current sample image. The initial image recognition model can be understood as a neural network model capable of image recognition. Due to the continuous increase of new category objects, it is necessary to use lifelong learning method to train the initial image recognition model, so as to adapt to the increasing new type objects, for example, the new category objects include new protected bird species and new automobile defect parts.

[0077] Specifically, the image recognition method provided in the present specification can input the received image to be identified into the trained target image recognition model, and use the target image recognition model to recognize the image to be identified, thereby obtaining an image recognition result. Wherein, the image to be identified can be an image provided by an image provider. The image provider can be a user, and the computing device can be obtained. The computing device includes but is not limited to a client, a server, a smart terminal, etc.

[0078] Further, in the embodiments provided in the specification, in the process of identifying the target image by the target image recognition model, feature extraction is first needed, and the extracted image features are classified to accurately obtain the image recognition result of the image to be identified. Specifically, the input of the image to be identified into the target image recognition model to obtain the image recognition result of the image to be identified includes:

[0079] inputting the image to be identified into the target image recognition model, using the feature extraction module in the target image recognition model to extract features of the image to be identified, and obtaining image features of the image to be identified;

[0080] using the image recognition module in the target image recognition model to recognize the image features, and obtaining the image recognition result of the image to be identified.

[0081] The feature extraction module can be understood as a network layer or an encoding layer in the target image recognition model for feature extraction. By inputting the image to be identified into the feature extraction module, the image features of the image to be identified can be obtained. It should be noted that the feature extraction module can include multiple network layers or multiple encoding layers.

[0082] The image recognition module can be understood as a network layer or an encoding layer in the target image recognition model for image recognition. In actual application, the image recognition module can be a classification layer for classifying the image to be identified. By inputting the image features into the classification layer, the classification result of the image features can be obtained. For example, by inputting the image features of a bird image into the classification layer, the classification result of whether the image is a protected bird can be obtained.

[0083] Specifically, the image recognition method provided in the specification can input the received image to be identified into the trained target image recognition model. The feature extraction module in the target image recognition model is used to extract features of the image to be identified, and the image features of the image to be identified are obtained. Then the image features are input into the image recognition module in the target image recognition model, so as to quickly and accurately obtain the image recognition result of the image to be identified.

[0084] In an embodiment provided in the specification, in view of the problems existing in the application process of the image recognition model, the method helps the model continuously learn new knowledge by means of a basic pre-training model and designs an incremental domain adaptation method. The method extracts and stores the early-stage features of the new and old task training data, and supplements the information of the early-stage features to the middle-stage features to realize the light fine-tuning of the model output, so that the model can adapt to the dynamically changing domain along with the learning task. Compared with the previous method, the scheme realizes a small amount of memory occupation while achieving a better small sample class incremental image recognition performance. Specifically, before receiving the to-be-identified image and inputting the to-be-identified image into the target image recognition model to obtain the image recognition result of the to-be-identified image, the method further includes:

[0085] determining a current sample image of an initial image recognition model, and performing feature extraction on the current sample image to obtain a current sample image feature;

[0086] obtaining a historical sample image feature of the initial image recognition model, wherein the historical sample image feature is an image feature generated in the process of training the initial image recognition model according to a historical sample image;

[0087] processing the historical sample image feature and the current sample image feature to obtain a target sample image feature;

[0088] training the initial image recognition model based on the target sample image feature to obtain a target image recognition model.

[0089] wherein the initial image recognition model can be understood as the pre-training model or the basic model.

[0090] Specifically, in the image recognition method provided in the specification, before image recognition, the initial image recognition model needs to be trained based on the current sample image features of the current sample image and the historical sample image features of the historical sample image, so as to obtain the target image recognition model. The training process first determines the current sample image for the initial image recognition model. The current sample image can be understood as the image of the new type object described above. After determining the current sample image, the current sample image is input into the initial image recognition model, and the feature extraction module in the initial image recognition model is used to extract features of the current sample image, so as to obtain the current sample image features of the current sample image. Then, the historical sample image features of the initial image recognition model are obtained from the database. It should be noted that the image recognition method provided in the specification will store the sample image features into the database when extracting the sample image features in the process of training the model, so as to facilitate subsequent training of the model based on the sample image features, and avoid the problem that the recognition ability of the neural network model for the old type object will continuously decrease with the continuous increase of the new type object, and the problem of the decrease of the image recognition ability of the model. Therefore, in the process of training the initial image recognition model by using the historical sample image, the historical sample image features of the historical sample image are stored into the database.

[0091] After obtaining the historical sample image features, the feature processing module in the initial image recognition model is used to fuse the historical sample image features and the current sample image features, to obtain target sample image features. Finally, the target sample image features are used to train the initial image recognition model, to obtain the target image recognition model.

[0092] It should be noted that the application scenarios of the image recognition method provided in the specification include many scenarios where new class data is scarce and incremental recognition is required. For example, the application of the image recognition method provided in the specification in the automobile defect part recognition scenario, wherein the model training process in the image recognition method provided in the specification can be referred to Figure 3 . The initial sample recognition model is a pre-trained basic model, and the training data is an automobile defect part image. Figure 3The task in the method can be understood as a learning task for various new types of objects. For example, a car defective part recognition task. In the car production line, defective parts have a great impact on the quality of the car. However, in actual industrial production, there are very few examples of defective parts, and over time, some new types of defective parts may appear due to machine failure, aging, and other reasons, and the model needs to be able to incrementally recognize these to maintain the quality of car production. Therefore, first, the early stage network layer of the model is used to extract the current task training data to obtain the early stage features of the current task training data, and the early stage features are stored. Then, the early stage features are extracted by the middle stage network layer of the model to obtain the middle stage features, and the historical early stage features stored in the database are used to fine-tune the current middle stage features to obtain training features. Finally, the training features output by the training data are used to initialize the weight of the classification layer of the model, and the initialized weight is used to classify the test data output features. Thus, the training of the pre-trained base model is completed.

[0093] The image recognition method provided in the specification receives a to-be-identified image, and uses the current sample image features of the current sample image and the historical sample image features of the historical sample image to process the target image recognition model obtained by training the initial image recognition model, thereby obtaining an accurate image recognition result. Moreover, the target image recognition model is trained based on the current sample image features of the current sample image and the historical sample image features of the historical sample image, and the initial image recognition model is trained based on the historical sample image features. Therefore, through the target image recognition model, the problem that the recognition ability of the neural network model for the old type of object will decrease continuously with the continuous increase of the new type of object, and the problem of reduced image recognition ability of the model are avoided, and the accuracy of the image recognition result is improved.

[0094] Figure 4 A flowchart of an image recognition model training method according to an embodiment of the specification is shown, which specifically includes the following steps.

[0095] Step 402: Determine the current sample image of the initial image recognition model, and perform feature extraction on the current sample image to obtain the current sample image features.

[0096] In an embodiment provided in the specification, the feature extraction on the current sample image to obtain the current sample image features includes:

[0097] The current sample image is input into the initial image recognition model, and the feature extraction module in the initial image recognition model is used to perform feature extraction on the current sample image to obtain the current sample image features of the current sample image.

[0098] Specifically, the image recognition method provided in the specification can input a current sample image into an initial image recognition model. The current sample image is subjected to feature extraction by a feature extraction module in the initial image recognition model to obtain current sample image features of the current sample image. In an embodiment provided in the specification, the feature extraction module includes at least two feature extraction modules. Based on this, the inputting of the current sample image into the initial image recognition model and the feature extraction of the current sample image by the feature extraction module in the initial image recognition model to obtain the current sample image features of the current sample image include:

[0099] inputting the current sample image into the initial image recognition model and performing feature extraction on the current sample image by a first feature extraction module in the initial image recognition model to obtain first sample image features;

[0100] performing feature extraction on the first sample image features by a second feature extraction module in the initial image recognition model to obtain the current sample image features of the current sample image.

[0101] In an embodiment provided in the specification, after the inputting of the current sample image into the initial image recognition model and the feature extraction of the current sample image by the first feature extraction module in the initial image recognition model to obtain the first sample image features, the method further includes:

[0102] calculating mean features of the first sample image features and storing the mean features.

[0103] In the above example, Figure 5 is a schematic diagram of feature storage in an image recognition model training method provided in an embodiment of the specification, referring to Figure 5 It can be known that the image processing model training method provided in the specification inputs training data into an early-stage network coding layer (a first feature extraction module) to obtain early-stage features of training data of various categories, then calculates mean values of the early-stage features of the training data of various categories, and finally stores the mean values of the early-stage features in a feature library. The feature library can be understood as a database for storing features, which is convenient for use when the model is trained next time. Then, the training data and test data of each task are sequentially input into the early-stage and middle-stage network layers, and then the middle-stage features are obtained.

[0104] Step 404: Obtain historical sample image features of the initial image recognition model. The historical sample image features are image features generated in the process of training the initial image recognition model according to historical sample images.

[0105] Specifically, the image recognition model training method provided in the specification will obtain historical sample image features generated in the training process of the initial image recognition model from the database.

[0106] In the above example, the mean features stored in the database during the training process of the image recognition model are obtained from the database.

[0107] Step 406: processing the historical sample image features and the current sample image features to obtain target sample image features.

[0108] Specifically, in an embodiment provided in the specification, in order to enable the image recognition model to process each incremental task that comes in succession without reducing the model performance, the saved early-stage features are injected into the encoder stage features by the fine-tuning module to achieve lightweight fine-tuning of the model output. Specifically, the processing of the historical sample image features and the current sample image features to obtain target sample image features includes:

[0109] The feature processing module in the initial image recognition model is used to associate the historical sample image features and the current sample image features to obtain target sample image features.

[0110] The feature processing module can be understood as a module that can adjust the current sample image features using the historical sample image features. The feature processing module can be the fine-tuning module that can inject the saved early-stage features into the middle-stage features. The feature processing module can be a Transformer block. Therefore, this method uses a single-head Transformer block as the fine-tuning module, and in specific implementation, the middle-stage features are Q, and the saved early-stage features are K and V. Figure 6 is a schematic diagram of a fine-tuning module in an image recognition model training method provided in an embodiment of the specification, referring to Figure 6 It can be seen that the middle-stage features Q are input into the Transformer block, and the Transformer block fine-tunes the middle-stage features Q using the early-stage features K and V obtained from the database M to obtain fine-tuned features.

[0111] Further, in an embodiment provided in the specification, the use of the feature processing module in the initial image recognition model to associate the historical sample image features and the current sample image features to obtain target sample image features includes:

[0112] The feature processing module in the initial image recognition model is used to take the current sample image feature as a query condition to calculate a similarity coefficient of the historical sample image feature and the query condition.

[0113] The current sample image feature is weighted by using the similarity coefficient to obtain a weighted feature, and the weighted feature is added to the current sample image feature to obtain a target sample image feature.

[0114] The similarity coefficient refers to a coefficient for representing the similarity between the current sample image feature as a query condition and the historical sample image feature.

[0115] In the above example, the image recognition model training method provided in the specification is used. After obtaining the middle-stage feature, the fine-tuning module takes the middle-stage feature of the training data as a query statement, calculates a similarity coefficient of the query statement and the stored early-stage feature mean, then weights the stored early-stage feature by using the similarity coefficient, and finally adds the weighted result to the query statement to obtain a training feature.

[0116] Step 408: training the initial image recognition model based on the target sample image feature to obtain a target image recognition model.

[0117] In an embodiment provided in the specification, for the process of training a neural network model by using lifelong learning, the recognition ability of the model for old types of objects will continuously decrease, resulting in a decrease in the image recognition ability of the model. The method uses the adjusted feature for model training and testing, thereby ensuring that the problem of the decrease in the image recognition ability of the model is avoided. Specifically, the training of the initial image recognition model based on the target sample image feature to obtain a target image recognition model includes steps one and two.

[0118] Step one: generating an update weight based on the target sample image feature, and updating the original weight of the initial image recognition model by using the update weight to obtain a to-be-tested image recognition model.

[0119] Specifically, the generation of an update weight based on the target sample image feature and the updating of the original weight of the initial image recognition model by using the update weight to obtain a to-be-tested image recognition model include:

[0120] The target sample image feature is input into a third feature extraction module of the initial image recognition model to obtain a model training feature.

[0121] A feature mean of the model training feature is calculated, and the feature mean is taken as an update weight.

[0122] Determine the original weight of the image recognition module in the initial image recognition model, and replace the original weight with the updated weight to obtain a to-be-tested image recognition model.

[0123] The third feature extraction module can be understood as a network layer or an encoding layer for extracting features.

[0124] In the above example, Figure 7 is a schematic diagram of classification layer weight initialization in an image recognition model training method provided by an embodiment of the present specification, based on Figure 7 It can be known that after obtaining the training features and the test features, they are input to the network encoding layer in the later stage to obtain the features in the later stage. The classification layer weight is initialized by obtaining the feature mean values of the training data in each category, thereby completing the parameter adjustment of the image recognition model, so that the model can process each incremental task in succession, and the performance of the model is stable. For example, the original classification layer weight dimension is D, and the new task includes 5 categories. Then, 5 feature vectors are randomly initialized, and then the feature mean values of the training data corresponding to the 5 categories are obtained. Then, the features are initialized using these mean values, and the initialized features are connected with the previous classification layer weight, so that the 5 feature vectors are replaced using the feature mean values in the later stage, and the new classification layer weight dimension is N*D. Wherein, N represents the number of categories of the new task; D represents the dimension of the weight.

[0125] Step two: based on the current test image corresponding to the current sample image, model testing is performed on the to-be-tested image recognition model, and in a case where the model testing is passed, the to-be-tested image recognition model is determined as a target image recognition model.

[0126] Specifically, based on the current test image corresponding to the current sample image, model testing is performed on the to-be-tested image recognition model, and in a case where the model testing is passed, the to-be-tested image recognition model is determined as a target image recognition model, including:

[0127] Input the current test image corresponding to the current sample image into the to-be-tested image recognition model to obtain a test image recognition result of the current test image.

[0128] In a case where the test image recognition result meets a preset test passing condition, the to-be-tested image recognition model is determined as a target image recognition model.

[0129] With the above example, after the classification layer weight initialization of the model, the test data corresponding to the training sample is determined, and the test sample data is input into the image recognition model after weight initialization to obtain the classification result of the test sample data. Based on the sample label of the test sample data and the classification result, the loss function of the image recognition model is calculated. In the case of loss function convergence, the test of the model is determined to pass, thereby obtaining the trained target image recognition model, which is convenient for subsequent accurate recognition of the to-be-recognized image based on the target image recognition model.

[0130] It should be noted that the image recognition model training method provided in the specification is an incremental domain adaptation method for solving small sample class incremental learning. With the help of the basic pre-training model, the early stage features of the training data are extracted and stored, and the saved early stage features are injected into the encoder through the fine-tuning module to realize lightweight fine-tuning of the model output, so that the model can process each incremental task in succession. Based on this, in order to ensure the smooth execution of model training, the fine-tuning module needs to be trained in advance. Specifically, before the target sample image feature is obtained by using the feature processing module in the initial image recognition model to associate the historical sample image feature and the current sample image feature, it further includes:

[0131] determining the training sample corresponding to the feature processing module, and training the feature processing module using the training sample to obtain the trained feature processing module.

[0132] Specifically, the image recognition model training method provided in the specification proposes a pseudo-incremental knowledge injection training strategy for the fine-tuning module. Since the data in the incremental stage is small and the old data is not available, it is difficult to train the fine-tuning module in the incremental stage. Therefore, in order to learn the parameters of the fine-tuning module, a pseudo-incremental knowledge injection strategy is designed. The pseudo-incremental knowledge injection strategy relies on old class sample data and trains the model by simulating the actual incremental process. Based on this, the determination of the training sample corresponding to the feature processing module and the training of the feature processing module using the training sample to obtain the trained feature processing module includes:

[0133] determining the training sample corresponding to the feature processing module, and dividing the training sample into a first training sample and a second training sample;

[0134] training the feature processing module based on the first training sample to obtain a first loss function;

[0135] training the feature processing module based on the first training sample and the second training sample data to obtain a second loss function;

[0136] Train the feature processing module based on the first loss function and the second loss function to obtain a trained feature processing module.

[0137] In the above example, the method divides the old class sample data into two parts: a pseudo basic set and a pseudo incremental set. The pseudo incremental set contains only a small amount of training data for each class, such as 1 training sample. Then, the early-stage network encoding layer features of the training data in the pseudo basic set are extracted and used as pseudo old class knowledge. At the same time, the early-stage network encoding layer features of the training data in the pseudo incremental set are extracted and used as pseudo new class knowledge. After determining the pseudo old class knowledge and the pseudo new class knowledge, the pseudo new class knowledge is used to inject knowledge into the pseudo incremental set data, and the output of the model to the pseudo incremental set data is adjusted. Finally, the prototype features of the pseudo incremental set training data are calculated and used as classification weights to classify the adjusted model output of the pseudo incremental set test data, and then the cross-entropy is used to calculate loss 1 (i.e., the first loss function).

[0138] After determining the first loss function, the pseudo new class knowledge and the pseudo old class knowledge are combined as pseudo full knowledge, and the pseudo incremental set and the pseudo basic set are combined as a pseudo full set. Then, the pseudo full knowledge is used to inject knowledge into the pseudo full set data, and the output of the model to the pseudo full set data is adjusted. Finally, the prototype features of the pseudo full set training data are calculated and used as classification weights to classify the adjusted model output of the pseudo full set test data, and then the cross-entropy is used to calculate loss 2 (i.e., the second loss function).

[0139] The model is optimized by combining the cross-entropy calculation of loss 1 and the cross-entropy calculation of loss 2 until the training stopping condition is reached, and a trained fine-tuning module is obtained.

[0140] The image recognition model training method provided by the present specification trains an initial image recognition model based on the current sample image features of the current sample image and the historical sample image features of the historical sample image, thereby obtaining a target image recognition model with high stability and accuracy, avoiding the problem that as new types of objects increase, the recognition ability of the neural network model for old types of objects will continuously decrease, and the problem of reduced image recognition ability of the model, and improving the accuracy of the image recognition result.

[0141] The above is a schematic scheme of an image recognition model training method of the present embodiment. It should be noted that the technical scheme of the image recognition model training method belongs to the same concept as the technical scheme of the image recognition method described above, and the details of the image recognition model training method that are not described in detail can be referred to the corresponding or corresponding parts of the description of the image recognition method technical scheme. Therefore, the details of the image recognition method technical scheme that are not described in detail can be referred to the corresponding or corresponding parts of the description of the image recognition model training method technical scheme.

[0142] The following description is combined with the accompanying Figure 8 The image recognition model training method provided in the specification is further described taking the application of the image recognition model training method in a small sample class incremental learning scenario based on a pre-trained base model as an example. Among them, Figure 8 A processing process flowchart of an image recognition model training method provided by an embodiment of the specification is shown, which specifically includes the following steps.

[0143] Step 802: In the model training process, first, the training data and test data of each task are input into the early stage network layer of the image recognition model for feature extraction, to obtain the early stage features of the training data and test data.

[0144] It should be noted that before model training, various categories of training data need to be input into the early stage network coding layer of the image recognition model to obtain the early stage features of the training data of each category, then the mean value of the early stage features of each category of training data is calculated respectively, and finally the mean value is stored in the feature library.

[0145] Step 804: The early stage features of the training data and test data are input into the middle stage network layer of the image recognition model for feature extraction, to obtain the middle stage features of the training data and test data.

[0146] Step 806: The middle stage features of the training data and the middle stage features of the test data are input into the fine-tuning module for feature fine-tuning.

[0147] The fine-tuning process for the middle stage features of the training data is as follows: the fine-tuning module takes the middle stage features of the training data as a query statement, calculates the similarity coefficient of the query statement and the stored early stage feature mean value, then weights the stored early stage features using the similarity coefficient, and finally adds the weighted result to the query statement to obtain the training features.

[0148] The fine-tuning process for the middle stage features of the test data is as follows: the fine-tuning module takes the middle stage features of the test data as a query statement, calculates the similarity coefficient of the query statement and the stored early stage feature mean value, then weights the stored early stage features using the similarity coefficient, and finally adds the weighted result to the query statement to obtain the test features.

[0149] Step 808: The obtained training features and test features are input into the late stage network coding layer of the image recognition network to obtain the late stage features of the training data and test data.

[0150] Step 810: determine the post-stage features of the training data of each task, and calculate the mean of the post-stage features respectively; replace the classification layer weight with the mean, so as to initialize the classification layer weight by the mean. After the initialization of the classification layer weight, the post-stage features of the test data are input into the classification layer for classification, and the classification result of the test data is obtained. Based on the classification result and the label of the test data, the loss function of the image recognition model is calculated, and the model is trained based on the loss function until the training stopping condition is reached.

[0151] It should be noted that, since the feature needs to be adjusted by the fine-tuning module in the training process of the image recognition model, the fine-tuning module needs to be trained by samples before the model training, so as to ensure the smooth progress of the subsequent training process of the image recognition model. The fine-tuning module is a Transformer block, and the training process of the fine-tuning module is as shown in the following steps.

[0152] Step one: divide the old class training data into two parts: pseudo base set and pseudo incremental set. Among them, each class in the pseudo incremental set contains only a small amount of training data, such as 1 sample; each class in the pseudo base set contains more training data.

[0153] Step two: extract the early-stage network encoding layer features of the pseudo base set training data by using the early-stage network encoding layer in the image recognition model, and take them as pseudo old class knowledge. At the same time, extract the early-stage network encoding layer features of the pseudo incremental set training data, and take them as pseudo new class knowledge.

[0154] Step three: inject knowledge into the pseudo incremental set data through the pseudo new class knowledge, and adjust the output of the fine-tuning module for the pseudo incremental set data.

[0155] Step four: extract the prototype features of the pseudo incremental set training data by the early-stage and middle-stage network layers of the image recognition model. Take the prototype features as classification weight to classify the adjusted output of the fine-tuning module for the test data, and then calculate the loss 1 by using cross entropy

[0156] Step five: combine the pseudo new class knowledge and the pseudo old class knowledge as pseudo full knowledge.

[0157] Step six: combine the pseudo incremental set and the pseudo base set as a pseudo full set.

[0158] Step seven: inject knowledge into the pseudo full set data through the pseudo full knowledge, and adjust the output of the fine-tuning module for the pseudo full set data.

[0159] Step eight: extract the prototype features of the pseudo-full set training data through the early and middle network layers of the image recognition model. The prototype features are used as classification weights to classify the adjusted model output of the pseudo-full set test data, and then the cross-entropy is used to calculate the loss 2

[0160] Step nine: optimize the model by combining the loss 1 obtained in step four and the loss 2 obtained in step eight, and obtain the trained fine-tuning module (Transformer block).

[0161] Based on the above, the method designs an incremental domain adaptation method for solving small sample class incremental learning. With the help of the basic pre-training model, the early stage features of the training data are extracted and stored. The saved early stage features are injected into the middle stage of the encoder through the fine-tuning module to realize the lightweight fine-tuning of the model output, so that the model can process each incremental task that comes in succession. In order to learn the training parameters of the fine-tuning module, a matching pseudo-incremental knowledge injection training strategy is designed. By sampling the old data, a pseudo-base set and a pseudo-incremental set are constructed to simulate the actual incremental process and optimize the model.

[0162] Figure 9 A flowchart of another image recognition method provided according to an embodiment of the present specification is shown, which is applied to a cloud-side device and specifically includes the following steps.

[0163] Step 902: obtaining an image to be identified sent by an end-side device;

[0164] Step 904: inputting the image to be identified into a target image recognition model to obtain an image recognition result of the image to be identified.

[0165] Wherein, the target image recognition model is obtained by training an initial image recognition model according to current sample image features of a current sample image and historical sample image features of historical sample images, and the initial image recognition model is obtained by training according to the historical sample image features;

[0166] Step 906: returning the image recognition result to the end-side device.

[0167] The cloud-side device can be one or more servers, one or more hosts. In an embodiment provided in the specification, the cloud-side device can also be composed of a cloud-side computing device and / or a cloud-side storage device. The cloud-side computing device can be one or more servers. The cloud-side storage device can be one or more database servers, cloud disks, etc. The end-side device can be a client, a terminal, a computer, a server, a mobile phone, a smart mobile device, etc.

[0168] Specifically, another image recognition method provided in the specification can be applied to the cloud-side device. The image to be recognized sent by the end-side device can be obtained, and the trained target image recognition model can be used to perform image recognition processing on the image to be recognized to obtain the image recognition result of the image to be recognized, and the image recognition result of the image to be recognized can be sent to the end-side device.

[0169] For example, taking the end-side device as a smart phone and the cloud-side device as a cloud server as an example, the another image recognition method is described. Specifically, in actual application, the user can send the collected bird image to the cloud server through the smart phone. After receiving the bird image, the cloud server inputs the bird image into the target image recognition model, and uses the target image recognition model to recognize the bird image to obtain the image recognition result of whether the bird is a protected bird. After determining the image recognition result of the bird image, the cloud server can return the image recognition result to the smart phone, and the smart phone can display the image recognition result of the bird image to the user.

[0170] Based on this, the application provides another image recognition method applied to a cloud device. After obtaining a to-be-recognized image sent by an end-side device, the current sample image feature of a current sample image and the historical sample image feature of a historical sample image are used to process the to-be-recognized image by a target image recognition model obtained by training an initial image recognition model, so as to obtain an accurate image recognition result, and the accurate image recognition result is returned to the end-side device, thereby saving the computing resources of the end-side device. Moreover, the target image recognition model is obtained by training the initial image recognition model based on the current sample image feature of the current sample image and the historical sample image feature of the historical sample image, and the initial image recognition model is obtained by training based on the historical sample image feature. Therefore, by using the target image recognition model, the problem that the recognition ability of a neural network model for an old type of object will continuously decrease with the continuous increase of new types of objects, and the problem of reduced image recognition ability of the model are avoided, and the accuracy of the image recognition result is improved.

[0171] Corresponding to the method embodiments, the specification also provides image recognition device embodiments. The device includes:

[0172] An image recognition module is configured to receive a to-be-recognized image, input the to-be-recognized image into a target image recognition model, and obtain an image recognition result of the to-be-recognized image. The target image recognition model is obtained by training an initial image recognition model based on a current sample image feature of a current sample image and a historical sample image feature of a historical sample image, and the initial image recognition model is obtained by training based on the historical sample image feature.

[0173] Optionally, the image recognition module is further configured to:

[0174] input the to-be-recognized image into the target image recognition model, perform feature extraction on the to-be-recognized image by using a feature extraction module in the target image recognition model, and obtain an image feature of the to-be-recognized image;

[0175] perform image recognition on the image feature by using an image recognition module in the target image recognition model, and obtain an image recognition result of the to-be-recognized image.

[0176] Optionally, the image recognition device further includes a model training module configured to:

[0177] determine a current sample image of an initial image recognition model, perform feature extraction on the current sample image, and obtain a current sample image feature;

[0178] obtaining historical sample image features of the initial image recognition model, wherein the historical sample image features are image features generated in a process of training the initial image recognition model according to historical sample images;

[0179] processing the historical sample image features and the current sample image features to obtain target sample image features;

[0180] training the initial image recognition model based on the target sample image features to obtain a target image recognition model.

[0181] The image recognition device provided in the specification can use the current sample image features of the current sample image and the historical sample image features of the historical sample image to train the target image recognition model obtained from the initial image recognition model after receiving the image to be recognized, and then process the image to be recognized by the target image recognition model, so as to obtain an accurate image recognition result. In addition, the target image recognition model is obtained by training the initial image recognition model based on the current sample image features of the current sample image and the historical sample image features of the historical sample image, and the initial image recognition model is obtained by training based on the historical sample image features. Therefore, by using the target image recognition model, the problem of continuously decreasing recognition ability of the neural network model for old types of objects and the problem of decreasing image recognition ability of the model can be avoided as new types of objects are continuously added, and the accuracy of the image recognition result is improved.

[0182] The above is a schematic scheme of an image recognition device according to the embodiment. It should be noted that the technical scheme of the image recognition device belongs to the same concept as the technical scheme of the image recognition method described above, and the details of the technical scheme of the image recognition device that are not described in detail can be referred to the description of the technical scheme of the image recognition method.

[0183] Corresponding to the method embodiments described above, the specification also provides an image recognition model training device embodiment, which comprises:

[0184] The first feature acquisition module is configured to determine a current sample image of an initial image recognition model, and to extract features from the current sample image to obtain current sample image features;

[0185] The second feature acquisition module is configured to obtain historical sample image features of the initial image recognition model, wherein the historical sample image features are image features generated in a process of training the initial image recognition model according to historical sample images;

[0186] The feature processing module is configured to process the historical sample image features and the current sample image features to obtain target sample image features;

[0187] a model training module configured to train the initial image recognition model based on the target sample image feature, and obtain a target image recognition model.

[0188] Optionally, the first feature acquisition module is further configured to:

[0189] input the current sample image into the initial image recognition model, perform feature extraction on the current sample image by using a feature extraction module in the initial image recognition model, and obtain a current sample image feature of the current sample image.

[0190] Optionally, the feature processing module is further configured to:

[0191] perform feature correlation on the historical sample image feature and the current sample image feature by using a feature processing module in the initial image recognition model, and obtain a target sample image feature.

[0192] Optionally, the feature processing module is further configured to:

[0193] perform feature correlation on the historical sample image feature and the current sample image feature by using a feature processing module in the initial image recognition model, and obtain a target sample image feature.

[0194] weight the current sample image feature by using the similarity coefficient, obtain a weighted feature, and add the weighted feature and the current sample image feature to obtain a target sample image feature.

[0195] Optionally, the model training module is further configured to:

[0196] generate an update weight based on the target sample image feature, update an original weight of the initial image recognition model by using the update weight, and obtain a to-be-tested image recognition model;

[0197] perform model testing on the to-be-tested image recognition model based on a current test image corresponding to the current sample image, and determine the to-be-tested image recognition model as a target image recognition model in a case where the model testing is passed.

[0198] Optionally, the model training module is further configured to:

[0199] input the target sample image feature into a third feature extraction module of the initial image recognition model, and obtain a model training feature;

[0200] calculate a feature mean of the model training feature, and use the feature mean as an update weight;

[0201] Determine original weights of the image recognition modules in the initial image recognition model, and replace the original weights with updated weights to obtain a to-be-tested image recognition model.

[0202] Optionally, the model training module is further configured to:

[0203] Input the current test image corresponding to the current sample image into the to-be-tested image recognition model to obtain a test image recognition result of the current test image.

[0204] In a case where the test image recognition result meets a preset test passing condition, the to-be-tested image recognition model is determined as a target image recognition model.

[0205] Optionally, the image recognition model training apparatus further comprises a feature processing module training module configured to:

[0206] Determine training samples corresponding to the feature processing module, and train the feature processing module using the training samples to obtain a trained feature processing module.

[0207] Optionally, the feature processing module training module is configured to:

[0208] Determine training samples corresponding to the feature processing module, and divide the training samples into first training samples and second training samples.

[0209] Train the feature processing module based on the first training samples to obtain a first loss function.

[0210] Train the feature processing module based on the first training samples and the second training samples to obtain a second loss function.

[0211] Train the feature processing module based on the first loss function and the second loss function to obtain a trained feature processing module.

[0212] The image recognition model training apparatus provided in the specification is trained based on current sample image features of current sample images and historical sample image features of historical sample images, so that a target image recognition model with high stability and high accuracy is obtained, and problems such as continuous decline of the recognition ability of a neural network model for old types of objects and decline of the image recognition ability of a model are avoided, and the accuracy of an image recognition result is improved.

[0213] The above is a schematic scheme of the image recognition model training device of the embodiment. It should be noted that the technical scheme of the image recognition model training device and the technical scheme of the image recognition model training method described above belong to the same concept, and the details of the technical scheme of the image recognition model training device that are not described in detail can be referred to the description of the technical scheme of the image recognition model training method described above.

[0214] Corresponding to the method embodiment described above, the specification also provides another image recognition device embodiment. The image recognition device is applied to a cloud side device and includes:

[0215] The acquisition module is configured to acquire a to-be-recognized image sent by an end side device.

[0216] The recognition module is configured to input the to-be-recognized image into a target image recognition model to obtain an image recognition result of the to-be-recognized image, wherein the target image recognition model is obtained by training an initial image recognition model according to a current sample image feature of a current sample image and a historical sample image feature of a historical sample image, and the initial image recognition model is obtained by training according to the historical sample image feature.

[0217] The return module is configured to return the image recognition result to the end side device.

[0218] The image recognition device applied to the cloud side device provided by the specification is used to process the to-be-recognized image by using the target image recognition model obtained by training the initial image recognition model according to the current sample image feature of the current sample image and the historical sample image feature of the historical sample image after acquiring the to-be-recognized image sent by the end side device, so as to obtain an accurate image recognition result, and the accurate image recognition result is returned to the end side device, thereby saving the computing resources of the end side device. In addition, since the target image recognition model is obtained by training the initial image recognition model based on the current sample image feature of the current sample image and the historical sample image feature of the historical sample image, and the initial image recognition model is obtained by training according to the historical sample image feature, the problem that the recognition ability of the neural network model for the old type object will continuously decrease with the continuous increase of the new type object, and the problem of the reduced image recognition ability of the model are avoided through the target image recognition model, thereby improving the accuracy of the image recognition result.

[0219] The above is a schematic scheme of another image recognition device of the embodiment. It should be noted that the technical scheme of the another image recognition device and the technical scheme of the another image recognition method described above belong to the same concept, and the details of the technical scheme of the another image recognition device that are not described in detail can be referred to the description of the technical scheme of the another image recognition method described above.

[0220] Figure 10 A structural block diagram of a computing device 1000 is shown, which is provided according to one embodiment of the present specification. The components of the computing device 1000 include, but are not limited to, a memory 1010 and a processor 1020. The processor 1020 is connected with the memory 1010 through a bus 1030, and a database 1050 is used to save data.

[0221] The computing device 1000 further includes an access device 1040, which enables the computing device 1000 to communicate via one or more networks 1060. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1040 can include one or more of any type of network interface (e.g., a network interface card (NIC)), wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a near-field communication (NFC) interface, and the like.

[0222] In one embodiment of the present specification, the above-mentioned components of the computing device 1000 and other components not shown in the above-mentioned components can be connected with each other, for example, through a bus. It should be understood that Figure 10 the computing device structure block diagram shown is merely for the purpose of example, and is not a limitation on the scope of the present specification. Other components can be added or replaced by those skilled in the art as needed. Figure 10 the computing device structure block diagram shown is merely for the purpose of example, and is not a limitation on the scope of the present specification. Other components can be added or replaced by those skilled in the art as needed.

[0223] The computing device 1000 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a PC. The computing device 1000 can also be a mobile or stationary server.

[0224] The processor 1020 is configured to execute computer-executable instructions, which, when executed by the processor 920, implement the steps of the above-mentioned two image recognition methods and the image recognition model training method.

[0225] The above is a schematic scheme of the computing device of the embodiment. It should be noted that the technical scheme of the computing device and the technical schemes of the two image recognition methods and the image recognition model training method belong to the same concept, and the details of the technical scheme of the computing device that are not described in detail can be referred to the description of the technical schemes of the two image recognition methods and the image recognition model training method.

[0226] An embodiment of the present specification also provides a computer readable storage medium storing computer executable instructions, which, when executed by a processor, implement the steps of the two image recognition methods and the image recognition model training method.

[0227] The above is a schematic scheme of the computer readable storage medium of the embodiment. It should be noted that the technical scheme of the storage medium and the technical schemes of the two image recognition methods and the image recognition model training method belong to the same concept, and the details of the technical scheme of the storage medium that are not described in detail can be referred to the description of the technical schemes of the two image recognition methods and the image recognition model training method.

[0228] An embodiment of the present specification also provides a computer program, which, when executed in a computer, causes the computer to perform the steps of the two image recognition methods and the image recognition model training method.

[0229] The above is a schematic scheme of the computer program of the embodiment. It should be noted that the technical scheme of the computer program and the technical schemes of the two image recognition methods and the image recognition model training method belong to the same concept, and the details of the technical scheme of the computer program that are not described in detail can be referred to the description of the technical schemes of the two image recognition methods and the image recognition model training method.

[0230] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims can be performed in an order other than that in which they are recited and still achieve desirable results. Also, the order in which processes are depicted in the figures does not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing can be advantageous or necessary.

[0231] The computer readable medium can include any entity or apparatus capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, software distribution medium, etc. It should be noted that the computer readable medium can include appropriate additions or subtractions according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0232] It should be noted that for the foregoing method embodiments, the descriptions are each simply a combination of a series of acts for the sake of brevity, but those skilled in the art should know that the present application is not limited by the order of the acts described, because some steps can be performed in other orders or at the same time in accordance with the present application. Secondly, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the acts and modules involved are not necessarily essential to the present application.

[0233] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0234] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The alternative embodiments do not describe all the details and limit the present application to the specific embodiments described. Obviously, according to the content of the present application, many modifications and changes can be made. The present application selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present application, so that those skilled in the art can well understand and use the present application. The present application is limited only by the claims and their full scope and equivalents.

Claims

1. An image recognition method, comprising: receiving an image to be recognized and inputting the image to be recognized into a target image recognition model to obtain an image recognition result of the image to be recognized, wherein the target image recognition model is obtained by training an initial image recognition model according to target sample image features, the initial image recognition model is obtained by training according to historical sample image features of historical sample images, the target sample image features are obtained by adding weighted features to current sample image features of a current sample image, and the weighted features are obtained by using a feature processing module in the initial image recognition model to calculate a similarity coefficient of the historical sample image features and a query condition by taking the current sample image features as the query condition, and weighting the current sample image features by using the similarity coefficient to obtain the weighted features.

2. The image recognition method of claim 1, wherein the inputting the image to be recognized into the target image recognition model to obtain the image recognition result of the image to be recognized comprises: inputting the image to be recognized into the target image recognition model, extracting features of the image to be recognized by using a feature extraction module in the target image recognition model to obtain image features of the image to be recognized; and performing image recognition on the image features by using an image recognition module in the target image recognition model to obtain the image recognition result of the image to be recognized.

3. The image recognition method of claim 1, further comprising, before the receiving the image to be recognized and inputting the image to be recognized into the target image recognition model to obtain the image recognition result of the image to be recognized: determining a current sample image of an initial image recognition model and extracting features of the current sample image to obtain current sample image features; obtaining historical sample image features of the initial image recognition model, wherein the historical sample image features are image features generated in a process of training the initial image recognition model according to historical sample images; processing the historical sample image features and the current sample image features to obtain target sample image features; and training the initial image recognition model based on the target sample image features to obtain the target image recognition model.

4. An image recognition model training method, comprising: determining a current sample image of an initial image recognition model and extracting features of the current sample image to obtain current sample image features; obtaining historical sample image features of the initial image recognition model, wherein the historical sample image features are image features generated in a process of training the initial image recognition model according to historical sample images; ​ processing the historical sample image features and the current sample image features to obtain target sample image features, wherein the processing the historical sample image features and the current sample image features to obtain target sample image features comprises: weighting the historical sample image features based on the current sample image features by using a feature processing module in the initial image recognition model to obtain weighted features, the obtaining the weighted features comprising: taking the current sample image features as a query condition by using the feature processing module in the initial image recognition model, calculating a similarity coefficient of the historical sample image features and the query condition, weighting the current sample image features by using the similarity coefficient to obtain the weighted features; and adding the weighted features and the current sample image features to obtain the target sample image features; training the initial image recognition model based on the target sample image features to obtain a target image recognition model.

5. The image recognition model training method of claim 4, wherein the feature extraction of the current sample image to obtain current sample image features comprises: inputting the current sample image into the initial image recognition model, and extracting features of the current sample image by using a feature extraction module in the initial image recognition model to obtain current sample image features of the current sample image.

6. The image recognition model training method of claim 4, wherein the training of the initial image recognition model based on the target sample image features to obtain a target image recognition model comprises: generating updated weights based on the target sample image features, and updating original weights of the initial image recognition model by using the updated weights to obtain a to-be-tested image recognition model; model testing of the to-be-tested image recognition model based on a current test image corresponding to the current sample image, and determining the to-be-tested image recognition model as a target image recognition model in a case where the model testing is passed.

7. The image recognition model training method of claim 6, wherein the generating of updated weights based on the target sample image features, and the updating of original weights of the initial image recognition model by using the updated weights to obtain a to-be-tested image recognition model comprises: inputting the target sample image features into a third feature extraction module of the initial image recognition model to obtain model training features; calculating a feature mean of the model training features, and taking the feature mean as the updated weights; determining original weights of an image recognition module in the initial image recognition model, and replacing the original weights with the updated weights to obtain the to-be-tested image recognition model.

8. The image recognition model training method of claim 6, wherein the model testing of the to-be-tested image recognition model based on a current test image corresponding to the current sample image, and the determining of the to-be-tested image recognition model as a target image recognition model in a case where the model testing is passed comprises: input the current test image corresponding to the current sample image into the to-be-tested image recognition model to obtain a test image recognition result of the current test image; in a case where it is determined that the test image recognition result meets a preset test passing condition, determine the to-be-tested image recognition model as a target image recognition model. 9.The image recognition model training method of claim 4, before the obtaining the weighted feature by weighting the historical sample image feature based on the current sample image feature using the feature processing module in the initial image recognition model, further comprising: determining a training sample corresponding to the feature processing module, and training the feature processing module using the training sample to obtain a trained feature processing module. 10.The image recognition model training method of claim 9, wherein the determining the training sample corresponding to the feature processing module, and training the feature processing module using the training sample to obtain the trained feature processing module comprises: determining a training sample corresponding to the feature processing module, and dividing the training sample into a first training sample and a second training sample; training the feature processing module based on the first training sample to obtain a first loss function; training the feature processing module based on the first training sample and the second training sample data to obtain a second loss function; training the feature processing module based on the first loss function and the second loss function to obtain the trained feature processing module. 11.An image recognition method applied to a cloud-side device, comprising: obtaining a to-be-recognized image sent by an edge-side device; inputting the to-be-recognized image into a target image recognition model to obtain an image recognition result of the to-be-recognized image, wherein the target image recognition model is obtained by training an initial image recognition model according to target sample image features, the initial image recognition model is obtained by training according to historical sample image features of historical sample images, the target sample image features are obtained by adding weighted features to current sample image features of a current sample image, and the weighted features are obtained by using a feature processing module in the initial image recognition model to calculate a similarity coefficient of the historical sample image features and a query condition by taking the current sample image features as the query condition, and weighting the current sample image features using the similarity coefficient to obtain the weighted features; returning the image recognition result to the edge-side device. 12.A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the image recognition method of any one of claims 1 to 3, the image recognition model training method of any one of claims 4 to 10, and the image recognition method of claim 11.

Images