Method and device for updating object recognition model, electronic device, and computer program

Abstract

The present application provides an object recognition model updating method and an apparatus therefor, an electronic device, a computer-readable storage medium, and a computer program product, which relate to artificial intelligence technology. The method includes the steps of: obtaining an object recognition model, where the object recognition model is trained based on a training sample set of each object recognition task; obtaining category center parameters of the object recognition model; obtaining a target training sample set for a new object recognition task; determining target category center parameters based on sample features of each image sample in each training sample set and the sample features of each image sample in the target training sample set; updating the category center parameters of the object recognition model to the target category center parameters to obtain a target object recognition model, where the target object recognition model is used to recognize a target category to which an object to be recognized belongs based on the target category center parameters.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application is based on a Chinese patent application bearing application number 202211678065.2, filed with the China Patent Office on December 26, 2022, and claims priority to that Chinese patent application, the entire contents of which are incorporated herein by reference.

[0002] The present application relates to the field of artificial intelligence technology, and in particular to an object recognition model updating method and apparatus, an electronic device, a computer-readable storage medium, and a computer program product. [Background technology]

[0003] Artificial intelligence (AI) is a comprehensive technology in computer science that researches the design principles and implementation methods of various intelligent machines, thereby endowing them with the capabilities of sensing, reasoning, and decision-making. AI technology is a comprehensive academic field that encompasses a wide range of fields, including natural language processing technology and machine learning / deep learning. As technology develops, AI technology is being applied to more and more fields and is playing an increasingly important role.

[0004] Object recognition (e.g., recognizing game characters in game images) is also an important application field of artificial intelligence. For example, an object recognition model is used to realize object recognition for a specific object recognition task (e.g., recognizing various categories of game characters in a specific game). In related art, when adding recognition capabilities for a new object recognition task to an object recognition model, the existing object recognition model needs to be retrained using training samples for the new object recognition task, which results in low implementation efficiency. In addition, the need to retrain using training samples for the new object recognition task also affects the recognition effectiveness of the existing object recognition task. Summary of the Invention [Problem to be solved by the invention]

[0005] Embodiments of the present application provide an object recognition model updating method and device, an electronic device, a computer-readable storage medium, and a computer program product, which can improve the implementation efficiency of adding recognition capabilities for new object recognition tasks to an object recognition model and ensure the recognition effectiveness of the object recognition model. [Means for solving the problem]

[0006] The technical solution of the embodiments of the present application is realized as follows:

[0007] An embodiment of the present application provides a method for updating an object recognition model, the method comprising: obtaining an object recognition model, wherein the object recognition model is used to recognize objects of multiple categories in at least one object recognition task, the object recognition model being trained based on a training sample set for each of the object recognition tasks, the training sample set for the object recognition task including image samples of objects of each category in the object recognition task; obtaining category center parameters of the object recognition model, the category center parameters being determined based on sample features of each of the image samples in each of the training sample sets; obtaining a target training sample set for a new object recognition task, the target training sample set including image samples of multiple categories of objects in the new object recognition task; determining a target category center parameter based on sample features of each of the image samples in each of the training sample sets and sample features of each of the image samples in the target training sample set; updating category center parameters of the object recognition model to the target category center parameters to obtain a target object recognition model; Here, the target object recognition model is used to recognize a target category to which an object to be recognized belongs based on the target category center parameter, and the target category is one of multiple categories in the new object recognition task and multiple categories in the at least one object recognition task.

[0008] An embodiment of the present application further provides an apparatus for updating an object recognition model, the apparatus comprising: a first acquisition module configured to acquire an object recognition model, wherein the object recognition model is used to recognize objects of multiple categories in at least one object recognition task, the object recognition model being trained based on a training sample set for each of the object recognition tasks, the training sample set for the object recognition task including image samples of objects of each category in the object recognition task; and a second acquisition module configured to acquire category center parameters of the object recognition model, the category center parameters being determined based on sample features of each of the image samples in each of the training sample sets; and a third acquisition module configured to acquire a target training sample set for a new object recognition task, the target training sample set including image samples of multiple categories of objects in the new object recognition task; a determination module configured to determine a target category center parameter based on sample features of each of the image samples in each of the training sample sets and sample features of each of the image samples in the target training sample set; an updating module configured to update category center parameters of the object recognition model to the target category center parameters to obtain a target object recognition model; Here, the target object recognition model is used to recognize a target category to which an object to be recognized belongs based on the target category center parameter, and the target category is one of multiple categories in the new object recognition task and multiple categories in the at least one object recognition task.

[0009] An embodiment of the present application further provides an electronic device, the electronic device comprising: a memory storing computer-executable instructions; and a processor that executes computer-executable instructions stored in the memory to perform a method for updating an object recognition model according to an embodiment of the present application.

[0010] An embodiment of the present application further provides a computer-readable storage medium having stored thereon computer-executable instructions that, when executed by a processor, cause the method for updating an object recognition model according to an embodiment of the present application to be performed.

[0011] An embodiment of the present application further provides a computer program product including computer-executable instructions that, when executed by a processor, cause the computer to perform a method for updating an object recognition model according to an embodiment of the present application. [Effects of the Invention]

[0012] The present embodiment has the following beneficial effects.

[0013] In the present embodiment, the object recognition model obtained by training has category center parameters, which are determined based on the sample features of each image sample in each training sample set and can be obtained without model training. When a new object recognition task needs to be added to the object recognition model, target category center parameters can be determined based on the sample features of each image sample in each training sample set for the object recognition task and the sample features of each image sample in a target training sample set for the new object recognition task, and the category center parameters of the object recognition model can be updated to the target category center parameters. There is no need to retrain the object recognition model. The target object recognition model thus obtained not only has the recognition capabilities for at least one object recognition task, but also the recognition capabilities for the new object recognition task. Because there is no need to retrain the object recognition model, 1) the implementation efficiency of adding the recognition capabilities for the new object recognition task to the object recognition model is improved; and 2) adding the recognition capabilities for the new object recognition task to the object recognition model does not affect the recognition effectiveness of the existing object recognition task of the object recognition model, thereby ensuring the recognition effectiveness of the target object recognition model for the existing object recognition task and improving the recognition accuracy of the target object recognition model to which the new object recognition task has been added. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a schematic diagram of the architecture of an object recognition model updating system 100 according to an embodiment of the present application. [Figure 2] 1 is an exemplary flowchart of a method for updating an object recognition model according to an embodiment of the present application; [Figure 3] 1 is an exemplary flowchart of a method for updating an object recognition model according to an embodiment of the present application; [Figure 4]FIG. 2 is an exemplary structural diagram of an object recognition model according to an embodiment of the present application; [Figure 5] 1 is an exemplary flowchart of a method for updating an object recognition model according to an embodiment of the present application; [Figure 6] FIG. 1 is a schematic diagram of updating an object recognition model according to an embodiment of the present application; [Figure 7] 1 is an exemplary flowchart of a method for updating an object recognition model according to an embodiment of the present application; [Figure 8] 1 is an exemplary flowchart of a method for updating an object recognition model according to an embodiment of the present application; [Figure 9A] FIG. 1 is a schematic diagram of an application of an object recognition model according to an embodiment of the present application. [Figure 9B] FIG. 1 is a schematic diagram of an application of an object recognition model according to an embodiment of the present application. [Figure 10] 1 is an exemplary structural diagram of an object recognition model updating apparatus according to an embodiment of the present application; [Figure 11] 5 is an exemplary structural diagram of an electronic device 500 for implementing a method for updating an object recognition model according to an embodiment of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0015] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be described in more detail hereinafter with reference to the drawings, and the described embodiments should not be understood as limitations on the present application, and all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present application.

[0016] In the following description, the term "some embodiments" refers to a subset of all possible embodiments, and it is understood that the terms "some embodiments" may refer to the same or different subsets of all possible embodiments, which may be combined with each other without conflict.

[0017] The terms "first / second / third" referred to in the following description are merely for distinguishing between similar objects and do not represent a particular order of the objects; it is understood that "first / second / third" may, in some cases, be interchangeable with a particular order or precedence order, whereby the embodiments of the present application described herein may be performed in an order other than that illustrated or described.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. The terms used herein are for the purpose of describing the present application only and are not intended to be limiting of the present application.

[0019] Before describing the embodiments of the present application in more detail, the nouns and terms referred to in the embodiments of the present application will be explained, and the nouns and terms referred to in the embodiments of the present application are applicable to the following description.

[0020] 1) A "client" is an application that runs on a terminal and provides various services, such as a client that supports object recognition processing.

[0021] 2) The term "in response to" refers to a condition or state upon which the operation being performed depends, and when the dependent condition or state is met, the operation or operations being performed may be performed in real time or with a set delay, and there is no execution order restriction between the operations being performed, unless otherwise specified.

[0022] The embodiments of the present application provide an object recognition model updating method and apparatus, an electronic device, a computer-readable storage medium, and a computer program product, which can improve the implementation efficiency of adding recognition capabilities for new object recognition tasks to an object recognition model and ensure the recognition effectiveness of the object recognition model. Each of the above will be described below.

[0023] It should be noted that when the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data should comply with the relevant laws, regulations and standards of the relevant countries or regions.

[0024] The following describes an object recognition model updating system according to an embodiment of the present application. Referring to Fig. 1, Fig. 1 is a schematic diagram of the architecture of an object recognition model updating system 100 according to an embodiment of the present application, in which to support one exemplary application, a terminal (terminal 400-1 is shown as an example) is connected to a server 200 via a network 300, which may be a wide area network or a local area network, or a combination of both, and which realizes data transmission using a wireless or wired link.

[0025] The terminal (e.g., 400-1) is configured to send a model update request for the object recognition model to the server 200 in response to a model update command for the object recognition model, the server 200 receiving the model update request sent from the terminal, and in response to the model update request, obtaining an object recognition model, wherein the object recognition model is used to recognize objects of multiple categories in at least one object recognition task, the object recognition model being trained based on a training sample set for each object recognition task, the training sample set for the object recognition task including image samples of objects of each category in the object recognition task, and obtaining category center parameters for the object recognition model; The category center parameters are determined based on the sample features of each image sample in each training sample set to obtain a target training sample set for the new object recognition task, the target training sample set including image samples of multiple categories of objects in the new object recognition task, the target category center parameters are determined based on the sample features of each image sample in each training sample set and the sample features of each image sample in the target training sample set, the category center parameters of the object recognition model are updated to the target category center parameters, and a target object recognition model is obtained, in this way a target object recognition model supporting at least one of the object recognition task and the new object recognition task is obtained.

[0026] In some embodiments, after obtaining the target object recognition model, the server 200 can actively send the target object recognition model to the terminal so that the terminal can use it when performing object recognition processing; of course, the terminal can also actively obtain the target object recognition model from the server 200 when performing object recognition processing; in this case, when the terminal actively obtains it, the server 200 sends the target object recognition model to the terminal.

[0027] As an example, a terminal (e.g., 400-1) may be provided with a client that supports object recognition processing. When performing object recognition processing, a user can trigger an action recognition command on the terminal (e.g., 400-1) through the client, and the terminal acquires a target object recognition model from the server 200 in response to the action recognition command, and simultaneously acquires an object image of an object to be recognized, and performs object recognition on the object image using the target object recognition model based on a target category center parameter to obtain a target category to which the object to be recognized belongs, and the target category is one of the multiple categories in the new object recognition task and the multiple categories in at least one object recognition task.

[0028] In some embodiments, the object recognition model update method according to the embodiments of the present application may be implemented by various electronic devices, for example, independently by a terminal, independently by a server, or collaboratively by a terminal and a server. The object recognition model update method according to the embodiments of the present application may be applied to various scenarios, for example, but not limited to, cloud technology, artificial intelligence, intelligent transportation, driving assistance, games, audio / video, and images.

[0029] In some embodiments, the electronic device that performs the object recognition model updating method according to the embodiment of the present application may be various types of terminals or servers. Here, the server (e.g., server 200) may be an independent physical server, or a server cluster or distributed system consisting of multiple physical servers. The terminal (e.g., terminal 400-1) may be, but is not limited to, a smartphone, a tablet computer, a notebook computer, a desktop computer, an intelligent voice dialogue device (e.g., a smart speaker), a smart home appliance (e.g., a smart TV), a smart watch, an in-vehicle terminal, a wearable device, a virtual reality (VR) device, etc. The terminal and the server may be directly or indirectly connected via wired or wireless communication, and the embodiment of the present application is not limited thereto.

[0030] In some embodiments, the object recognition model updating method according to the present disclosure can be implemented using cloud technology. Cloud technology refers to a hosting technology that integrates a set of resources, such as hardware, software, and networks, within a wide area network or a local area network to realize computing, storage, processing, and sharing of data. Cloud technology is a collective term for network technology, information technology, integration technology, management platform technology, and application technology based on the application of the cloud computing business model. It can form a resource pool that can be used as needed, flexibly, and conveniently. Cloud computing technology provides important support. The back-end services of technical network systems require large amounts of computing and storage resources. As an example, a server (e.g., server 200) may be a cloud server that provides basic cloud computing services, such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), big data, and artificial intelligence platforms.

[0031] In some embodiments, a single blockchain may be configured by multiple servers, and the servers may be nodes on the blockchain. Information connections may exist between nodes in the blockchain, and information may be transmitted between nodes through the information connections. Here, data related to the object recognition model update method according to embodiments of the present application (e.g., object recognition model, target object recognition model, training sample set, target training sample set, etc.) may be stored on the blockchain.

[0032] In some embodiments, a terminal or a server can implement the object recognition model updating method according to the present embodiment by executing a computer program. For example, the computer program may be a native program or software module in an operating system, i.e., a program that needs to be installed in an operating system to run, such as a native application (APP), or a mini-program, i.e., a program that can be executed simply by downloading it to a browser environment, or a mini-program that can be embedded in any APP. In short, the computer program may be any form of application, module, or plug-in.

[0033] The following describes a method for updating an object recognition model according to an embodiment of the present application. In some embodiments, the method for updating an object recognition model according to an embodiment of the present application can be implemented by various electronic devices, for example, independently by a terminal, independently by a server, or cooperatively by a terminal and a server. Taking the example of implementation by a server as an example, referring to FIG. 2 , which is an exemplary flowchart of the method for updating an object recognition model according to an embodiment of the present application, the method for updating an object recognition model according to an embodiment of the present application includes the following steps:

[0034] In step 101, the server obtains an object recognition model.

[0035] Here, the object recognition model is used to recognize objects of multiple categories in at least one object recognition task, and the object recognition model is trained based on a training sample set for each object recognition task, and the training sample set for the object recognition task includes image samples of objects of each category in the object recognition task.

[0036] In practical applications, a user can trigger a model update command for an object recognition model through a client (e.g., a client supporting object recognition processing) configured on the terminal, whereby the terminal responds to the model update command by sending a model update request for the object recognition model to a server. Upon receiving the model update request sent from the terminal, the server obtains a pre-trained object recognition model in response to the model update request. The object recognition model can be used to recognize objects of multiple categories in at least one object recognition task. The object recognition model is trained based on a training sample set for each object recognition task (including image samples of objects of each category in the object recognition task). The object recognition task may be a recognition task for a category to which an object in an image of a target service belongs. For example, the target service may be a game service, in which case the different object recognition tasks may be object recognition tasks for different games (e.g., Game 1, Game 2), or the target service may be a video service, in which case the different object recognition tasks may be object recognition tasks for different videos (e.g., Movie 1, Movie 2).

[0037] In some embodiments, referring to FIG. 3, FIG. 3 is an exemplary flowchart of a method for updating an object recognition model according to an embodiment of the present application. FIG. 3 shows that step 101 in FIG. 2 can be realized through steps 1011 to 1014. In step 1011, an initial object recognition model is obtained, and training sample sets for each object recognition task are obtained, with each image sample in the training sample set being labeled. In step 1012, sample features of each image sample in each training sample set are obtained, and category center parameters of the object recognition model are determined based on the sample features of each image sample in each training sample set. In step 1013, object recognition is performed on each image sample in each training sample set using the initial object recognition model based on the category center parameters, and a recognition result for each image sample is obtained. In step 1014, the model parameters of the initial object recognition model are updated based on the difference between the recognition result and the label of each image sample, resulting in an object recognition model different from the category center parameters.

[0038] In step 1011, the image samples are object images of objects of each category in the object recognition task. The sample amount of image samples in each training sample set may be the same. Balancing the sample amount ensures that each object recognition task has equal learning opportunities, thereby obtaining an object recognition model with balanced recognition accuracy for each object recognition task. In step 1012, first, each image sample in each training sample set is extracted to obtain sample features for each image sample. For example, embedding features (i.e., embedding) are extracted for each image sample to obtain the sample features. Then, a clustering process with a target number of clustering centers is performed on the sample features of each image sample in each training sample set to obtain a target number of feature clusters. The sample features corresponding to the clustering centers of each feature cluster are combined to obtain category center parameters for the object recognition model. In step 1013, the category center parameters calculated in step 1012 are used as category center parameters for the initial object recognition model. Then, based on the category center parameters, the initial object recognition model is called to perform object recognition on each image sample in each training sample set, thereby obtaining recognition results for each image sample. In step 1014, for each image sample, a loss function value of the initial object recognition model is determined based on the difference between the recognition result of the image sample and the label. If the value of the loss function exceeds a predetermined threshold, an error signal of the initial object recognition model is determined based on the loss function. The error signal can then be backpropagated within the initial object recognition model. During the propagation process, model parameters of each layer in the initial object recognition model are updated to train the initial object recognition model and obtain the object recognition model. Note that the model parameters updated in step 1014 do not include category center parameters.

[0039] By applying the above embodiment, an initial object recognition model is trained using a training sample set to obtain an object recognition model, and the object recognition model has category center parameters determined based on sample features of image samples, where the category center parameters can be obtained without the need for model training. In this way, if it is necessary to add recognition capabilities for a new object recognition task to the object recognition model, this can be achieved by updating the category center parameters, and retraining the object recognition model is not necessary. This 1) improves the implementation efficiency of adding recognition capabilities for a new object recognition task to the object recognition model, and 2) adds recognition capabilities for a new object recognition task to the object recognition model without affecting the recognition effectiveness of the object recognition model for the existing object recognition task.

[0040] In some embodiments, the server may obtain a training sample set for each object recognition task in the following manner: performing the following processes for each object recognition task, respectively, including: obtaining an object video for the object recognition task, where the object video includes multiple frames of video images; determining target video images of multiple frames among the multiple frames of video images, where the target video images include multiple categories of objects in the object recognition task; selecting, for each category of objects in the object recognition task, a target number of first video images including the objects from the multiple frames of target video images, and setting the first video images as image samples of the objects of the corresponding category; and constructing a training sample set for the object recognition task based on the image samples of the objects of each category in the object recognition task.

[0041] An object video for an object recognition task can be obtained, the object video including multiple frames of video images. Object detection can be performed on each frame of the video image to determine target video images of multiple frames among the multiple frames of video images, where the target video images include objects of multiple categories for the object recognition task. The object detection process can be realized by an object detection model. For example, the object detection model can be a target detector trained based on the Common Objects in Context (COCO) dataset (You Only Look Once (YOLO)). Alternatively, a pre-trained object detector can be employed based on service data (e.g., video images of each frame in a game video). For each category of object in the object recognition task, a target number of first video images containing the object are selected from the multiple frames of target video images, and the first video images are used as image samples of the object in the corresponding category. In this way, the image samples of each category of object in the object recognition task are combined to construct a training sample set for the object recognition task.

[0042] In some embodiments, the initial object recognition model includes a feature extraction layer, and the server can obtain sample features for each image sample in each training sample set in the following manner: the feature extraction layer performs feature extraction on each image sample in each training sample set to obtain sample features for each image sample. Correspondingly, after updating the model parameters of the initial object recognition model, the server can further update category center parameters of the object recognition model in the following manner: the feature extraction layer with updated model parameters performs feature extraction on each image sample in each training sample set to obtain new sample features for each image sample, determine new category center parameters of the object recognition model based on the new sample features of each image sample in each training sample set, and update the category center parameters of the object recognition model to the new category center parameters.

[0043] The category center parameters of the object recognition model may be generated based on sample features extracted by the feature extraction layer of the initial object recognition model. During the training process of the initial object recognition model, the model parameters of the feature extraction layer of the initial object recognition model are updated, so the category center parameters of the object recognition model also need to be updated. That is, the feature extraction layer with updated model parameters performs feature extraction on each image sample in each training sample set to obtain new sample features for each image sample. Then, new category center parameters of the object recognition model are determined based on the new sample features for each image sample in each training sample set, thereby updating the category center parameters of the object recognition model to the new category center parameters. In this way, the category center parameters obtained in step 102 become the new category center parameters. By applying the above embodiment, after training of the initial object recognition model is completed, new category center parameters can be determined based on the feature extraction layer with updated model parameters, thereby improving the accuracy of object recognition based on the object recognition model.

[0044] In some embodiments, the number of image samples of objects of each category in the training sample set is multiple, and based on the sample features of each image sample in each training sample set, the server can determine category center parameters of the object recognition model in the following manner: determine multiple sample features corresponding to objects of each category from the sample features of each image sample in each training sample set, perform clustering on the multiple sample features corresponding to objects of each category based on a target number of clustering centers to obtain a target number of sample feature clusters, and generate category center parameters of the object recognition model based on target sample features corresponding to the clustering centers of each sample feature cluster.

[0045] Here, the training sample set includes image samples of objects of each category in the object recognition task. Therefore, feature extraction is performed on the image samples of objects of each category to obtain sample features for the image samples of objects of each category. The sample features for the image samples of objects of each category are then subjected to a clustering process with a target number of clustering centers on the sample features corresponding to each category of object, thereby obtaining a target number of sample feature clusters. For example, the clustering process can be performed using the k-means algorithm, where the target number is the k value in the k-means algorithm, which can be set in advance or determined based on the silhouette coefficient method or elbow method. Finally, target sample features corresponding to the clustering centers of each sample feature cluster are combined to obtain category center parameters for the object recognition model. For example, when combining a target number of target sample features, the target number of target sample features can be connected to obtain the category center parameters. In this way, by performing a clustering process on the sample features of samples in the training sample set, category center parameters are generated, ensuring the accuracy of the category center parameters and enabling the category center parameters to better represent multiple categories in the object recognition task, thereby improving the object recognition accuracy of the model.

[0046] In some embodiments, the category center parameter includes multiple subcategory center parameters, and the subcategory center parameters correspond one-to-one to the category centers of each category in at least one object recognition task, i.e., one subcategory center parameter corresponds to one category center, and each category includes at least one category center, which is used to represent the representation of the object of that category, and different category centers correspond to different representations. For example, in a game object recognition task, a category center may be at least one game object prototype of the game object of the category to which it belongs, and the game object prototype is a different representation of the game object. For example, the object's standard styles when the game object activates game skill 1, game skill 2, and game skill 3, and when standing and walking can be regarded as five game object prototypes of the game object. In view of this, the server can perform object recognition on each image sample in each training sample set based on the category center parameters in the following manner, and obtain recognition results for each image sample: perform feature extraction on each image sample in each training sample set to obtain sample features for each image sample; perform first object recognition on the sample features of each image sample in each training sample set based on each subcategory center parameter to obtain the probability that the object in each image sample belongs to each category center; for each image sample, determine the category to which the object in the image sample belongs based on the probability that the object in the image sample belongs to each category center, and use the category to which the object in the image sample belongs as the recognition result.

[0047] In some embodiments, the server may perform feature extraction on each image sample in each training sample set to obtain sample features for each image sample in the following manner: perform a convolution process on each image sample in each training sample set to obtain convolution features for each image sample; perform a pooling process on the convolution features of each image sample to obtain pooled features for each image sample; perform an embedding feature extraction process on the pooled features of each image sample to obtain embedded features for each image sample; perform a feature mapping process on the embedded features of each image sample to obtain mapped features for each image sample; and perform a normalization process on the pooled features of each image sample to obtain sample features for each image sample. Here, the convolution process may be implemented based on preset convolution process parameters (e.g., convolution kernel size, number of convolution kernels, step size, etc.), the pooling process may also be implemented based on preset pooling process parameters (e.g., pooling window size, step size, etc.), and the embedded feature extraction process may be implemented by a pre-trained embedding feature extraction network (i.e., an Embedding Network).

[0048] As can be seen from the above example, each subcategory center parameter is actually a target sample feature corresponding to the clustering center of a sample feature cluster. Therefore, when performing first object recognition, the following process can be performed for each subcategory center parameter: determine the similarity (e.g., cosine similarity) or distance (e.g., Euclidean distance) between the target sample feature corresponding to the subcategory center parameter and the sample feature of each image sample, and use the similarity or distance as the probability that the object in each image sample belongs to the category center corresponding to the subcategory center parameter. Then, the following process can be performed for each image sample: determine the category to which the object in the image sample belongs based on the probability that the object in the image sample belongs to each category center. For example, the category in which the category center corresponding to the maximum probability (i.e., the maximum similarity or the maximum distance) is located can be determined as the category to which the object in the image sample belongs. In this way, the category center parameters are used to recognize the category to which the object in the image sample belongs.

[0049] 4 as an example, which is an exemplary structural diagram of an object recognition model according to an embodiment of the present application. The initial object recognition model includes a feature extraction layer, a first object recognition layer, and a second object recognition layer, and correspondingly, the initial object recognition model enables the server to perform object recognition on each image sample in each training sample set based on category center parameters in the following manner to obtain a recognition result for each image sample: use the feature extraction layer to perform feature extraction on each image sample in each training sample set to obtain sample features for each image sample; use the first object recognition layer to perform first object recognition on the sample features of each image sample in each training sample set based on each subcategory center parameter to obtain a probability that the object in each image sample belongs to each category center; use the second object recognition layer to determine, for each image sample, a category to which the object in the image sample belongs based on the probability that the object in the image sample belongs to each category center, and the category to which the object in the image sample belongs is the recognition result.

[0050] 4, the feature extraction layer includes a convolution feature extraction layer, a pooling layer, an embedding feature extraction layer, a feature mapping layer, and a normalization layer. In view of this, the convolution feature extraction layer is used to perform a convolution process on each image sample in each training sample set to obtain convolution features for each image sample; the pooling layer is used to perform a pooling process on the convolution features of each image sample to obtain pooled features for each image sample; the embedding feature extraction layer is used to perform an embedding feature extraction process on the pooled features of each image sample to obtain embedded features for each image sample; the feature mapping layer is used to perform a feature mapping process on the embedding features of each image sample to obtain mapped features for each image sample; and the normalization layer is used to perform a normalization process on the pooled features of each image sample to obtain sample features for each image sample.

[0051] In some embodiments, the number of object recognition tasks is M, and an object recognition model can be obtained by sequentially training an initial object recognition model based on the training sample set of each object recognition task, i.e., obtain an initial object recognition model, obtain a training sample set for each object recognition task, train the initial object recognition model based on the training sample set of the first object recognition task, obtain an intermediate object recognition model for the first object recognition task, train an intermediate object recognition model for the (i-1)th object recognition task using the training sample set of the i-th object recognition task, obtain the intermediate object recognition model for the i-th object recognition task, traverse i ("traverse i object recognition tasks") to obtain the intermediate object recognition model for the M-th object recognition task, and the intermediate object recognition model for the M-th object recognition task is the object recognition model, where M and i are integers greater than 1, and i is less than or equal to M.

[0052] In step 102, the category center parameters of the object recognition model are obtained.

[0053] Here, category center parameters are determined based on sample features of each image sample in each training sample set.

[0054] In practical application, the category center parameters may be generated based on sample features extracted by the feature extraction layer of the object recognition model for each image sample in each training sample set. Specifically, a plurality of sample features corresponding to objects of each category are determined from the sample features of each image sample in each training sample set. Clustering is performed on the plurality of sample features corresponding to the objects of each category based on a target number of clustering centers to obtain a target number of sample feature clusters. The category center parameters of the object recognition model are generated based on target sample features corresponding to the clustering centers of each sample feature cluster. Since the training sample set includes image samples of objects of each category in the object recognition task, feature extraction is performed on the image samples of objects of each category to obtain sample features for the image samples of the objects of each category. The sample features for the image samples of the objects of each category are then clustered with a target number of clustering centers to obtain a target number of sample feature clusters. For example, the clustering process can be implemented using a k-means algorithm, where the target number is the k value in the k-means algorithm, and can be preset or determined based on the silhouette coefficient method or elbow method. Finally, the target sample features corresponding to the clustering centers of each sample feature cluster are combined to obtain the category center parameters of the object recognition model. For example, when combining a target number of target sample features, the target number of target sample features can be connected to obtain the category center parameters.

[0055] In some embodiments, the category center parameters of the object recognition model include multiple subcategory center parameters, each of which has a one-to-one correspondence with the category center of each category in the at least one object recognition task. That is, the category center parameters are used to indicate the category center of each category in the at least one object recognition task. When recognizing an object to be recognized using the object recognition model, the category center parameters can be used to obtain a probability that the object to be recognized belongs to each category in the at least one object recognition task.

[0056] In some other embodiments, the category center parameters of the object recognition model may be configured by task category center parameters of each object recognition task. Taking a target recognition task within at least one object recognition task as an example, the task category center parameters of the target recognition task include multiple subtask category center parameters, including a first subtask category center parameter and a second subtask category center parameter. The first subtask category center parameter corresponds one-to-one to the category center of each category in the target recognition task, and the second subtask category center parameter corresponds to a recognition task other than the target recognition task in the at least one object recognition task. That is, the task category center parameter of the target recognition task is used to indicate the category center of each category in the target recognition task and other category centers of other recognition tasks (here, the category centers of each category in other recognition tasks are collectively referred to as other category centers and are not distinguished). In this way, when recognizing an object to be recognized based on the object recognition model, the task category center parameters of the object recognition tasks can be employed for recognition. Furthermore, when recognizing an object to be recognized, if it is previously set which object recognition task (e.g., a target recognition task) to adopt for recognition, the task category center parameters of the set object recognition task can be directly adopted for recognition. Taking the task category center parameters of a target recognition task as an example, when recognizing an object to be recognized using an object recognition model based on the task category center parameters, it is possible to obtain the possibility that the object to be recognized belongs to each category in the target recognition task and the possibility that it belongs to other categories in other recognition tasks.

[0057] In step 103, a target training sample set for a new object recognition task is obtained.

[0058] Here, the target training sample set includes image samples of multiple categories of objects in a new object recognition task. In practical application, when constructing the target training sample set, the sample amount of image samples of each category of objects in the target training sample set may be the same as or different from the sample amount of image samples of each category of objects in the training sample set. The new object recognition task is different from any of the object recognition tasks in the at least one object recognition task.

[0059] In practical application, a new object video for a new object recognition task can be obtained, the new object video including multiple frames of video images. Object detection can be performed on each frame of the video image to determine target video images of multiple frames among the multiple frames of video images, where the target video images include multiple categories of objects for the new object recognition task. The object detection process can be realized by an object detection model. For example, the object detection model can be a Yolov3 detector trained based on the Coco dataset, or a pre-trained object detector based on service data (e.g., video images of each frame in a game video). For each category of object in the new object recognition task, a target number of second video images containing the object are selected from the multiple frames of target video images, and the second video images are used as image samples of the object in the corresponding category. In this way, the image samples of each category of object in the new object recognition task are combined to form a target training sample set for the new object recognition task.

[0060] In step 104, target category center parameters are determined based on the sample features of each image sample in each training sample set and the sample features of each image sample in the target training sample set.

[0061] In practical applications, after obtaining a target training sample set for a new object recognition task, a target category center parameter for the new object recognition task can be determined. The target category center parameter may be generated based on the sample features of each image sample in the target training sample set for the new object recognition task and the sample features of each image sample in each training sample set. In some embodiments, the number of image samples of each category of object in the training sample set is multiple, and the number of image samples of each category of object in the target training sample set is multiple. Referring to FIG. 5, FIG. 5 is an exemplary flowchart of a method for updating an object recognition model according to an embodiment of the present application, and FIG. 5 shows that step 104 in FIG. 2 can be realized through steps 1041 to 1043. In step 1041, a plurality of sample features corresponding to objects of each category in the training sample set are determined from the sample features of each image sample in each training sample set, and a plurality of sample features corresponding to objects of each category in the target training sample set are determined from the sample features of each image sample in the target training sample set. In step 1042, for a plurality of sample features corresponding to objects of each category, clustering is performed on the plurality of sample features based on a target number of clustering centers to obtain a target number of sample feature clusters. In step 1043, a target category center parameter is generated based on target sample features corresponding to the clustering centers of each sample feature cluster.

[0062] In step 1041, sample features for each image sample can be obtained by performing feature extraction on the image sample using a feature extraction layer of an object recognition model. In step 1042, clustering is performed on a plurality of sample features based on a target number of clustering centers. This can be achieved using a clustering algorithm, such as the k-means algorithm, where the target number is the k value in the k-means algorithm, which can be preset or determined based on the silhouette coefficient method or elbow method. In step 1043, target sample features corresponding to the clustering centers of each sample feature cluster can be combined to obtain a target category center parameter. The combination process can involve connecting each target sample feature, thereby obtaining the target category center parameter. Here, the target category center parameter includes multiple target subcategory center parameters, and the target subcategory center parameters correspond one-to-one to the category centers of each category in the target object recognition task, and the target object recognition task includes at least one object recognition task and a new object recognition task. Therefore, each target subcategory center parameter is actually a target sample feature corresponding to the clustering center of a sample feature cluster obtained by the clustering process. In this way, the target category center parameter is generated by performing a clustering process on the sample features of the samples in the training sample set and the target training sample set, which ensures the accuracy of the target category center parameter and allows the target category center parameter to better represent multiple categories in the target object recognition task, thereby improving the object recognition accuracy of the model.

[0063] In some embodiments, based on the sample features of each image sample in each training sample set and the sample features of each image sample in the target training sample set, the server can determine a target category center parameter in the following manner: perform the following process for each training sample set, respectively, the process including: for the sample features of each image sample in the training sample set, a process of determining a feature similarity between the sample features of the image sample and the sample features of each image sample in the target training sample set; and a process of excluding from the training sample set the sample features of the image samples whose feature similarity satisfies a similarity condition to obtain a first training sample set; and determine a target category center parameter based on the sample features of each image sample in each first training sample set and the sample features of each image sample in the target training sample set.

[0064] In practical applications, the training sample set may contain image samples that are relatively similar to the image samples in the target training sample set for the new object recognition task. To reduce the impact of similar image samples on the recognition accuracy of the new object recognition task, the following process can be performed: for each image sample in the training sample set, the feature similarity between the sample feature of the image sample and the sample feature of each image sample in the target training sample set can be determined, and then a first training sample set is obtained by excluding from the training sample set the sample features of image samples whose feature similarity satisfies the similarity condition (image samples whose feature similarity reaches a similarity threshold (which may be preset), and sorting by feature similarity in descending order to select the top target number (which may be preset) of image samples). Then, a target category center parameter is determined based on the sample features of each image sample in each first training sample set and the sample features of each image sample in the target training sample set. Specifically, first, a plurality of sample features corresponding to each category of object in the first training sample set is determined from the sample features of each image sample in each first training sample set, and a plurality of sample features corresponding to each category of object in the target training sample set is determined from the sample features of each image sample in the target training sample set, then, for the plurality of sample features corresponding to each category of object, clustering is performed on the plurality of sample features based on a target number of clustering centers to obtain a target number of sample feature clusters, and finally, a target category center parameter is generated based on the target sample features corresponding to the clustering centers of each sample feature cluster. In this way, image samples whose similarity between the training sample set and the target training sample set meet a similarity condition can be excluded, thereby reducing the impact of similar image samples on the recognition accuracy of the new object recognition task, and the recognition accuracy of the new object recognition task can be improved based on the target category center parameter.

[0065] In step 105, the category center parameters of the object recognition model are updated to the target category center parameters to obtain the target object recognition model.

[0066] Here, the target object recognition model is used to recognize a target category to which an object to be recognized belongs based on a target category center parameter, and the target category is one of multiple categories in a new object recognition task and multiple categories in at least one object recognition task.

[0067] In some embodiments, when the category center parameter is used to indicate the category center of each category in the at least one object recognition task, the target category center parameter is further used to indicate the category center of each category in the at least one object recognition task and each category in the new object recognition task. When a target object recognition model is used to recognize an object to be recognized based on the target category center parameter, the likelihood that the object to be recognized belongs to each category in the at least one object recognition task and each category in the new object recognition task can be obtained. In this case, by updating the category center parameter of the object recognition model to the target category center parameter, a target object recognition model that not only has the recognition ability for the at least one object recognition task but also has the recognition ability for the new object recognition task can be obtained.

[0068] In some other embodiments, the category center parameters consist of task category center parameters for each object recognition task. When the task category center parameter for the target recognition task is used to indicate the category center of each category in the target recognition task and other category centers in other recognition tasks, the target category center parameter can be further used to indicate the category center of each category in the new recognition task and other category centers in other recognition tasks. That is, the target category center parameter can be understood as the task category center parameter for the new object recognition task. When recognizing an object to be recognized using a target object recognition model based on the target category center parameter, the possibility that the object to be recognized belongs to each category in the new object recognition task and the possibility that the object to be recognized belongs to other object recognition tasks can be obtained. In this case, updating the category center parameter of the object recognition model with the target category center parameter can obtain a target object recognition model capable of recognizing the new object recognition task. In practical applications, the target category center parameter can be added as a task category center parameter to update the category center parameter of the object recognition model. Therefore, when performing recognition of the object to be recognized in the new object recognition task, the target category center parameter can be called from the updated category center parameter.

[0069] In some embodiments, when a new object recognition task (e.g., a target new object recognition task) needs to be added again, target category center parameters corresponding to the target new object recognition task can be obtained based on each of the above training sample sets and the training sample set of the target new object recognition task by performing the above step 104. In this way, for each new object recognition task, target category center parameters of the new object recognition task can be obtained based on the above step 104, and for the target category center parameters of each new object recognition task, by updating the category center parameters of the object recognition model to the target category center parameters of the new object recognition task, a target object recognition model that supports the new object recognition task and at least one of the above object recognition tasks can be obtained.

[0070] Furthermore, when performing object recognition, the category center parameters of the object recognition model can be updated to the target category center parameters of the corresponding new object recognition task according to which new object recognition task needs to be performed, thereby obtaining a target object recognition model and realizing object recognition for the new object recognition task. In this way, there is no need to retrain the object recognition model, improving the realization efficiency of adding the recognition capability of the new object recognition task to the object recognition model. Adding the recognition capability of the new object recognition task to the object recognition model does not affect the recognition effect of the object recognition model for the existing object recognition task, ensuring the recognition effect of the target object recognition model for the existing object recognition task, and improving the recognition accuracy of the target object recognition model to which the new object recognition task has been added.

[0071] For example, refer to FIG. 6, which is a schematic diagram of updating an object recognition model according to an embodiment of the present application. Here, the object recognition model includes a convolutional feature extraction layer (which can be constructed based on a convolutional neural network), a pooling feature extraction layer, an embedding feature extraction layer, a first object recognition layer (cosine-match), and a second object recognition layer (softmax layer). In practical application, the softmax layer does not need to set model parameters that need to be learned, and the cosine-match layer can set the category center parameter w of the object recognition model. As shown in FIG. 6, the new object recognition task includes game recognition task 1 and game recognition task 2. Here, the category center parameter W1 for game recognition task 1 is used to represent the multiple category centers [u1..., uN] corresponding to each category (game object) in game recognition task 1, and the category center uO of other categories (game objects) in other game recognition tasks (different from game recognition task 1), and the category center parameter W2 for game recognition task 2 is used to represent the multiple category centers [u1..., uM] corresponding to each category (game object) in game recognition task 2, and the category center uO of other categories (game objects) in other game recognition tasks (different from game recognition task 2).

[0072] In some embodiments, the target category center parameters include a plurality of target subcategory center parameters, each of which corresponds one-to-one to a category center of each category in the target object recognition task, and the target object recognition task includes at least one object recognition task and a new object recognition task. Referring to FIG. 7 , FIG. 7 is an exemplary flowchart of a method for updating an object recognition model according to an embodiment of the present application, the method including the following steps: In step 201, a first object recognition is performed on an object image of an object to be recognized based on each target subcategory center parameter using the target object recognition model, to obtain an initial probability that the object to be recognized belongs to each category center. In step 202, a probability that the object to be recognized belongs to each category is determined based on the initial probability corresponding to each category center. In step 203, an object category to which the object to be recognized belongs is determined based on the probability that the object to be recognized belongs to each category.

[0073] As can be seen from the above example, each target subcategory center parameter is actually a target sample feature corresponding to the clustering center of a sample feature cluster obtained by the clustering process. Therefore, in step 201, first, feature extraction is performed on the object image of the object to be recognized using the target object recognition model to obtain object image features. Then, the following process can be performed for each target subcategory center parameter, including determining the similarity (e.g., cosine similarity) or distance (e.g., Euclidean distance) between the target sample feature corresponding to the target subcategory center parameter and the object image feature of the object to be recognized, and using the similarity or distance as an initial probability that the object to be recognized belongs to the category center corresponding to the target subcategory center parameter. In step 202, the following process can be performed for each category, including determining the maximum initial probability (i.e., maximum similarity or maximum distance) from the initial possibilities corresponding to each category center of the category, and using the maximum initial probability as the probability that the object to be recognized belongs to the category. In step 203, in some embodiments, the category in which the category center corresponding to the maximum likelihood is located may be the object category to which the object to be recognized belongs.

[0074] In some other embodiments, referring to FIG. 8, FIG. 8 is an exemplary flowchart of an object recognition model updating method according to an embodiment of the present application. FIG. 8 shows that step 203 in FIG. 7 can be further implemented through steps 2031 to 2034. In step 2031, the maximum probability is determined from the probability that the object to be recognized belongs to each category, and a first recognition task in which the category corresponding to the maximum probability is located is determined, and the first recognition task belongs to the target object recognition task. In step 2032, a first probability that the object to be recognized belongs to each category in a second recognition task is determined from the probability that the object to be recognized belongs to each category, and the maximum first probability is determined from the multiple first possibilities, where the second recognition task is a recognition task other than the first recognition task in the target object recognition task. In step 2033, the task entropy of the first recognition task is determined based on the maximum probability and the maximum first probability. In step 2034, if the task entropy is less than the task entropy threshold, it is determined that the object to be recognized belongs to the object category in the first recognition task that corresponds to the highest likelihood.

[0075] In step 2031, a first recognition task is determined to include the category corresponding to the maximum likelihood. In step 2032, a first likelihood and a maximum first likelihood that the object to be recognized belongs to each category in a second recognition task (a recognition task different from the first recognition task in the target object recognition task) are determined. In step 2033, a task entropy for the first recognition task is determined based on the maximum likelihood and the maximum first likelihood. Here, the task entropy can be obtained by the following formula: task entropy = -a * ln(a) - b * ln(b), where a is the maximum first likelihood and b is the maximum likelihood. In practical application, the smaller the task entropy, the higher the likelihood that the object to be recognized belongs to the object category in the first recognition task. Therefore, a task entropy threshold is determined in advance. If the task entropy corresponding to the object to be recognized is smaller than the task entropy threshold, it can be determined that the object to be recognized belongs to the object category in the first recognition task, i.e., that the object to be recognized belongs to the object category in the first recognition task that corresponds to the maximum likelihood. In this way, by determining the magnitude relationship between the task entropy and the task entropy threshold, it is possible to effectively distinguish between the target category to which the object to be recognized belongs and other categories, thereby improving the accuracy of object recognition.

[0076] In some embodiments, if the task entropy is equal to or greater than the task entropy threshold, the server may determine an object category to which the object to be recognized belongs in the following manner: if the number of second recognition tasks is one, determine that the object to be recognized belongs to a first category in the second recognition task, where the first category corresponds to the first maximum possibility; if the number of second recognition tasks is multiple, determine an object category to which the object to be recognized belongs based on a first possibility that the object to be recognized belongs to each category in the second recognition task.

[0077] If there are multiple second recognition tasks, the implementation method of step 203 can be used to determine whether the object to be recognized belongs to a certain category in the second recognition task, and to determine the object category in the specific second recognition task to which the object to be recognized belongs. Specifically, the maximum probability and a target second recognition task in which the category corresponding to the maximum probability is located are determined from the possibility that the object to be recognized belongs to each category in the second recognition task, a first possibility that the object to be recognized belongs to each category in a third recognition task is determined from the possibility that the object to be recognized belongs to each category in the second recognition task, and the maximum first possibility is determined from the multiple first possibilities in the third recognition task, where the third recognition task is a recognition task other than the target second recognition task in the multiple second recognition tasks, and the task entropy of the target second recognition task is determined based on the maximum possibility and the maximum first possibility, and the task entropy is calculated as the task entropy. If the task entropy is less than the threshold, it is determined that the object to be recognized belongs to the object category in the target second recognition task that corresponds to the maximum possibility. If the task entropy is greater than or equal to the task entropy threshold (the object to be recognized does not belong to the target second recognition task), if there is one third recognition task, it is determined that the object to be recognized belongs to the first category in the third recognition task, where the first category corresponds to the maximum first possibility in the third recognition task. If there are multiple third recognition tasks, the object category to which the object to be recognized belongs is determined based on the first possibility that the object to be recognized belongs to each category in the third recognition task. Here, the processing steps when there are multiple third recognition tasks can refer to the processing steps when there are multiple second recognition tasks, and they will not be repeated here. These operations can be repeated to determine the object category in the target recognition task to which the object to be recognized belongs.

[0078] When the above-described embodiment of the present application is applied, the object recognition model obtained by the training has category center parameters, which are determined based on the sample features of each image sample in each training sample set and can be obtained without model training. If it is necessary to add recognition capabilities for a new object recognition task to the object recognition model, target category center parameters can be determined based on the sample features of each image sample in each training sample set for the object recognition task and the sample features of each image sample in a target training sample set for the new object recognition task, and the category center parameters of the object recognition model can be updated to the target category center parameters, without the need to retrain the object recognition model. The target object recognition model obtained in this case not only has recognition capabilities for at least one object recognition task, but also has recognition capabilities for the new object recognition task. In this way, since there is no need to retrain the object recognition model, 1) the implementation efficiency of adding the recognition capability of a new object recognition task to the object recognition model is improved, and 2) while adding the recognition capability of a new object recognition task to the object recognition model, the recognition effect of the existing object recognition task of the object recognition model is guaranteed without affecting the recognition effect of the object recognition model, and the recognition accuracy of the target object recognition model to which the new object recognition task has been added can be improved.

[0079]

[0013] The following describes an exemplary application of the present embodiment in a practical application scenario. Before describing the present embodiment, an object recognition model in the related art will be described. In the related art, different object recognition tasks (e.g., recognizing game objects of different categories in different games) are usually realized by training one object recognition model using training samples of each object recognition task. When a new object recognition task is added, the object recognition model needs to be retrained using training samples of the new object recognition task so that the object recognition model has the recognition ability for the new object recognition task. This results in low implementation process efficiency, and the retraining of the new object recognition task affects the recognition effect of the existing object recognition task.

[0080] In view of this, embodiments of the present application provide a method for updating an object recognition model to solve at least the problems existing in the related art. In some embodiments, the object recognition model can be used to recognize game objects of each category in at least one game recognition task, as described in detail below.

[0081] (1) Object detection. Here, an object detection model is used to perform object detection on the video image of each frame in an object video (e.g., a game video containing game objects), and then provide object screenshots (i.e., image samples of the object) for subsequent extraction of embeddings (i.e., the above-mentioned sample features) of each category of object. In practical application, the object detection model may employ the open-source Yolov3 detector trained based on the Coco dataset, or may employ a pre-trained object detector based on service data (e.g., the video image of each frame in a game video).

[0082] (2) Building an object recognition model. Here, an object recognition model can be built using deep learning methods in machine learning. The following points should be explained:

[0083] (2.1) To achieve high-speed recognition by the object recognition model, the object recognition model in the embodiment of the present application is based on an embedding feature extraction model, and the object recognition model is built on the embedding feature extraction model. In this way, the object recognition model in the embodiment of the present application does not require feature extraction of the object image of the object to be recognized from the lowest layer, which can reduce the occupation of inference calculation resources and is advantageous for the rapid addition of new object recognition tasks. Therefore, the embedding extracted by the embedding feature extraction model in the embodiment of the present application has expressive capabilities across object recognition tasks.

[0084] (2.2) In order to enable new object recognition tasks to be quickly added to the object recognition model with limited sample data, the object recognition layer of the object recognition model can be designed with a minimum number of waiting-to-train parameters. This is because the more waiting-to-train parameters, the larger the required sample data. However, the available sample data for new object recognition tasks is usually limited, so it may not be possible to support the learning requirements of excessive waiting-to-train parameters.

[0085] (2.3) The object recognition model must have the ability to distinguish between objects in a target recognition task and objects in a background recognition task in multiple different object recognition tasks. For example, in the case of game recognition task 1, all objects in other game recognition tasks other than non-game recognition task 1 belong to the object in the background recognition task. In other words, the object recognition model in the embodiment of the present application has the ability to recognize out-of-area data and can support the determination of the relationship between objects and object recognition tasks.

[0086] (2.4) Adding a new object recognition task to an object recognition model is a dynamic process. For example, if an object recognition model supports the recognition of n1 categories of objects in one object recognition task, the object recognition model updating method according to an embodiment of the present application can add a new object recognition task (used to recognize n2 categories of objects in the new object recognition task) to the object recognition model, thereby enabling the object recognition model to recognize two object recognition tasks. This operation can be repeated to support the recognition of more object recognition tasks. In contrast, the related art first trains an object classification model for n1 categories of objects in one object recognition task, and when a new object recognition task is added, it trains an object classification model for (n1 + n2) categories of objects, and then repeats this operation. However, in the related art, because the model classification branch parameters (from n1 categories to (n1+n2) categories) are changed again, the model classification effect of the object classification model for (n1+n2) category objects on n1 category objects changes significantly compared to the model classification effect of the n1 category object classification model on n1 category objects, resulting in a situation where the object classification effect of some categories deteriorates. Therefore, in the embodiment of the present application, by maintaining the classification of n1 category objects and adding a new classification branch for n2 category objects, the impact of a new object recognition task on an existing object recognition task can be more controlled, there is no need for repeated training, it is highly efficient, and it is advantageous for accurately optimizing subsequent service upgrades and maintenance for specific object recognition tasks.

[0087] (3) Model Structure of the Object Recognition Model. In actual applications, the object recognition model is built on the embedded feature extraction model. The structure of the embedded feature extraction model is as shown in Table 1 (basic feature extraction layer) and Table 2 (embedding feature extraction layer), and the structure of the object recognition model is as shown in Table 3. Here, the input of the basic feature extraction layer shown in Table 1 is the object image of the object to be recognized, the output of the basic feature extraction layer shown in Table 1 is the input of the embedding feature extraction layer shown in Table 2, the output of the embedding feature extraction layer shown in Table 2 is the input of the object recognition model shown in Table 3, and the output of the object recognition model shown in Table 3 is the object category of the object to be recognized. Here, the embedded feature extraction model includes the above-mentioned convolutional feature extraction layer (Conv1 to Conv5 shown in Table 1), the pooling feature extraction layer (pool layer shown in Table 2), and the embedded feature extraction layer (embedding layer shown in Table 2). [Table 1] [Table 2]

[0088] Here, the input of the embedding feature extraction layer shown in Table 2 is the output of the basic feature extraction layer shown in Table 1 above. Taking into consideration compatibility when applying the object recognition model, an embedded feature extraction model can be trained in advance, and the object recognition model can be built based on the embedded feature extraction model. In this way, in the embodiments of the present application, when training the object recognition model, there is no need to change the model parameters of the embedded feature extraction model; that is, when training the object recognition model, the model parameters of the models shown in Tables 1 and 2 can be fixed and not updated. [Table 3]

[0089] Here, the input of the object recognition model shown in Table 3 is the output of the Embedding feature extraction layer shown in Table 2 above, i.e., the Embedding. In the object recognition model shown in Table 3, first, a fully connected layer (FC, i.e., the feature mapping layer described above) is used to perform nonlinear mapping on the embedding features to obtain mapping features. Then, a normalization layer is used to normalize the mapping features (dividing each element of the input vector modulo the vector) and normalize the mapping features to the unit hypersphere. Next, a cosine-match layer (i.e., the first object recognition layer) is used to perform first object recognition and obtain the predicted probability that the object to be recognized belongs to each category center. Nx is the number of category center centers included in the object categories supported for recognition by the object recognition model, and Nc is the number of object categories supported for recognition by the object recognition model. For example, if the object recognition model supports recognition of 81 categories of game objects, Nc = 81, and Nx is determined by the number of category center centers of game objects in each category. Finally, a softmax layer (second object recognition layer) is used to determine the predicted probability that the object to be recognized belongs to each category, and the predicted probability of each category is mapped between 0 and 1.

[0090] Each category includes at least one category center. For example, in a game recognition task, a category center may be at least one game object prototype of the game object of the category to which it belongs. The game object prototype is a different expression form of the game object. For example, when a game object activates game skill 1, game skill 2, or game skill 3, the object's standard style when standing and walking can be regarded as five game object prototypes of the game object. [Table 4]

[0091] In practical applications, it is not necessary to set the model parameters to be learned in the normalization layer, cosine-match layer, and softmax layer. The Fc layer generates the embeddings required for object recognition, so it can set the model parameters to be learned. The cosine-match layer can set the category center parameter w of an object recognition model. This category center parameter w is generated based on the sample features of the training samples, eliminating the need for model training. In actual implementation, the sample features can be based on the embeddings of the training samples generated by the Fc layer. Therefore, during the object recognition model training process, the category center parameter w of the object recognition model can be updated after the model iteration is completed (as shown in Table 4). Adding a new object recognition task to the object recognition model can be achieved by updating the category center parameter w of the object recognition model.

[0092] (4) The training process of the object recognition model.

[0093] (4.1) Data Preparation. Here, a training sample set for each object recognition task is constructed by collecting image samples of objects of each category in each object recognition task of a target number of object recognition tasks to be learned. The training sample set for the target number of object recognition tasks may be referred to as a basic training sample set. For example, a target number of image samples, e.g., 25 image samples, can be collected for each category of objects in each object recognition task (ensuring that at least 20 of the image samples are used in the training set and at the same time, 5 image samples are used in the test set). When training an object recognition model based on the training sample set for each object recognition task, one object recognition task can be randomly selected from the target number of object recognition tasks as the target recognition task (e.g., game recognition task 1). In this case, each category in the target recognition task becomes the target category, and each category in the other recognition tasks becomes the background category. In this way, after training based on the training sample set for the target recognition task is completed, the object recognition model can recognize whether the object to be recognized belongs to the target category or the background category, and the specific category within the target category. Subsequently, after the training based on the training sample of the target recognition task is completed, one object recognition task (different from the previously specified object recognition task) is randomly selected from the target number of object recognition tasks as the target recognition task for training, and this operation is repeated to achieve the purpose of training an object recognition model based on the training sample set of each object recognition task.

[0094] (4.2) Training of Object Recognition Model. Here, when the object recognition model is finally applied, the object categories it supports for recognition may change (e.g., currently it is applying Game Recognition Task 1, and in two weeks it will need to apply a new Game Recognition Task 2). At the same time, the number of image samples for each category of objects may not be large (e.g., more than 25 images). In this case, to support the rapid addition of new object recognition tasks to the object recognition model, i.e., to enable the object recognition model to be applied to new object recognition tasks without retraining when finally applied, in the present embodiment, the following object recognition model training and data sampling process is designed to support direct updating of the object recognition model without retraining it, and to support downstream object recognition tasks (i.e., newly added object recognition tasks). When training an object recognition model, first, before starting training, randomly select one object recognition task from a target number of object recognition tasks as the target recognition task, and construct a training sample set for that target recognition task while confirming that it is close to the actual data distribution. Then, train that target recognition task based on the training sample set until the training completion standard for that target recognition task is reached (for example, training a specified number of N_project times), and then randomly select another object recognition task as the target recognition task, construct a training sample set, and train that target recognition task. This process is repeated until all object recognition tasks have reached the training completion standard.

[0095] (4.2.1) Construction of a training sample set. Randomly select one object recognition task from the target number of object recognition tasks as the target recognition task. For n (e.g., n = 30) categories of objects in the target recognition task, k1 image samples are sampled for each object i in each category to obtain a training sample set for the target recognition task. For example, k1 can satisfy k1 = max(N_i_sample, 25), where N_i_sample is a random value selected from the range [25, total number of image samples for object i in each category]. For example, the number of image samples for object i in each category is 25 or more. Then, select the remaining k2 (e.g., 5) image samples from the image samples of the n categories of objects to construct a test set. Training of the training sample set is completed after a specified number of iterations (e.g., 10) is performed using the training sample set. Then, another target recognition task is randomly generated to construct a training sample set and a test set, and a new round of training is continued, repeating this process until convergence is reached (e.g., the loss no longer decreases and the accuracy of the test results no longer improves). Note that the k1 limit is intended to balance (equalize) the amount of samples for each category when there is a shortage of samples, thereby ensuring that each category has an equal opportunity to learn and improving the model training effect. This effectively improves the recognition ability of the object recognition model with limited training samples. When there is a sufficient amount of samples, the value of k1 can be set as needed and is not limited to the conditions that k1 must satisfy.

[0096] (4.2.2) Model Parameter Update Process During Training. In this embodiment, the model parameters of the object recognition model (including the convolution template parameter α and the bias parameter β) are updated using stochastic gradient descent (SGD). In each iteration, the error of the prediction result is calculated and backpropagated to the object recognition model, the gradient is calculated, and the model parameters of the object recognition model are updated. The specific process is as follows: All parameters of the object recognition model waiting to be learned are set to a to-be-learned state. During training, the object recognition model performs forward calculations on input image samples to obtain prediction results, compares the prediction results with the labels of the image samples, calculates a loss value for the object recognition model, and backpropagates the loss value to the object recognition model. The model parameters are updated using stochastic gradient descent, thereby achieving one-time optimization of the model parameters. After multiple optimization rounds, a high-performance object recognition model is finally obtained.

[0097] (4.2.3) Category Center Parameter. As shown in FIG. 6, the cosine-match layer is configured with a category center parameter w of the object recognition model, which is used to represent multiple category centers [u1..., uN] corresponding to each category in each object recognition task. In an embodiment of the present application, each category includes multiple category centers. For example, in a game recognition task, a category center can represent multiple game object prototypes of game objects in the category to which it belongs. The game object prototypes are different representations of the game object. For example, the standard styles of an object when the game object activates game skill 1, game skill 2, and game skill 3, as well as when it is standing and walking, can be considered as five game object prototypes of the game object. Other actions and states of the game object can be considered as slight variations of these five game object prototypes. Therefore, the object recognition model can find the game object prototype corresponding to a game object in a specific state in an image sample simply by learning the representations of these five game object prototypes (i.e., the category center parameter). In this way, by setting the category center parameter w in the object recognition model, the object recognition model can find the category center to which the object to be recognized belongs during object recognition simply by learning the category center parameter w. In embodiments of the present application, multiple category centers can be set to prevent each image sample from converging to the same category center (e.g., the same game object prototype), thereby avoiding a loss of diversity in object representations due to object representations becoming too close to a single center, and further avoiding the problem that object representations cannot simultaneously support object duplication elimination and other object recognition that requires attention to different object representations, i.e., avoiding the overfitting problem of single-center classification for object recognition in multiple recognition tasks.

[0098] Generating category center parameters: For each image sample of each object, generate a sample embedding output by the Fc layer shown in Table 3. Then, perform a clustering process with Kn clustering centers on all sample embeddings of each object, e.g., a clustering process with five clustering centers, to obtain Kn category centers, which are recorded in the category storage unit and used to construct category center parameters. For example, the object recognition model in the current training stage supports the recognition of objects in 81 categories (including 80 target categories for the target recognition task within the target number of object recognition tasks, and the remaining one category is a background category). In this case, for the target recognition task, Kn category centers are generated for objects of each target category in the target recognition task, obtaining a total of 80Kn category centers; for the others category (i.e., the background category, which is each category in the recognition task other than the target recognition task in the target number of object recognition tasks), r*Kn category centers are generated, where r=Nothers / Nhero (Nhero is the number of image samples of objects in each category of the others category, and Others is the total number of image samples of objects in the others category). For example, if Others=1000 and Nhero=25, then r=40; in the current object recognition model, Kn=5 and Nc=81, and the total number of category centers is Nx=(Nc-1)*Kn+r*Kn=600.

[0099] Updating the category center parameter: The category center parameter is generated based on the sample features of the training sample and does not need to be obtained by model training. In actual implementation, the sample features may be based on the embedding of the training sample generated by the Fc layer of the object recognition model. Therefore, during the learning process of each object recognition task, the model parameters of the object recognition model change after each iteration, so the category center parameter w of the object recognition model needs to be updated. This can be achieved by following the above-mentioned category center parameter generation method.

[0100] (4.2.4) Loss calculation for the object recognition model. For each image sample, a sample embedding (denoted as p) is generated by the Fc layer shown in Table 3 through the object recognition model. Then, the cosine similarity between p and all Nx category centers is calculated to obtain the predicted result of the image sample at each category center. From the Kn category centers with pre-assigned categories, the category center with the highest cosine similarity (denoted as y) is selected as the target sample embedding for the image sample to learn. At the same time, the cosine similarity between p and other non-category category centers must be minimized. The loss for each image sample can be determined by the following equation (1):

[0101]

number

number

[0102] For each training sample set, calculate the sample loss of each image sample in the training sample set based on the above formula (1), and then calculate the average value of each sample loss to obtain the loss of the object recognition model.

[0103] (5) Application of object recognition model. For each object image to be recognized, object recognition is performed using the cosine-match layer shown in Table 3, and a predicted value (i.e., predicted probability) is obtained that the object to be recognized belongs to each category center. In other words, predicted values ​​for Nx category centers are output, and then predicted values ​​for Nc categories are obtained by taking the maximum predicted value from the predicted values ​​for multiple category centers for each category as the predicted value for that category. The predicted values ​​for each category are mapped between 0 and 1, and prediction results for the Nc categories of the object to be recognized are obtained.

[0104] (6) Adding a new object recognition task to the object recognition model. The above process trains an object classification representation (i.e., extracting embeddings), ensuring that the object recognition model can support adding new object recognition tasks without retraining, even if the object recognition task changes. Specifically, a target training sample set for the new object recognition task (e.g., a certain number of image samples of each category of objects in the new object recognition task) is constructed. Then, embeddings for each image sample in the target training sample set are obtained based on the Fc layer of the object recognition model. The cosine similarity between the embeddings for each image sample in the target training sample set and the embeddings for each image sample in the base training sample set is calculated, and the 5% of image samples with the highest cosine similarity are removed from the base training sample set to obtain a target base training sample set. Finally, target category center parameters are generated based on the target base training sample set and the target training sample set. The target category center parameters are used to update the existing category center parameters of the object recognition model, resulting in a target object recognition model that can support the new object recognition task.

[0105] (7) Controlling the interference of background category objects with target category objects when recognizing object categories. Before applying the object recognition model, a task entropy threshold is first determined, which includes the following steps: (1) Input all test samples into the object recognition model to obtain predicted values ​​for Nc categories; (2) Determine the maximum predicted value (e.g., 0.7) among the Nc categories, and determine a background category different from the category corresponding to the maximum predicted value among the Nc categories, as well as the maximum background category predicted value among the background categories; (3) For each test sample, calculate the entropy of the maximum predicted value and the maximum background category predicted value, and use this as the entropy of the target recognition task to which the test sample belongs (i.e., the category corresponding to the maximum predicted value), which is called the task entropy of the test sample. For example, for a test sample with a maximum background category prediction value of 0.1 and a maximum prediction value of 0.7, its task entropy is -0.1*ln(0.1)-0.7*ln(0.7)=0.48. For a test sample with a maximum background category prediction value of 0.45 and a maximum prediction value of 0.45, its task entropy is 0.72. As can be seen, the larger the task entropy, the less likely the test sample is to belong to a category in the target recognition task. (4) For the task entropies of all test samples, a threshold query is performed within the range of 0.10 to 0.99, with a step size of 0.02, depending on whether the test sample belongs to a category in the target recognition task. The optimal task entropy threshold thr that can be used to distinguish whether the test sample belongs to a category in the target recognition task is found.

[0106] When applying an object recognition model to perform object recognition, an object image of the object to be recognized is first input into the object recognition model, which outputs predicted values ​​for each category and determines the maximum predicted value and corresponding target category. The task entropy of the object image is then calculated. If the task entropy is less than the task entropy threshold thr and the maximum predicted value is greater than the maximum background category predicted value, the object to be recognized is considered to belong to the target category; otherwise, it is considered to belong to the background category. Because a wide range of background categories exists, objects similar to objects in the target category may appear in the background category. Therefore, directly taking the maximum predicted value will likely result in background category objects being recognized as target category objects. Therefore, entropy must be used to judge information; if the target category and background category cannot be distinguished, entropy will always be large. In this way, combining information entropy and a probabilistic index for judgment improves the accuracy of object recognition.

[0107] 9A, which is a schematic diagram of the application of an object recognition model according to an embodiment of the present application. Here, 1) for a newly received video, a) the object detection model is used to perform object detection for each frame in the video to obtain an object screenshot. b) The object to be recognized in the object screenshot is recognized using the object recognition model according to an embodiment of the present application to obtain a deduplication embedding + recognition result, where the deduplication embedding is the output of Table 2 above and the recognition result is the output of Table 3 above. (c) The deduplication embeddings of the video are stored in a deduplication inventory from front to back in the order of frame numbers in the video, and the frame number corresponding to each deduplication embedding and the deduplication embedding below each frame number are recorded.

[0108] 2) Reconstructing the Video Slice Inventory: Based on the deduplication embedding library of historical videos, the object recognition model in Table 3 above is applied to perform object recognition on the deduplication embeddings, obtaining recognition results for each deduplication embedding. Based on the recognition results for the deduplication embeddings for each frame number in the video, if two previous and following frames contain the same object (e.g., both contain object A), the two previous and following frames are considered to belong to a video fragment of a specific object. Similarly, the previous and following objects are integrated for all frames of the video. Based on the time periods in which the objects appear, the original video is divided into multiple video fragments, each corresponding to one object (note that if multiple objects appear in a frame, that frame can be discarded). The video fragments and the corresponding object deduplication embeddings are stored in the object video library, and the number of views of the original video in which the video fragment is located is stored as the popularity of the object.

[0109] 3) When querying videos based on a query object (e.g., a game object), a deduplication search is required for the multiple queried videos. Therefore, object detection is first performed on the input query video (including the query object). Then, deduplication embeddings and recognition results for the query video are extracted, and search weights for the objects in the query video are obtained based on the recognition results. The deduplication embeddings are used to search for similar inventory embeddings from the deduplication inventory (where the threshold for deduplication embedding similarity is Ks, and similarities exceeding the threshold Ks are considered similar). Based on the search weights, videos with similar inventory embeddings are recalled from the video slice library. Based on the inventory embeddings of the recalled videos, the similarity between the inventory embeddings of each recalled video and the deduplication embedding of the query video is calculated. Based on the threshold, the number of similar frames is counted, and the overlap rate of the two videos is calculated based on the ratio of the number of similar frames to the total number of frames. Finally, deduplication search results corresponding to the query video are output based on the overlap rate.

[0110] 4) Regarding video recommendation, based on the query object, the video corresponding to the inventory embedding similar to the de-duplicated embedding of the query object is retrieved from the object video inventory as the recall video, and the object category of the recall video is compared with the object category of the query video. If the object categories of the two are different, the video is not recalled. Finally, the recall videos with the same object category are sorted in descending order of popularity, and recommended videos are output to the user based on the sorting results.

[0111] 5) Adding a new object recognition task: The object recognition model is upgraded to add a new object recognition task to the object recognition model. When constructing a new object recognition task, new category-centered parameters that support the new object recognition task are generated based on the above steps, and then the new category-centered parameters can be combined to perform object recognition for the new object recognition task. In other words, for the added, different new object recognition task, object recognition can be performed by directly combining the new category-centered parameters of the new object recognition task based on the object recognition model.

[0112] 9B, which is a schematic diagram of the application of an object recognition model according to an embodiment of the present application, wherein a front end A receives an image to be recognized (including an object to be recognized) and then uploads it to a back end, and the back end recognizes the object to be recognized using an object recognition model according to an embodiment of the present application, and outputs a recognition result (i.e., an object category to which the object to be recognized belongs) and returns it to the front end A.

[0113] Applying the above-described embodiments of the present application provides: a) the ability to effectively learn object recognition with limited training samples, reducing the impact on the recognition of objects of the target category due to the simultaneous presence of a large number of objects of other categories, and improving the feature representation ability with limited samples; and b) the ability to incrementally learn new object recognition tasks without affecting existing object recognition capabilities, ensuring the recognition effectiveness of each object recognition task.

[0114] The following describes an object recognition model updating device according to an embodiment of the present application. Referring to Fig. 10, Fig. 10 is an exemplary structural diagram of the object recognition model updating device according to an embodiment of the present application. The object recognition model updating device according to an embodiment of the present application includes: a first acquisition module 1010 configured to acquire an object recognition model, the object recognition model being used to recognize objects of multiple categories in at least one object recognition task, the object recognition model being trained based on a training sample set for each of the object recognition tasks, the training sample set for the object recognition task including image samples of objects of each category in the object recognition task; a second acquisition module 1020 configured to acquire category center parameters of the object recognition model, the category center parameters being determined based on sample features of each of the image samples in each of the training sample sets; and a third acquisition module 1030 configured to acquire a target training sample set for a new object recognition task, the target training sample set being trained based on a training sample set for each of the object recognition tasks including image samples of objects of each category in the object recognition task. The training sample set includes image samples of multiple categories of objects in the new object recognition task, and the third acquisition module 1030 is configured to determine a target category center parameter based on sample features of each of the image samples in each of the training sample sets and sample features of each of the image samples in the target training sample set. The update module 1050 is configured to update the category center parameter of the object recognition model to the target category center parameter to obtain a target object recognition model, wherein the target object recognition model is used to recognize a target category to which an object to be recognized belongs based on the target category center parameter, and the target category is one of multiple categories in the new object recognition task and multiple categories in the at least one object recognition task.

[0115] In some embodiments, the first acquisition module 1010 is further configured to: acquire an initial object recognition model; acquire a training sample set for each of the object recognition tasks, each of the image samples in the training sample sets being labeled; acquire sample features of each of the image samples in each of the training sample sets; determine category center parameters of the object recognition model based on the sample features of each of the image samples in each of the training sample sets; perform object recognition on each of the image samples in each of the training sample sets using the initial object recognition model based on the category center parameters; obtain a recognition result for each of the image samples; update model parameters of the initial object recognition model based on a difference between the recognition result and the label of each of the image samples; and obtain the object recognition model, wherein the model parameters are different from the category center parameters.

[0116] In some embodiments, the first acquisition module 1010 is further configured to perform the following processes for each of the object recognition tasks, respectively: acquiring an object video for the object recognition task, where the object video includes multiple frame video images; determining target video images of multiple frames from the multiple frame video images, where the target video images include multiple categories of objects in the object recognition task; selecting, for each category of objects in the object recognition task, a target number of first video images including objects of the category from the multiple frame target video images, and setting the first video images as image samples of objects of the category; and constructing a training sample set for the object recognition task based on the image samples of objects of each category in the object recognition task.

[0117] In some embodiments, the first acquisition module 1010 is further configured to perform feature extraction on each of the image samples in each of the training sample sets by the feature extraction layer to obtain sample features for each of the image samples; correspondingly, after updating the model parameters of the initial object recognition model, perform feature extraction on each of the image samples in each of the training sample sets by the feature extraction layer with updated model parameters to obtain new sample features for each of the image samples; determine new category center parameters of the object recognition model based on the new sample features of each of the image samples in each of the training sample sets; and update the category center parameters of the object recognition model to the new category center parameters.

[0118] In some embodiments, the number of image samples of objects of each category in the training sample set is multiple, and the first acquisition module 1010 is further configured to determine multiple sample features corresponding to objects of each category from sample features of each of the image samples in each of the training sample sets, perform clustering on the multiple sample features corresponding to objects of each category based on a target number of clustering centers to obtain a target number of sample feature clusters, and generate category center parameters of the object recognition model based on target sample features corresponding to the clustering centers of each of the sample feature clusters.

[0119] In some embodiments, the category center parameters include a plurality of subcategory center parameters, each of which corresponds one-to-one to a category center of each category in the at least one object recognition task; and the first acquisition module 1010 is further configured to: perform feature extraction on each of the image samples in each of the training sample sets to obtain sample features for each of the image samples; perform first object recognition on the sample features of each of the image samples in each of the training sample sets based on each of the subcategory center parameters to obtain a probability that an object in each of the image samples belongs to each of the category centers; determine, for each of the image samples, a category to which an object in the image sample belongs based on the probability that an object in the image sample belongs to each of the category centers; and determine the category to which the object in the image sample belongs as the recognition result.

[0120] In some embodiments, the first acquisition module 1010 is further configured to: perform a convolution process on each of the image samples in each of the training sample sets to obtain convolution features for each of the image samples; perform a pooling process on the convolution features for each of the image samples to obtain pooled features for each of the image samples; perform an embedded feature extraction process on the pooled features for each of the image samples to obtain embedded features for each of the image samples; perform a feature mapping process on the embedded features for each of the image samples to obtain mapping features for each of the image samples; and perform a normalization process on the pooled features for each of the image samples to obtain sample features for each of the image samples.

[0121] In some embodiments, the number of the object recognition tasks is M, and the first acquisition module 1010 is further configured to acquire an initial object recognition model, acquire a training sample set for each of the object recognition tasks, train the initial object recognition model based on the training sample set for the first object recognition task, obtain an intermediate object recognition model for the first object recognition task, train an intermediate object recognition model for the (i-1)th object recognition task using the training sample set for the i-th object recognition task, obtain the intermediate object recognition model for the i-th object recognition task, traverse the i to obtain an intermediate object recognition model for the M-th object recognition task, and use the intermediate object recognition model for the M-th object recognition task as the object recognition model, where M and i are integers greater than 1, and i is less than or equal to M.

[0122] In some embodiments, the number of image samples of objects of each category in the training sample set is multiple, and the number of image samples of objects of each category in the target training sample set is multiple, and the determination module 1040 is further configured to: determine a plurality of sample features corresponding to objects of each category in the training sample set from the sample features of each of the image samples in the target training sample set; determine a plurality of sample features corresponding to objects of each category in the target training sample set from the sample features of each of the image samples in the target training sample set; cluster the plurality of sample features corresponding to objects of each category based on a target number of clustering centers to obtain a target number of sample feature clusters; and generate the target category center parameter based on target sample features corresponding to the clustering centers of each of the sample feature clusters.

[0123] In some embodiments, the determination module 1040 further performs the following processes for each of the training sample sets, the processes including: for each of the image sample features in the training sample set, determining a feature similarity between the sample feature of the image sample and the sample feature of each of the image samples in the target training sample set; and excluding from the training sample set the sample features of the image samples whose feature similarity satisfies a similarity condition to obtain a first training sample set; and is configured to determine a target category center parameter based on the sample features of each of the image samples in each of the first training sample sets and the sample features of each of the image samples in the target training sample set.

[0124] In some embodiments, the target category center parameters include a plurality of target subcategory center parameters, the target subcategory center parameters having a one-to-one correspondence with category centers of each category in a target object recognition task, the target object recognition task including the at least one object recognition task and the new object recognition task, and the update module 1050 is further configured to: perform a first object recognition on an object image of an object to be recognized using the target object recognition model based on each of the target subcategory center parameters; obtain an initial possibility that the object to be recognized belongs to each of the category centers; determine a possibility that the object to be recognized belongs to each of the categories based on the initial possibilities corresponding to each of the category centers; and determine an object category to which the object to be recognized belongs based on the possibility that the object to be recognized belongs to each of the categories.

[0125] In some embodiments, when each of the categories has a plurality of category centers, the update module 1050 is further configured to perform the following process for each of the categories, respectively, including determining a maximum initial possibility from the initial possibilities corresponding to each category center of the category, and determining the maximum initial possibility as a possibility that the object to be recognized belongs to the category.

[0126] In some embodiments, the update module 1050 is further configured to: determine a maximum probability from the possibilities that the object to be recognized belongs to each of the categories, and a first recognition task in which a category corresponding to the maximum probability is located, the first recognition task belonging to the target object recognition task; determine a first probability that the object to be recognized belongs to each category in a second recognition task from the possibilities that the object to be recognized belongs to each of the categories; determine a maximum first probability from the plurality of first possibilities, where the second recognition task is a recognition task other than the first recognition task in the target object recognition task; determine a task entropy of the first recognition task based on the maximum probability and the maximum first probability; and determine that the object to be recognized belongs to the object category in the first recognition task corresponding to the maximum probability if the task entropy is smaller than a task entropy threshold.

[0127] In some embodiments, when the task entropy is greater than or equal to a task entropy threshold, the update module 1050 is further configured to: determine, if the number of second recognition tasks is one, that the object to be recognized belongs to a first category in the second recognition task, the first category corresponding to the maximum first possibility; and, if the number of second recognition tasks is multiple, determine an object category to which the object to be recognized belongs based on a first possibility that the object to be recognized belongs to each category in the second recognition task.

[0128] When the above-described embodiment of the present application is applied, the object recognition model obtained by the training has category center parameters, which are determined based on the sample features of each image sample in each training sample set and can be obtained without model training. If it is necessary to add recognition capabilities for a new object recognition task to the object recognition model, target category center parameters can be determined based on the sample features of each image sample in each training sample set for the object recognition task and the sample features of each image sample in a target training sample set for the new object recognition task, and the category center parameters of the object recognition model can be updated to the target category center parameters, without the need to retrain the object recognition model. The target object recognition model obtained in this case not only has recognition capabilities for at least one object recognition task, but also has recognition capabilities for the new object recognition task. In this way, since there is no need to retrain the object recognition model, 1) the implementation efficiency of adding the recognition capability of a new object recognition task to the object recognition model is improved, and 2) while adding the recognition capability of a new object recognition task to the object recognition model, the recognition effect of the existing object recognition task of the object recognition model is guaranteed without affecting the recognition effect of the object recognition model, and the recognition accuracy of the target object recognition model to which the new object recognition task has been added can be improved.

[0129] An electronic device that implements the object recognition model updating method according to an embodiment of the present application will be described below. Referring to FIG. 11 , FIG. 11 is an exemplary structural diagram of an electronic device 500 that implements the object recognition model updating method according to an embodiment of the present application. The electronic device 500 may be a server or a terminal. The electronic device 500 that implements the object recognition model updating method according to an embodiment of the present application includes at least one processor 510, a memory 550, at least one network interface 520, and a user interface 530. The components in the electronic device 500 are coupled together by a bus system 540. It is understood that the bus system 540 is used to realize connection and communication between these components. In addition to a data bus, the bus system 540 further includes a power bus, a control bus, and a status signal bus. However, for clarity of explanation, various buses are referred to as the bus system 540 in FIG. 11 .

[0130] The processor 510 may be an integrated circuit chip having signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc., where the general-purpose processor may be a microprocessor or any conventional processor, etc.

[0131] The memory 550 may be removable, non-removable, or a combination thereof, and optionally includes one or more storage devices that are physically remote from the processor 510. The memory 550 may include volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 550 described in the present embodiments is intended to include any suitable type of memory.

[0132] The memory 550 may store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof. In an embodiment of the present application, the memory 550 may store computer-executable instructions that, when executed by the processor 510, cause the processor 510 to perform a method for updating an object recognition model according to an embodiment of the present application.

[0133] An embodiment of the present application further provides a computer program product including computer-executable instructions stored in a computer-readable storage medium, wherein a processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions to cause the electronic device to perform a method for updating an object recognition model according to an embodiment of the present application.

[0134] An embodiment of the present application further provides a computer-readable storage medium having stored thereon computer-executable instructions that, when executed by a processor, cause the processor to perform a method for updating an object recognition model according to an embodiment of the present application.

[0135] In some embodiments, the computer-readable storage medium may be a memory such as a RAM, a ROM, a flash memory, a magnetic surface memory, an optical disk, or a CD-ROM, or may be a variety of devices including one or any combination of the above memories.

[0136] In some embodiments, the computer-executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted, or declarative or procedural languages), and arranged in any form, such as as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0137] As an example, the computer-executable instructions may, but do not necessarily, correspond to a file in a file system, and may be stored in part of a file that stores other programs or data, such as in one or more scripts within a HyperText Markup Language (HTML) document, in a single file of the discussed program, or in multiple joint files (e.g., files that store one or more modules, subprograms, or code portions).

[0138] As an example, the computer-executable instructions may be arranged to be executed on one electronic device, or on multiple electronic devices at the same location, or on multiple electronic devices distributed across multiple locations and interconnected by a communications network.

[0139] The above description is merely an example of the present application and does not limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and scope of the present application shall all be included in the scope of protection of the present application.

Claims

1. A method for updating an object recognition model, executed by an electronic device, comprising: obtaining an object recognition model, the object recognition model being used to recognize objects of multiple categories in at least one object recognition task, the object recognition model being trained based on a training sample set for each object recognition task, the training sample set for the object recognition task including image samples of objects of each category in the object recognition task; obtaining category center parameters of the object recognition model, the category center parameters being determined based on sample features of each image sample in each training sample set; obtaining a target training sample set for a new object recognition task, the target training sample set including image samples of multiple categories of objects in the new object recognition task; determining a target category center parameter based on sample features of each image sample in each training sample set and sample features of each image sample in the target training sample set; updating category center parameters of the object recognition model to the target category center parameters to obtain a target object recognition model; The method for updating an object recognition model, wherein the target object recognition model is used to recognize a target category to which an object to be recognized belongs based on the target category center parameter, and the target category is one of multiple categories in the new object recognition task and multiple categories in the at least one object recognition task.

2. The step of obtaining the object recognition model includes: obtaining an initial object recognition model and obtaining a training sample set for each object recognition task, wherein each image sample in the training sample set is labeled; obtaining sample features for each image sample in each training sample set; and determining category center parameters of the object recognition model based on the sample features for each image sample in each training sample set; performing object recognition on each image sample in each training sample set using the initial object recognition model based on the category center parameters, and obtaining a recognition result for each image sample; 2. The method for updating an object recognition model according to claim 1, further comprising: updating model parameters of the initial object recognition model based on a difference between the recognition result and the label of each image sample to obtain the object recognition model, wherein the model parameters are different from the category center parameters.

3. The step of obtaining a training sample set for each object recognition task includes: For each object recognition task, the process includes: obtaining an object video for the object recognition task, the object video including multiple frames of video images; determining target video images of a plurality of frames of the plurality of frames of video images, the target video images including objects of a plurality of categories in the object recognition task; selecting a target number of first video images including the object of each category from the target video images of the plurality of frames for each category of object in the object recognition task, and setting the first video images as image samples of the object of the category; and constructing a training sample set for the object recognition task based on image samples of objects of each category in the object recognition task.

4. The initial object recognition model includes a feature extraction layer, and the step of obtaining sample features for each image sample in each training sample set includes: The feature extraction layer performs feature extraction on each image sample in each training sample set to obtain sample features for each image sample; After updating the model parameters of the initial object recognition model, the object recognition model updating method includes: performing feature extraction on each image sample in each training sample set using the feature extraction layer with updated model parameters to obtain new sample features for each image sample; determining new category center parameters of the object recognition model based on new sample features for each image sample in each training sample set; 3. The method of claim 2, further comprising the step of: updating category center parameters of the object recognition model to the new category center parameters.

5. The number of image samples of objects of each category in the training sample set is plural, and the step of determining category center parameters of the object recognition model based on sample features of each image sample in each training sample set includes: determining a plurality of sample features corresponding to objects of each category from the sample features of each image sample in each training sample set; performing clustering on a plurality of sample features corresponding to objects of each category based on a target number of clustering centers to obtain a target number of sample feature clusters; and generating category center parameters of the object recognition model based on target sample features corresponding to clustering centers of each sample feature cluster.

6. the category center parameters include a plurality of subcategory center parameters, the subcategory center parameters corresponding one-to-one to the category centers of each category in the at least one object recognition task; performing object recognition on each image sample in each training sample set based on the category center parameters to obtain a recognition result for each image sample, performing feature extraction on each image sample in each training sample set to obtain sample features for each image sample; performing a first object recognition on the sample features of each image sample in each training sample set based on each subcategory center parameter, and obtaining a probability that the object in each image sample belongs to each category center; 3. The method for updating an object recognition model according to claim 2, further comprising: a step of determining, for each image sample, a category to which the object in the image sample belongs based on the possibility that the object in the image sample belongs to each category center, and setting the category to which the object in the image sample belongs as the recognition result.

7. The step of performing feature extraction on each image sample in each training sample set to obtain sample features for each image sample includes: performing a convolution process on each image sample in each training sample set to obtain convolutional features for each image sample; performing a pooling process on the convolutional features of each image sample to obtain pooled features of each image sample; performing an embedding feature extraction process on the pooled features of each image sample to obtain an embedding feature of each image sample; performing a feature mapping process on the embedding features of each image sample to obtain a mapping feature for each image sample; The method for updating an object recognition model according to claim 6, further comprising: performing a normalization process on the pooled features of each image sample to obtain sample features of each image sample.

8. The number of the object recognition tasks is M, and the step of obtaining the object recognition model includes: obtaining an initial object recognition model and obtaining a training sample set for each object recognition task; training the initial object recognition model based on a training sample set of a first object recognition task to obtain an intermediate object recognition model of the first object recognition task; training an intermediate object recognition model for the (i-1)th object recognition task using the training sample set of the i-th object recognition task to obtain an intermediate object recognition model for the i-th object recognition task; traversing the i to obtain an intermediate object recognition model for the M-th object recognition task, and setting the intermediate object recognition model for the M-th object recognition task as the object recognition model; The method for updating an object recognition model according to claim 1 , wherein M and i are integers greater than 1, and i is equal to or less than M.

9. The number of image samples of each category of object in the training sample set is plural, and the number of image samples of each category of object in the target training sample set is plural; determining a target category center parameter based on sample features of each image sample in each training sample set and sample features of each image sample in the target training sample set, determining a plurality of sample features corresponding to each category of objects in each training sample set from the sample features of each image sample in each training sample set; determining a plurality of sample features corresponding to each category of objects in the target training sample set from sample features of each image sample in the target training sample set; performing clustering on a plurality of sample features corresponding to objects of each category based on a target number of clustering centers to obtain a target number of sample feature clusters; and generating the target category center parameters based on target sample features corresponding to clustering centers of each sample feature cluster.

10. determining a target category center parameter based on sample features of each image sample in each training sample set and sample features of each image sample in the target training sample set, performing the following steps for each training sample set, the steps including: for each image sample in the training sample set, determining a feature similarity between the sample feature of the image sample and the sample feature of each image sample in the target training sample set; and excluding from the training sample set the sample features of the image samples whose feature similarity satisfies a similarity condition to obtain a first training sample set; and determining a target category center parameter based on sample features of each image sample in each first training sample set and sample features of each image sample in the target training sample set.

11. the target category center parameters include a plurality of target subcategory center parameters, the target subcategory center parameters having a one-to-one correspondence with the category centers of each category in a target object recognition task, the target object recognition task including the at least one object recognition task and the new object recognition task; The method for updating the object recognition model includes: performing a first object recognition on an object image of an object to be recognized based on each target subcategory center parameter using the target object recognition model, and obtaining an initial probability that the object to be recognized belongs to each category center; determining the likelihood that the object to be recognized belongs to each category based on the initial likelihood corresponding to each category center; The method for updating an object recognition model according to claim 1 , further comprising: determining an object category to which the object to be recognized belongs based on a possibility that the object to be recognized belongs to each category.

12. When each category has a plurality of category centers, the step of determining an object category to which the object to be recognized belongs based on the initial possibilities corresponding to each category center includes: The method includes the step of performing the following processing for each category, 12. The object recognition model updating method according to claim 11, further comprising the step of determining a maximum initial possibility from the initial possibilities corresponding to each category center of the category, and determining the maximum initial possibility as the possibility that the object to be recognized belongs to the category.

13. The step of determining an object category to which the object to be recognized belongs based on the possibility that the object to be recognized belongs to each category includes: determining a maximum probability from the probability that the object to be recognized belongs to each category, and a first recognition task in which a category corresponding to the maximum probability is located, wherein the first recognition task belongs to the target object recognition task; determining a first possibility that the object to be recognized belongs to each category in a second recognition task from a possibility that the object to be recognized belongs to each category, and determining a maximum first possibility from a plurality of the first possibilities, the second recognition task is a recognition task other than the first recognition task in the target object recognition task; determining a task entropy for the first recognition task based on the maximum likelihood and the maximum first likelihood; and determining that the object to be recognized belongs to an object category in the first recognition task that corresponds to the maximum likelihood if the task entropy is less than a task entropy threshold.

14. If the task entropy is equal to or greater than a task entropy threshold, the method for updating the object recognition model includes: If the number of the second recognition tasks is one, determining that the object to be recognized belongs to a first category in the second recognition task, the first category corresponding to the maximum first possibility; 14. The object recognition model updating method according to claim 13, further comprising: when the number of second recognition tasks is plural, determining an object category to which the object to be recognized belongs based on a first possibility that the object to be recognized belongs to each category in the second recognition task.

15. An object recognition model update device, a first acquisition module configured to acquire an object recognition model, the object recognition model being used to recognize objects of multiple categories in at least one object recognition task, the object recognition model being trained based on a training sample set for each object recognition task, the training sample set for the object recognition task including image samples of objects of each category in the object recognition task; and a second acquisition module configured to acquire category center parameters of the object recognition model, the category center parameters being determined based on sample features of each image sample in each training sample set; and a third acquisition module configured to acquire a target training sample set for a new object recognition task, the target training sample set including image samples of multiple categories of objects in the new object recognition task; a determination module configured to determine a target category center parameter based on sample features of each image sample in each training sample set and sample features of each image sample in the target training sample set; an updating module configured to update category center parameters of the object recognition model to the target category center parameters to obtain a target object recognition model; An object recognition model updating device, wherein the target object recognition model is used to recognize a target category to which an object to be recognized belongs based on the target category center parameter, and the target category is one of multiple categories in the new object recognition task and multiple categories in the at least one object recognition task.

16. a memory storing computer-executable instructions; and a processor configured to execute computer-executable instructions stored in the memory to perform the method for updating an object recognition model according to any one of claims 1 to 14.

17. A computer program that causes a computer to implement the object recognition model updating method described in any one of claims 1 to 14.

Images