Object recognition model generation method and apparatus, and storage medium

Abstract

The application discloses an object recognition model generation method and device and a storage medium, belongs to the technical field of artificial intelligence, and comprises the following steps: obtaining a first object recognition model, the first object recognition model comprising a feature extraction component and multiple detection components, different detection components being trained based on different image sets, the category spaces of the different image sets being not completely same, and each detection component outputting an object recognition result of a corresponding category according to a category serial number in the corresponding image set; determining a first image set, the category space of the first image set comprising multiple first target categories; generating a first serial number list according to the category serial numbers of the first target categories in the category space of the first image set; extracting a second detection component from the first detection component based on the first serial number list, the first detection component being a detection component corresponding to the first object recognition model for the first image set; and generating a second object recognition model based on the second detection component. The embodiments of the application solve the problems of repeated training and repeated generation of models.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus and storage medium for generating object recognition models. Background Technology

[0002] Object recognition, or target detection, is an important task in computer vision, aiming to locate and identify targets in images or videos. Related technologies determine corresponding target categories based on different business requirements, then perform image sampling based on these target categories, and train an object recognition model adapted to the business requirements based on the image sampling results. This object recognition model can only identify the target category corresponding to that specific business requirement.

[0003] Different business needs may correspond to different target categories. For different business needs, it is necessary to train an object recognition model that can identify the specific target category, which leads to the problem of repeated model training and generation. Summary of the Invention

[0004] This application provides a method, apparatus, and storage medium for generating an object recognition model, which can solve the aforementioned problems of repeated model training and generation.

[0005] According to one aspect of the embodiments of this application, a method for generating an object recognition model is provided, the method comprising:

[0006] A first object recognition model is obtained. The first object recognition model includes a feature extraction component and multiple detection components connected to the feature extraction component. Different detection components are trained based on different image sets. The category spaces of different image sets are not completely the same. Each detection component is used to output the object recognition result of the corresponding category according to the category number in the corresponding image set.

[0007] Obtain multiple primary target categories to be identified;

[0008] A first image set is determined, wherein the category space of the first image set includes the plurality of first target categories;

[0009] A first sequence number list is generated based on the category number of each first target category in the category space of the first image set;

[0010] Based on the first sequence number list, a second detection component is extracted from the first detection component. The first detection component is the detection component corresponding to the first image set in the first object recognition model.

[0011] Based on the second detection component, a second object recognition model is generated for identifying the plurality of first target categories.

[0012] According to one aspect of the embodiments of this application, an object recognition model generation apparatus is provided, the apparatus comprising:

[0013] The model acquisition module is used to acquire a first object recognition model. The first object recognition model includes a feature extraction component and multiple detection components connected to the feature extraction component. Different detection components are trained based on different image sets. The category spaces of different image sets are not completely the same. Each detection component is used to output the object recognition result of the corresponding category according to the category number in the corresponding image set.

[0014] The model generation module is used to perform the following operations:

[0015] Obtain multiple primary target categories to be identified;

[0016] A first image set is determined, wherein the category space of the first image set includes the plurality of first target categories;

[0017] A first sequence number list is generated based on the category number of each first target category in the category space of the first image set;

[0018] Based on the first sequence number list, a second detection component is extracted from the first detection component. The first detection component is the detection component corresponding to the first image set in the first object recognition model.

[0019] Based on the second detection component, a second object recognition model is generated for identifying the plurality of first target categories.

[0020] According to one aspect of the embodiments of this application, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement the above-described object recognition model generation method.

[0021] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, code set, or instruction set is stored in the storage medium, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the above-described object recognition model generation method.

[0022] According to one aspect of the embodiments of this application, a computer program product is provided, the computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform an action to implement the above-described object recognition model generation method.

[0023] The technical solution provided in this application can bring the following beneficial effects:

[0024] This application proposes a method, apparatus, and storage medium for generating object recognition models. This application obtains a comprehensive object recognition model (first object recognition model) capable of recognizing categories in the category space of various image sets through a single large-scale multi-image set training. Then, based on specific business requirements, a specific target category (first target category) is determined. A specific object recognition model (second object recognition model) adapted to this specific business requirement is directly customized and derived from the comprehensive object recognition model based on this target category. Clearly, this application does not require targeted model training for specific business requirements. If there are multiple business requirements, it only needs to customize and derive specific object recognition models for each business requirement based on the comprehensive object recognition model, thereby reducing the cost and time consumption of repeated model development and training due to different business requirements. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a schematic diagram of an application runtime environment provided in one embodiment of this application;

[0027] Figure 2 This is a flowchart of an object recognition model generation method provided in one embodiment of this application;

[0028] Figure 3 This is a schematic diagram of the structure of a first object recognition model provided in one embodiment of this application;

[0029] Figure 4 This is a schematic diagram of a second detection component extraction method provided in one embodiment of this application;

[0030] Figure 5 This is a schematic diagram of a row parameter extraction method provided in one embodiment of this application;

[0031] Figure 6 This is a flowchart of the training process of a first object recognition model provided in one embodiment of this application;

[0032] Figure 7 This is a schematic diagram of the training process of multiple detection components provided in one embodiment of this application, each trained using a corresponding image set;

[0033] Figure 8 This is a flowchart of a sampling method provided in one embodiment of this application;

[0034] Figure 9 This is a block diagram of an object recognition model generation apparatus provided in one embodiment of this application;

[0035] Figure 10 This is a structural block diagram of a computer device provided in one embodiment of this application. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be noted that all data used in the embodiments of this application has been fully authorized by the relevant parties before use.

[0037] Please refer to Figure 1 This diagram illustrates an application runtime environment provided in one embodiment of this application under an exemplary implementation. The application runtime environment may include: terminal 10 and server 20.

[0038] Terminal 10 includes, but is not limited to, electronic devices such as mobile phones, computers, smart voice interaction devices, smart home appliances, in-vehicle terminals, game consoles, e-book readers, multimedia playback devices, and wearable devices. Application clients can be installed on terminal 10.

[0039] In this embodiment, the application described above can be any application that runs an object recognition service and supports object recognition. Typically, the application is a video application. Of course, in addition to video applications, other types of applications can also run object recognition services and support object recognition, such as news applications, social applications, search applications, interactive entertainment applications, browser applications, shopping applications, content sharing applications, virtual reality (VR) applications, augmented reality (AR) applications, etc., and this embodiment does not limit this. Optionally, the terminal 10 runs a client of the above-mentioned application.

[0040] Server 20 provides background services to clients of applications in terminal 10, including an object recognition model generation service that generates object recognition models for object recognition. For example, server 20 can be a background server for the aforementioned application. Server 20 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Optionally, server 20 can simultaneously provide background services to applications in multiple terminals 10.

[0041] Optionally, terminal 10 and server 20 can communicate with each other via network 30. Terminal 10 and server 20 can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0042] Please refer to Figure 2 The diagram illustrates a flowchart of an object recognition model generation method according to an exemplary embodiment. This method can be applied to a computer device, which refers to an electronic device capable of data computation and processing. Figure 1 Server 20 in the middle. The method may include the following steps:

[0043] Step S201. Obtain a first object recognition model. The first object recognition model includes a feature extraction component and multiple detection components connected to the feature extraction component. Different detection components are trained based on different image sets. The category spaces of different image sets are not completely the same. Each detection component is used to output the object recognition result of the corresponding category according to the category number in the corresponding image set.

[0044] The first object recognition model is a comprehensive object recognition model that can recognize objects in various categories within the category space of different image sets. Each detection component can identify objects in the category space of its corresponding image set. Obviously, the more detection components there are, the wider the category space covered by the comprehensive object recognition model, and the stronger its comprehensive object recognition capability.

[0045] This application does not limit the structure of the feature extraction component and the detection component. Please refer to... Figure 3This diagram illustrates the structure of the first object recognition model in this embodiment. The feature extraction component is connected to multiple detection components. Each detection component is trained using its specific corresponding image set and can recognize objects in the category space of its corresponding image set. This embodiment does not limit the image sets used by each detection component; various open-source or custom image sets can be used. The more image sets and detection components there are, the stronger the comprehensive object recognition capability of the first object recognition model. The training task of the first object recognition model in this embodiment is a novel multi-dataset object detection task. Unlike conventional object detection, which uses one dataset to train one model, multi-dataset object detection uses multiple datasets to train one model. The training process of this first object recognition model is detailed below.

[0046] This application does not limit the object recognition results of the first object recognition model, which may include the category and bounding box information of each instance detected in the image. The bounding box describes the position and size information of an instance in the image, and can have various expression formats such as x1y1x2y2 (the upper left and lower right coordinates of the bounding box of the instance) or xywh (the center coordinates and width and height of the instance). The first object recognition model can be built on the basis of YOLO by introducing the concept of multiple detection components. YOLO (You Only Look Once) is a popular real-time object detection model that treats object detection as a regression problem.

[0047] YOLO (You Only Look Once) directly predicts bounding boxes and class probabilities from an image. YOLO's main features are speed (object detection requires only one forward pass), global inference (considering information from the entire image when predicting bounding boxes), and strong generalization ability (adapting better to new environments and targets). YOLO works by first dividing the image into multiple grid cells, then each grid cell predicting multiple bounding boxes and multiple class probabilities, and finally using non-maximum suppression to remove redundant bounding boxes. The feature extraction and detection components in the first object recognition model of this application can both be derived from YOLO. The difference is that YOLO is a single-class spatial detection model, while the first object recognition model connects multiple detection components to the feature extraction component, making it a multi-class spatial detection model.

[0048] Step S202. Obtain multiple first target categories to be identified; determine a first image set, wherein the category space of the first image set includes the multiple first target categories.

[0049] To determine specific business needs, the categories of objects that need to be identified can be identified. For example, in a scenario requiring animal identification, the business requirement might be to identify various animals, and the multiple primary target categories would be the various animal categories. Similarly, in a scenario requiring the identification of daily necessities, the business requirement might be to identify various daily necessities, and the multiple primary target categories would be the various daily necessities categories.

[0050] Different image sets do not have completely identical category spaces. For example, various animal categories may all reside in the category space of image set A, while various household item categories may all reside in the category space of image set B. In an exemplary embodiment, obtaining the plurality of first target categories to be identified includes: obtaining a target category set; performing image set-based matching on the categories in the target category set to obtain the plurality of first target categories corresponding to the first image set. The first image set is necessarily the image set used for training the first object recognition model and has its own corresponding detection components.

[0051] The target category set can be understood as a set of object categories required to be identified by specific business needs. The purpose of image set-based matching of categories in the target category set is to determine which image set's category space each category in the target category set belongs to, and to group categories in the same category space together to form multiple target categories corresponding to that category space. The matching result obtained based on the image set may cover at least one category space. For example, embodiments of this application may include multiple first target categories corresponding to the category space of a first image set, and multiple second target categories corresponding to the category space of a second image set. The second image set is necessarily the image set used for training the first object recognition model and has its own corresponding detection components.

[0052] Step S203. Generate a first sequence number list based on the category number of each first target category in the category space of the first image set.

[0053] Each category in the category space of the first image set has a corresponding index. The first index list can be obtained by arranging the indexes of each first target category in the first image set in order.

[0054] Step S204. Based on the first sequence number list, extract the second detection component from the first detection component, wherein the first detection component is the detection component corresponding to the first image set in the first object recognition model; based on the second detection component, generate a second object recognition model for recognizing the plurality of first target categories.

[0055] This application does not limit the specific method for generating a second object recognition model for identifying the multiple first target categories based on the second detection component. An existing or pre-trained feature extraction network can be directly connected to the second detection component to construct the second object recognition model. The feature extraction network only needs to have feature extraction capabilities; the method of obtaining these features is not limited in this application. In an exemplary embodiment, the feature extraction component in the first object recognition model can also be directly connected to the second detection component to obtain the second object recognition model. The second object recognition model in this application can be understood as a lightweight, specific object recognition model with the ability to recognize multiple first target categories. It can adapt to specific business needs and operate easily with minimal storage and computing resources.

[0056] In one embodiment, each detection component includes a fully connected network layer for outputting object recognition results. Each row of the parameter matrix corresponding to the fully connected network layer is used to output the object recognition result for the category corresponding to the category number indicated by its row number. Please refer to... Figure 4 This diagram illustrates a method for extracting a second detection component according to an embodiment of this application. The step of extracting a second detection component from a first detection component based on the first sequence number list includes:

[0057] S401. For each number in the first sequence number list, in the fully connected network layer of the first detection component, extract the row parameters indicated in the corresponding parameter matrix for the row number that is the same as the sequence number.

[0058] Please refer to Figure 5 The diagram illustrates a row parameter extraction method according to an embodiment of this application. Taking the Lvis image set as an example, the category space of this image set includes 1203 categories. The fully connected layer of the first detection component has a parameter matrix with 1203 rows, and the row numbers of this parameter matrix are 0-1202. For the i-th row, where i takes the value [0, 1202], all the parameters of the i-th row, i.e., the row parameters corresponding to i, are used to output the object recognition result of the i-th class in the category space of the Lvis image set. Lvis is an abbreviation for Large Vocabulary Instance Segmentation, a dataset focused on large-scale instance segmentation tasks. It was released by Facebook AlResearch in 2019. Lvis images are extremely large in scale, with a large number of categories and a long-tailed distribution of instance numbers, making it suitable for training and evaluation of low-sho detection and open-vocabulary detection models.

[0059] exist Figure 5 Originally, the Lvis detection head supported the detection of 1203 categories, with a weight size of 1203*C, where C corresponds to the columns of the parameter matrix of the fully connected network layer of the first detection component. For specific business needs, if only trash cans, bottles, and boxes need to be detected in the Lvis dataset (i.e., multiple first target categories include only trash cans, bottles, and boxes), their category space indices in the first image set would be 22, 132, 142. Therefore, a first index list Lvis[22, 132, 142, ...] can be generated. Based on this first index list, the row parameters corresponding to row numbers [22, 132, 142] are extracted to form a new parameter matrix of K*C. A new fully connected network layer is then generated based on this new parameter matrix, where K is the number of elements in the first index list. Clearly, by constructing different index lists for different business needs, specific detection components that meet different business requirements can be derived, thereby constructing specific object recognition models.

[0060] S402. Generate a sub-fully connected network layer based on the extracted row parameters, wherein the sub-fully connected network layer is used to constitute the second detection component.

[0061] In this embodiment of the application, for a specific business, it is not necessary to use the detection components corresponding to each category space of the first object recognition model. By using the fully connected network layer to achieve the segmentation of the recognition capability for a specific target category, a specific detection component for recognizing a specific target category can be obtained, thereby constructing a specific object recognition model.

[0062] In an exemplary embodiment, a second image set can also be determined, the category space of the second image set including the plurality of second target categories; a second sequence number list is generated according to the category number of each second target category in the second image set; based on the second sequence number list, a fourth detection component is extracted from the third detection component, the third detection component being the detection component corresponding to the second image set in the first object recognition model; based on the fourth detection component, a third object recognition model for recognizing the plurality of second target categories is generated. The process of generating the third object recognition model is based on the same inventive concept as the process of generating the second object recognition model, and will not be elaborated here. Obviously, the embodiments of this application train a multi-category spatial object recognition model (first object recognition model) capable of recognizing objects in the category space of multiple image sets using multiple image sets at once. When applied to a specific scenario, the required detection categories can be selected according to specific business needs, and new detection components can be extracted by cutting the fully connected network layer, thereby generating a specific object recognition model adapted to a specific scenario. It supports training a multi-category spatial object recognition model once, and multiple specific object recognition models that meet different detection category requirements can be customized and exported according to needs.

[0063] Related technologies, when applied to specific business applications, require training tailored to different business needs, leading to repetitive training and development issues. This invention leverages the object detection capabilities of the YOLO network, combined with multi-image set training, to train a network capable of recognizing nearly 2,000 categories (the first object recognition model). Specific categories from the fully connected network layers are extracted based on specific business requirements for customized network use. Unlike related technologies, this invention eliminates the need for repeatedly designing label spaces and training different models for different business needs. Instead, it achieves customized models through a single training iteration, extracting categories based on specific business requirements. This significantly reduces the resource and time costs associated with repetitive training.

[0064] Please refer to Figure 6 This document illustrates a flowchart of the training process for a first object recognition model according to an embodiment of this application. The first object recognition model is trained using the following method:

[0065] S601. For each of the image sets, obtain the image set number corresponding to the image set.

[0066] Please refer to Figure 7 This diagram illustrates the training process of multiple detection components in this application embodiment, each trained using a corresponding image set. Each image set has a corresponding number. This application embodiment does not limit the image sets used. Figure 7 The image sets in this application are all open source image sets, and this application will not elaborate on them in the embodiments. Figure 7 In this context, Dataset_Source represents the image set number.

[0067] S602. Sample a first preset number of first sample images from the image set.

[0068] Taking a certain image set as an example, if the image set is Figure 7 If `objects365` is specified, then `Dataset_Source = 1`, meaning the first sample image and the second sample image (described below) are extracted from `objects365`. This application embodiment does not limit the first preset quantity and the second preset quantity (described below), and can set them according to actual conditions. In some implementations, a ratio between the first preset quantity and the second preset quantity can be set, for example, it can be 1:1 or 2:1.

[0069] S603. Input each of the first sample images into the feature extraction component to obtain the corresponding first sample image features, and input the first sample image features into the detection component corresponding to the image set number to obtain the first sample recognition result corresponding to the first sample image.

[0070] Please refer to Figure 7After the first sample image undergoes multi-scale feature extraction by the Backbone (feature extraction component), multiple feature maps of different scales are obtained. Fusing these feature maps yields the features of the first sample image. These features, combined with the Dataset_Source, are passed to the Multidataset Detect Head. The Multidataset Detect Head can be understood as a component formed by multiple detection components. The Multidataset Detect Head calls the corresponding detection component from objects365 based on the Dataset_Source. Figure 7 The second column of the detection component outputs the identification result of the first sample. Figure 7 In the second column of the detection component, C represents the number of channels of the first sample image features, 365 represents the number of predicted categories, 5 represents the predicted bounding box information and confidence level, and 3 represents the number of predicted anchor boxes. The first sample recognition result includes the predicted value (confidence level) corresponding to each category number. The predicted value corresponding to each category number is output by the corresponding detection component after calculation based on the corresponding row parameters.

[0071] S604. Determine a preset category, wherein the preset category is a category in the category space of the image set that has not been matched by any of the sample recognition results; determine the sampling probability of the preset category based on the frequency of each preset category appearing in the image set.

[0072] In this application embodiment, the lower the frequency of a category, the higher its sampling probability. For example, the preset category C has the lowest frequency, therefore its sampling probability is the highest, and the sampling probability can be proportional to the reciprocal of the frequency. This application embodiment can uniformize the sampling probabilities of each preset category, making their sum equal to 1.

[0073] S605. Based on the sampling probability corresponding to each of the preset categories, a second preset number of second sample images are collected in the image set, and any second sample image includes an object corresponding to any of the preset categories; each second sample image is input into the feature extraction component to obtain the corresponding second sample image features, and the second sample image features are input into the detection component corresponding to the image set number to obtain the second sample recognition result corresponding to the second sample image.

[0074] The method for obtaining the identification results of the second sample is the same as that for obtaining the identification results of the first sample, and will not be elaborated here.

[0075] S606. Based on the difference between the first sample recognition result and the category label corresponding to the first sample image, a first classification loss is obtained; based on the difference between the second sample recognition result and the category label corresponding to the second sample image, a second classification loss is obtained; based on the first classification loss and the second classification loss, the parameters corresponding to the detection component corresponding to the image set number are adjusted until the training requirements are met.

[0076] For example, category labels can include label values ​​corresponding to each category. The label value of the i-th category can indicate whether the object belongs to the i-th category. If it does, the label value corresponding to the i-th category is 1; otherwise, it is 0. This application embodiment does not limit the calculation methods of the first classification loss and the second classification loss. For example, cross-entropy loss can be used. This application embodiment can use the weighted sum of the first classification loss and the second classification loss as the target loss, and adjust the parameters of the detection component corresponding to the image set number based on this target loss until the training requirements are met. This application embodiment does not limit the weights; they can be set according to actual conditions and do not constitute an obstacle to implementation. This application embodiment does not limit the method of adjusting the parameters of the detection component corresponding to the image set number; gradient descent can be used. This application embodiment also does not limit the training requirements, which can be that the number of parameter tuning attempts reaches a preset threshold or the target loss is less than a preset loss threshold. This application embodiment does not limit the preset threshold for the number of tuning attempts and the preset loss threshold, and does not constitute an obstacle to implementation.

[0077] In fact, due to the differences in data volume, category diversity, and category fineness among image sets, as well as the long-tailed distribution of categories within the image sets, overfitting and long-tailed distribution problems may occur between different image sets. Steps S604-S606 can effectively alleviate the category imbalance problem by dynamically sampling and adjusting the sampling probability according to the frequency, improve the model's learning ability for low-frequency categories, and thus solve the problem of reduced object recognition accuracy caused by long-tailed distribution.

[0078] In one exemplary implementation, please refer to Figure 8 The flowchart illustrates a sampling method according to an embodiment of this application. The step of sampling a first preset number of first sample images from the image set includes:

[0079] S801. Determine the first repetition factor of each category in the image set, wherein the first repetition factor is inversely correlated with the frequency of occurrence of the category in the image set.

[0080] If a category appears infrequently, its first repetition factor will be relatively large, meaning that this category will be sampled more frequently during training. This application does not limit the calculation method of the first repetition factor. In an exemplary implementation, the proportion of a category's occurrence (denoted by f) can be obtained first. Then, this proportion is divided by a hyperparameter, and the square root is taken. If the result is less than or equal to 1, the first repetition factor is set to 1; if the result is greater than 1, this result is used as the first repetition factor. This application does not limit the hyperparameter; for example, it can be 0.01.

[0081] S802. Determine the second repetition factor corresponding to each image in the image set, wherein the second repetition factor is the maximum value of the first repetition factor of the category to which the image is matched.

[0082] S803. Based on the second repetition factor corresponding to each of the images, sample the first preset number of the first sample images.

[0083] The frequency of each image in the sampling results is determined by its second repetition factor. This makes images containing less common categories more likely to appear during training, causing the model to focus more on these less common categories during training, thus improving the model's ability to recognize all categories.

[0084] The following are embodiments of the apparatus of this application, which can be used to execute embodiments of the method of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method of this application.

[0085] Please refer to Figure 9 The diagram illustrates a block diagram of an object recognition model generation apparatus according to an exemplary embodiment. This apparatus has the function of implementing the above-described object recognition model generation method; this function can be implemented in hardware or by hardware executing corresponding software. The apparatus can be a computer device or can be installed within a computer device. The apparatus may include:

[0086] The model acquisition module 901 is used to acquire a first object recognition model. The first object recognition model includes a feature extraction component and multiple detection components connected to the feature extraction component. Different detection components are trained based on different image sets. The category spaces of different image sets are not completely the same. Each detection component is used to output the object recognition result of the corresponding category according to the category number in the corresponding image set.

[0087] Model generation module 902 is used to perform the following operations:

[0088] Obtain multiple primary target categories to be identified;

[0089] A first image set is determined, wherein the category space of the first image set includes the plurality of first target categories;

[0090] A first sequence number list is generated based on the category number of each first target category in the category space of the first image set;

[0091] Based on the first sequence number list, a second detection component is extracted from the first detection component. The first detection component is the detection component corresponding to the first image set in the first object recognition model.

[0092] Based on the second detection component, a second object recognition model is generated for identifying the plurality of first target categories.

[0093] In an exemplary embodiment, each of the detection components includes a fully connected network layer for outputting object recognition results. Each row of parameters in the parameter matrix corresponding to the fully connected network layer is used to output the object recognition result of the category corresponding to the category number indicated by its row number. The model generation module 902 is used to perform the following operations:

[0094] For each number in the first sequence number list, in the fully connected network layer of the first detection component, extract the row parameters indicated in the corresponding parameter matrix for the row number that is the same as the sequence number;

[0095] A sub-fully connected network layer is generated based on the extracted row parameters, and the sub-fully connected network layer is used to constitute the second detection component.

[0096] In one exemplary implementation, obtaining the plurality of first target categories to be identified includes:

[0097] Obtain the target category set;

[0098] The categories in the target category set are matched based on the image set to obtain the plurality of first target categories corresponding to the first image set.

[0099] In one exemplary embodiment, the model generation module 902 is configured to perform the following operations:

[0100] The categories in the target category set are matched based on the image set to obtain multiple second target categories corresponding to the second image set, and the category space of the second image set includes the multiple second target categories;

[0101] A second sequence number list is generated based on the category number of each second target category in the second image set;

[0102] Based on the second sequence number list, a fourth detection component is extracted from the third detection component, wherein the third detection component is the detection component corresponding to the second image set in the first object recognition model;

[0103] Based on the fourth detection component, a third object recognition model is generated for identifying the plurality of second target categories.

[0104] In one exemplary embodiment, the model acquisition module 901 is configured to perform the following operations:

[0105] For each of the aforementioned image sets, obtain the image set number corresponding to that image set;

[0106] A first preset number of first sample images are sampled from the image set;

[0107] Each first sample image is input into the feature extraction component to obtain the corresponding first sample image features. The first sample image features are then input into the detection component corresponding to the image set number to obtain the first sample recognition result corresponding to the first sample image.

[0108] A preset category is determined, which is a category in the category space of the image set that is not matched by any of the sample recognition results;

[0109] The sampling probability of each preset category is determined based on the frequency of occurrence of each preset category in the image set;

[0110] Based on the sampling probability corresponding to each of the preset categories, a second preset number of second sample images are collected in the image set, and any second sample image includes an object corresponding to any of the preset categories;

[0111] Each second sample image is input into the feature extraction component to obtain the corresponding second sample image features. The second sample image features are then input into the detection component corresponding to the image set number to obtain the second sample recognition result corresponding to the second sample image.

[0112] A first classification loss is obtained based on the difference between the first sample recognition result and the category label corresponding to the first sample image; a second classification loss is obtained based on the difference between the second sample recognition result and the category label corresponding to the second sample image.

[0113] Based on the first classification loss and the second classification loss, the parameters of the detection component corresponding to the image set number are adjusted until the training requirements are met.

[0114] In one exemplary embodiment, the model acquisition module 901 is configured to perform the following operations:

[0115] Determine a first repetition factor for each category in the image set, wherein the first repetition factor is inversely correlated with the frequency of occurrence of the category in the image set;

[0116] Determine the second repetition factor corresponding to each image in the image set, where the second repetition factor is the maximum value of the first repetition factor of the category to which the image is matched;

[0117] Based on the second repetition factor corresponding to each of the images, the first preset number of first sample images are sampled.

[0118] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0119] Please refer to Figure 10 It illustrates a structural block diagram of a computer device according to an exemplary embodiment, used to perform the above-described object recognition model generation method. Specifically:

[0120] Computer device 1000 includes a central processing unit (CPU) 1001, a system memory 1004 including random access memory (RAM) 1002 and read-only memory (ROM) 1003, and a system bus 1005 connecting the system memory 1004 and the CPU 1001. Computer device 1000 also includes a basic input / output system (I / O system) 1006 to facilitate information transfer between various devices within the computer, and a mass storage device 1007 for storing the operating system 1013, application programs 1014, and other program modules 1015.

[0121] The basic input / output system 1006 includes a display 1008 for displaying information and an input device 1009 for user input, such as a mouse or keyboard. Both the display 1008 and the input device 1009 are connected to the central processing unit 1001 via an input / output controller 1010 connected to the system bus 1005. The basic input / output system 1006 may also include the input / output controller 1010 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 1010 also provides output to a display screen, printer, or other types of output devices.

[0122] Mass storage device 1007 is connected to central processing unit 1001 via a mass storage controller (not shown) connected to system bus 1005. Mass storage device 1007 and its associated computer-readable media provide non-volatile storage for computer device 1000. That is, mass storage device 1007 may include computer-readable media (not shown) such as hard disk or CD-ROM (Compact Disc Read-Only Memory) drive.

[0123] Without loss of generality, computer-readable media can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes RAM, ROM, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), flash memory or other solid-state storage technologies, CD-ROM, DVD (Digital Video Disc) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that computer storage media are not limited to the above-mentioned types. The system memory 1004 and mass storage device 1007 described above can be collectively referred to as memory.

[0124] According to various embodiments of this application, the computer device 1000 can also be connected to a remote computer on a network, such as the Internet. That is, the computer device 1000 can be connected to the network 1012 via the network interface unit 1011 connected to the system bus 1005, or the network interface unit 1011 can be used to connect to other types of networks or remote computer systems (not shown).

[0125] The aforementioned memory also includes a computer program stored in the memory and configured to be executed by one or more processors to implement the aforementioned object recognition model generation method.

[0126] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is executed by a processor to implement the object recognition model generation method.

[0127] Specifically, the object recognition model generation method includes:

[0128] A first object recognition model is obtained. The first object recognition model includes a feature extraction component and multiple detection components connected to the feature extraction component. Different detection components are trained based on different image sets. The category spaces of different image sets are not completely the same. Each detection component is used to output the object recognition result of the corresponding category according to the category number in the corresponding image set.

[0129] Obtain multiple primary target categories to be identified;

[0130] A first image set is determined, wherein the category space of the first image set includes the plurality of first target categories;

[0131] A first sequence number list is generated based on the category number of each first target category in the category space of the first image set;

[0132] Based on the first sequence number list, a second detection component is extracted from the first detection component. The first detection component is the detection component corresponding to the first image set in the first object recognition model.

[0133] Based on the second detection component, a second object recognition model is generated for identifying the plurality of first target categories.

[0134] In an exemplary embodiment, each detection component includes a fully connected network layer for outputting object recognition results. Each row of parameters in the parameter matrix corresponding to the fully connected network layer is used to output the object recognition result of the category corresponding to the category number indicated by its row number. The step of extracting the second detection component based on the first sequence number list includes:

[0135] For each number in the first sequence number list, in the fully connected network layer of the first detection component, extract the row parameters indicated in the corresponding parameter matrix for the row number that is the same as the sequence number;

[0136] A sub-fully connected network layer is generated based on the extracted row parameters, and the sub-fully connected network layer is used to constitute the second detection component.

[0137] In one exemplary implementation, obtaining the plurality of first target categories to be identified includes:

[0138] Obtain the target category set;

[0139] The categories in the target category set are matched based on the image set to obtain the plurality of first target categories corresponding to the first image set.

[0140] In one exemplary embodiment, the method further includes:

[0141] The categories in the target category set are matched based on the image set to obtain multiple second target categories corresponding to the second image set, and the category space of the second image set includes the multiple second target categories;

[0142] A second sequence number list is generated based on the category number of each second target category in the second image set;

[0143] Based on the second sequence number list, a fourth detection component is extracted from the third detection component, wherein the third detection component is the detection component corresponding to the second image set in the first object recognition model;

[0144] Based on the fourth detection component, a third object recognition model is generated for identifying the plurality of second target categories.

[0145] In one exemplary implementation, the first object recognition model is trained using the following method:

[0146] For each of the aforementioned image sets, obtain the image set number corresponding to that image set;

[0147] A first preset number of first sample images are sampled from the image set;

[0148] Each first sample image is input into the feature extraction component to obtain the corresponding first sample image features. The first sample image features are then input into the detection component corresponding to the image set number to obtain the first sample recognition result corresponding to the first sample image.

[0149] A preset category is determined, which is a category in the category space of the image set that is not matched by any of the sample recognition results;

[0150] The sampling probability of each preset category is determined based on the frequency of occurrence of each preset category in the image set;

[0151] Based on the sampling probability corresponding to each of the preset categories, a second preset number of second sample images are collected in the image set, and any second sample image includes an object corresponding to any of the preset categories;

[0152] Each second sample image is input into the feature extraction component to obtain the corresponding second sample image features. The second sample image features are then input into the detection component corresponding to the image set number to obtain the second sample recognition result corresponding to the second sample image.

[0153] A first classification loss is obtained based on the difference between the first sample recognition result and the category label corresponding to the first sample image; a second classification loss is obtained based on the difference between the second sample recognition result and the category label corresponding to the second sample image.

[0154] Based on the first classification loss and the second classification loss, the parameters of the detection component corresponding to the image set number are adjusted until the training requirements are met.

[0155] In an exemplary embodiment, sampling a first preset number of first sample images in the image set includes:

[0156] Determine a first repetition factor for each category in the image set, wherein the first repetition factor is inversely correlated with the frequency of occurrence of the category in the image set;

[0157] Determine the second repetition factor corresponding to each image in the image set, where the second repetition factor is the maximum value of the first repetition factor of the category to which the image is matched;

[0158] Based on the second repetition factor corresponding to each of the images, the first preset number of first sample images are sampled.

[0159] Optionally, the computer-readable storage medium may include: ROM (Read Only Memory), RAM (Random Access Memory), SSD (Solid State Drives), or optical disc, etc. The random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0160] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the object recognition model generation method described above.

[0161] It should be understood that "multiple" as mentioned herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This application does not limit this.

[0162] In addition, in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0163] The above are merely exemplary embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application shall be included within the protection scope of this application.

Claims

1. A method for generating an object recognition model, characterized in that, The method includes: A first object recognition model is obtained. The first object recognition model includes a feature extraction component and multiple detection components connected to the feature extraction component. Different detection components are trained based on different image sets. The category spaces of different image sets are not completely the same. Each detection component is used to output the object recognition result of the corresponding category according to the category number in the corresponding image set. Obtain multiple primary target categories to be identified; A first image set is determined, wherein the category space of the first image set includes the plurality of first target categories; A first sequence number list is generated based on the category number of each first target category in the category space of the first image set; Based on the first sequence number list, a second detection component is extracted from the first detection component. The first detection component is the detection component corresponding to the first image set in the first object recognition model. Based on the second detection component, a second object recognition model is generated for identifying the plurality of first target categories.

2. The method according to claim 1, characterized in that, Each of the detection components includes a fully connected network layer for outputting object recognition results. Each row of the parameter matrix corresponding to the fully connected network layer is used to output the object recognition result of the category corresponding to the category number indicated by its row number. The step of extracting the second detection component based on the first sequence number list includes: For each number in the first sequence number list, in the fully connected network layer of the first detection component, extract the row parameters indicated in the corresponding parameter matrix for the row number that is the same as the sequence number; A sub-fully connected network layer is generated based on the extracted row parameters, and the sub-fully connected network layer is used to constitute the second detection component.

3. The method according to claim 1 or 2, characterized in that, The acquisition of multiple first target categories to be identified includes: Obtain the target category set; The categories in the target category set are matched based on the image set to obtain the plurality of first target categories corresponding to the first image set.

4. The method according to claim 3, characterized in that, The method further includes: The categories in the target category set are matched based on the image set to obtain multiple second target categories corresponding to the second image set, and the category space of the second image set includes the multiple second target categories; A second sequence number list is generated based on the category number of each second target category in the second image set; Based on the second sequence number list, a fourth detection component is extracted from the third detection component, wherein the third detection component is the detection component corresponding to the second image set in the first object recognition model; Based on the fourth detection component, a third object recognition model is generated for identifying the plurality of second target categories.

5. The method according to claim 1, characterized in that, The first object recognition model is trained using the following method: For each of the aforementioned image sets, obtain the image set number corresponding to that image set; A first preset number of first sample images are sampled from the image set; Each first sample image is input into the feature extraction component to obtain the corresponding first sample image features. The first sample image features are then input into the detection component corresponding to the image set number to obtain the first sample recognition result corresponding to the first sample image. A preset category is determined, which is a category in the category space of the image set that is not matched by any of the sample recognition results; The sampling probability of each preset category is determined based on the frequency of occurrence of each preset category in the image set; Based on the sampling probability corresponding to each of the preset categories, a second preset number of second sample images are collected in the image set, and any second sample image includes an object corresponding to any of the preset categories; Each second sample image is input into the feature extraction component to obtain the corresponding second sample image features. The second sample image features are then input into the detection component corresponding to the image set number to obtain the second sample recognition result corresponding to the second sample image. Based on the difference between the first sample recognition result and the category label corresponding to the first sample image, a first classification loss is obtained; Based on the difference between the second sample recognition result and the category label corresponding to the second sample image, a second classification loss is obtained; Based on the first classification loss and the second classification loss, the parameters of the detection component corresponding to the image set number are adjusted until the training requirements are met.

6. The method according to claim 5, characterized in that, The step of sampling a first preset number of first sample images in the image set includes: Determine a first repetition factor for each category in the image set, wherein the first repetition factor is inversely correlated with the frequency of occurrence of the category in the image set; Determine the second repetition factor corresponding to each image in the image set, where the second repetition factor is the maximum value of the first repetition factor of the category to which the image is matched; Based on the second repetition factor corresponding to each of the images, the first preset number of first sample images are sampled.

7. An object recognition model generation device, characterized in that, The device includes: The model acquisition module is used to acquire a first object recognition model. The first object recognition model includes a feature extraction component and multiple detection components connected to the feature extraction component. Different detection components are trained based on different image sets. The category spaces of different image sets are not completely the same. Each detection component is used to output the object recognition result of the corresponding category according to the category number in the corresponding image set. The model generation module is used to perform the following operations: Obtain multiple primary target categories to be identified; A first image set is determined, wherein the category space of the first image set includes the plurality of first target categories; A first sequence number list is generated based on the category number of each first target category in the category space of the first image set; Based on the first sequence number list, a second detection component is extracted from the first detection component. The first detection component is the detection component corresponding to the first image set in the first object recognition model. Based on the second detection component, a second object recognition model is generated for identifying the plurality of first target categories.

8. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one instruction, at least one program, a code set, or an instruction set, the at least one instruction, the at least one program, the code set, or the instruction set being loaded and executed by the processor to implement the object recognition model generation method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or instruction set is loaded and executed by a processor to implement the object recognition model generation method as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, The computer program product includes computer instructions, the processor of the computer device reads the computer instructions, and the processor of the computer device executes the computer instructions to implement the object recognition model generation method as described in any one of claims 1 to 7.

Images