Object category recognition model training method and apparatus, and object category recognition method and apparatus

Abstract

The present application discloses an object category recognition model training method and apparatus, and an object category recognition method and apparatus, which can be applied to various scenarios such as cloud technology, artificial intelligence, intelligent transportation, and the Internet of vehicles. The method comprises: acquiring a newly added sample image of a sample object; on the basis of an initial object category recognition model, constructing a teacher model and a student model; inputting the newly added sample image into the teacher model for performing first object category recognition, and obtaining a first sample category; performing fusion on the first sample category and a newly added object category, and obtaining a fused category; inputting the newly added sample image into the student model for performing second object category recognition, and obtaining a second sample category; and, on the basis of a difference between the second sample category and the fused category, training the student model, so as to obtain a final object category recognition model. According to the model training method of the present application, the training duration is greatly shortened, and the model updating efficiency is improved.

Description

Object category recognition model training method, object category recognition method and device

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on February 9, 2024, with application number 202410178190.X and application name “Training method for object category recognition model, object category recognition method and device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to a training method for an object category recognition model, an object category recognition method, and a device. Background Art

[0003] In the delivery of AI-based (Artificial Intelligence) services, the amount of data collected for training deep neural network models increases as the project progresses. This is especially true in the field of visual perception (tasks such as image or video recognition, segmentation, and detection), where data volumes can easily accumulate to significant levels. During model iteration, this accumulating data increases training time and the number of training cards required (increasing business costs). It also makes it difficult to quickly respond to and resolve poorly performing application scenarios (bad cases), reducing business efficiency. Furthermore, due to confidentiality and business interests, it is common for historical data to be unavailable or unusable.

[0004] Related technologies use full data for model iteration and incremental data for model fine-tuning. When using full data for model iteration, if the data volume is large, the number of training cards and the training time are relatively high. Moreover, since the historical stock data is generally much larger than the incremental data, the model does not respond well to newly added bad cases. When using incremental data for iteration, as the training time lengthens or the number of fine-tuning increases, the model will gradually shift, resulting in catastrophic forgetting of knowledge about historical data, causing the overall model capability to decline rapidly and become unstable. Incremental learning methods are mainly aimed at quasi-incremental scenarios. These methods pay more attention to the anti-forgetting effect during the model update process. There is no instance incremental method that can achieve knowledge accumulation. Transfer learning only focuses on performance on new data, and the changes in the domain it focuses on are usually large, such as the transition from cartoon images to natural scenes. Summary of the Invention

[0005] The present application provides a training method, an object category recognition method and a device for an object category recognition model, which can train the initial object category recognition model based on newly added sample data when historical data cannot be obtained, thereby improving the training efficiency of the model while ensuring the improvement of the accuracy of the object category recognition model.

[0006] In one aspect, the present application provides a method for training an object category recognition model, the method comprising:

[0007] Acquire a new sample image of the sample object; the new sample image is annotated with a new object category, and the new object category is used to represent the category of the sample object;

[0008] Based on the initial object category recognition model, a teacher model and a student model are constructed; the teacher model and the student model are respectively obtained by copying the initial object category recognition model;

[0009] Inputting the newly added sample image into the teacher model to perform first object category recognition processing to obtain a first sample category, where the first sample category and the newly added object category are the same type of data;

[0010] Performing a fusion process on the first sample category and the newly added object category to obtain a fusion category;

[0011] Inputting the newly added sample image into the student model to perform second object category recognition processing to obtain a second sample category;

[0012] Based on the difference between the second sample category and the fusion category, the student model is trained to obtain a final object category recognition model.

[0013] Another aspect provides a method for identifying an object category, the method comprising:

[0014] Obtain an image of an object to be identified;

[0015] Inputting the image to be identified into an object category recognition model to perform object category recognition processing to obtain a target object category of the object to be identified;

[0016] The object category recognition model is trained based on the above training method.

[0017] Another aspect provides a training device for an object category recognition model, the device comprising:

[0018] A new sample image acquisition module is used to acquire a new sample image of a sample object; the new sample image is annotated with a new object category, and the new object category is used to represent the category of the sample object;

[0019] A student model construction module is used to construct a teacher model and a student model based on the initial object category recognition model; the teacher model and the student model are respectively obtained by copying the initial object category recognition model;

[0020] A first sample result determination module is configured to input the newly added sample image into the teacher model to perform first object category recognition processing to obtain a first sample category, where the first sample category and the newly added object category are the same type of data;

[0021] a sample category fusion module, configured to fuse the first sample category and the newly added object category to obtain a fused category;

[0022] A second sample result determination module is configured to input the newly added sample image into the student model to perform second object category recognition processing to obtain a second sample category;

[0023] The student model training module is used to train the student model based on the difference between the second sample category and the fusion category to obtain a final object category recognition model.

[0024] Another aspect provides an object category recognition device, the device comprising:

[0025] An image acquisition module for obtaining an image of an object to be identified, used for obtaining an image of an object to be identified;

[0026] a target object category determination module, configured to input the image to be identified into an object category recognition model for object category recognition processing to obtain a target object category of the object to be identified;

[0027] The object category recognition model is trained based on the above training method.

[0028] On the other hand, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the object category recognition model training method or object category recognition method as described above.

[0029] On the other hand, a computer storage medium is provided, which stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by a processor to implement the object category recognition model training method or object category recognition method as described above.

[0030] Another aspect provides a computer program product or computer program, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to implement the method for training an object category recognition model or the method for object category recognition described above.

[0031] The object category recognition model training method, object category recognition method, and device provided in this application have the following technical effects:

[0032] The present application discloses a training method for an object category recognition model, comprising: obtaining a new sample image of a sample object, wherein the new sample image is annotated with a new object category; constructing a teacher model and a student model based on an initial object category recognition model; inputting the new sample image into the teacher model for a first object category recognition process to obtain a first sample category, and fusing the first sample category and the new object category to obtain a fusion category, so that the fusion category can contain the decision boundary information of the existing model (i.e., the category indicated by the new object category), and can also correct the errors that occur in the object category recognition process of the teacher model (i.e., the errors contained in the first sample category). In this way, the information content of the fusion category can be enriched, so that when the new sample image is input into the student model for a second object category recognition process to obtain the second sample category, when the student model is trained based on the difference between the second sample category and the fusion category to obtain the final object category recognition model, the student model can accurately learn the decision boundary information and avoid the errors that occur in the teacher model, thereby improving the learning effect of the learning model and thus improving the accuracy of the final object category recognition model. Moreover, through this method, it is possible to achieve that in the process of training the initial object category recognition model, there is no need to obtain the initial sample images, and the initial object category recognition model can be trained using only the newly added sample images. Through category fusion processing, the historical knowledge can be forgotten while learning new data to improve the model performance, and a knowledge accumulation effect similar to that of the full data model can be achieved. Compared with the training process of the full data, the model training method of this application greatly shortens the training time and improves the model update efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] FIG1 is a schematic diagram of a training system for an object category recognition model provided by an embodiment of this specification;

[0034] FIG2 is a flow chart of a method for training an object category recognition model according to an embodiment of this specification;

[0035] FIG3 is a flow chart of a method for training an initial object category recognition model provided in an embodiment of this specification;

[0036] FIG4 is a flow chart of a method for inputting sample processing data into a teacher model and a student model respectively to obtain first loss information, provided by an embodiment of this specification;

[0037] FIG5 is a flow chart of a method for training a student model based on the difference between the second sample category and the fusion category provided in an embodiment of this specification;

[0038] FIG6 is a flow chart of a method for obtaining an object category recognition model based on target loss information provided in an embodiment of this specification;

[0039] FIG7 is a flow chart of a method for updating a teacher model based on updating model parameters of a student model to obtain an object category recognition model, provided by an embodiment of this specification;

[0040] FIG8 is a flowchart of a method for adjusting the model parameters of a student model based on target loss information until a training end condition is met, and using the student model at the end of training as an updater of the student model, provided by an embodiment of this specification;

[0041] FIG9 is a flow chart of a method for iteratively updating a student model and a teacher model to obtain an object category recognition model, provided by an embodiment of this specification;

[0042] FIG10 is a performance test curve obtained by updating and training multiple models based on multiple batches of new data provided in an embodiment of this specification;

[0043] FIG11 is a comparison diagram of the model training framework of this embodiment and the prior art;

[0044] FIG12 is a flow chart of an object category recognition method provided in an embodiment of this specification;

[0045] FIG13 is a test curve diagram of a performance test of a model of the prior art and a model trained by the present solution using the Cifar-100 dataset provided in an embodiment of this specification;

[0046] FIG14 is a test curve diagram of a performance test of a model of the prior art and a model trained by the present solution using the imagenet-100 dataset provided in an embodiment of this specification;

[0047] FIG15 is a schematic diagram of the structure of a training device for an object category recognition model provided in an embodiment of this specification;

[0048] FIG16 is a schematic diagram of the structure of an object category recognition device provided in an embodiment of this specification;

[0049] FIG17 is a schematic diagram of the structure of a server provided in an embodiment of this specification. DETAILED DESCRIPTION

[0050] First, some nouns or terms that appear in the description of the embodiments of this specification are explained as follows:

[0051] 1. Full data model: refers to the deep student model obtained by iterative model training using all accumulated data.

[0052] 2. Incremental data: refers to newly added data, usually with annotations, and its data volume is usually much smaller than the full data.

[0053] 3. Badcase data: The existing deep learning model cannot handle scene data or difficult data well. For example, for the apple recognition model, those apple images that cannot be recognized are badcases.

[0054] 4. Epoch: The process in which the deep student model completes one training session using all the training data.

[0055] 5. Batch: Due to the limited memory capacity of CPUs (Central Processing Units) or GPUs (Graphics Processing Units), it is usually impossible to train all the data at once. Instead, the data is divided into multiple groups and trained sequentially. This group of data is called a batch, and the amount of data in each batch is called the batch size.

[0056] 6. Machine Learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0057] 7. Deep Learning: The concept of deep learning originates from the study of artificial neural networks. A multilayer perceptron with multiple hidden layers is an example of a deep learning architecture. Deep learning combines lower-level features to form more abstract higher-level representations of attribute categories or features, thereby discovering distributed feature representations of data. The method of this embodiment uses deep learning training to obtain an object category recognition model.

[0058] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0059] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0060] Please refer to FIG. 1 , which is a schematic diagram of a training system for an object category recognition model provided in an embodiment of this specification. As shown in FIG. 1 , the training system for the object category recognition model may include at least a server 01 and a client 02 .

[0061] Specifically, in the embodiments of this specification, the server 01 may include an independently operated server, or a distributed server, or a server cluster composed of multiple servers. It may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Server 01 may include a network communication unit, a processor, a memory, and the like. Specifically, the server 01 can be used to train the student model to obtain an object category recognition model.

[0062] Specifically, in the embodiments of this specification, the client 02 may include a physical device such as a smartphone, desktop computer, tablet computer, laptop computer, digital assistant, smart wearable device, smart speaker, in-vehicle terminal, smart TV, etc. It may also include software running on the physical device, such as a web page provided by a service provider to a user, or an application provided by the service provider to a user. Specifically, the client 02 may be used to online query the target object category of an image of an object to be identified.

[0063] The following introduces a training method for an object category recognition model of the present application. Figure 2 is a flow chart of a training method for an object category recognition model provided in an embodiment of this specification. This specification provides method operation steps as described in the embodiment or flow chart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the order of many steps and does not represent the only execution order. When the actual system or server product is executed, it can be executed in sequence or in parallel according to the method shown in the embodiment or the accompanying drawings (for example, a parallel processor or a multi-threaded processing environment). Specifically, as shown in Figure 2, the method may include:

[0064] S201: Acquire a new sample image of a sample object, where the new sample image is annotated with a new object category.

[0065] In the embodiments of this specification, a sample image can be an image obtained by photographing a sample object, or an image extracted from a sample video corresponding to the sample object. A sample object can correspond to one or more sample images; sample objects can include, but are not limited to, physical or virtual objects in nature. Sample objects vary in different application scenarios; for example, a sample object can be an animal or plant in nature, and this embodiment can classify natural animals or plants; a sample object can also be an electronic product, and can classify electronic products, etc. New object categories can be used to characterize the categories of sample objects in new sample images; for example, if the sample object is a cat, the new object category can be Scottish Fold cat, etc.

[0066] Exemplarily, the newly added sample images refer to the sample images added during the training of the initial object category recognition model, and the newly added sample images may be one or more; the newly added object categories essentially refer to the object category labels of the newly added sample images, and the object category labels may be one or more; the newly added sample images may be labeled by image recognition methods, and the newly added object categories of the newly added sample images may also be manually labeled.

[0067] S203: Construct a teacher model and a student model based on the initial object category recognition model; the initial object category recognition model is obtained by using the initial sample image of the sample object for object category recognition training. Specifically, the initial object category recognition model is obtained by inputting the initial sample image of the sample object into a preset model for object category recognition training; the initial sample image is marked with an initial sample object category, and the initial sample object category is different from the newly added object category.

[0068] In the embodiments of the present specification, the newly added object category represents the newly added object category of the newly added sample image relative to the initial sample image; the initial sample image of the sample object can be input into a preset model for object category recognition training to obtain an initial object category recognition model; when there are newly added sample images, the model can be updated based on the initial object category recognition model, thereby improving the diversity of object categories recognized by the updated model. The initial sample object category is different from the newly added object category; for example, when the sample object is a cat, the newly added object category can be a Scottish Fold cat, and the initial sample object category can include a tabby cat, a Garfield cat, a Persian cat, etc. The initial object category recognition model can only recognize tabby cats, Garfield cats, and Persian cats; then the updated object category recognition model can not only recognize tabby cats, Garfield cats, and Persian cats, but also recognize Scottish Fold cats corresponding to the newly added object category.

[0069] For example, the initial object category recognition model can be copied to obtain a teacher model and a student model; the initial teacher model and student model are exactly the same model and are both initial object category recognition models. Large models are often single complex networks or a collection of several networks, with good performance and generalization capabilities, while small models have limited expressive power due to their smaller network size. Therefore, the knowledge learned by the large model can be used to guide the training of the small model, so that the small model has comparable performance to the large model, but with a significantly reduced number of parameters, thereby achieving model compression and acceleration. This is the application of knowledge distillation and transfer learning in model optimization. Knowledge distillation adopts a teacher-student model: a complex and large model is used as the teacher (teacher model), and the student (student model) has a relatively simple structure. The teacher is used to assist the training of the student model. The teacher has strong learning ability and can transfer its learned knowledge to the student model with relatively weak learning ability, thereby enhancing the generalization ability of the student model.

[0070] In the embodiment of this specification, as shown in FIG3 , the method for training and obtaining an initial object category recognition model includes:

[0071] S301: Acquire an initial sample image of a sample object;

[0072] S303: Inputting the initial sample image into a preset model for initial object category recognition processing to obtain an initial sample category; the initial object category recognition processing here refers to object category recognition processing performed by the preset model, and the initial sample category refers to the category of the sample object recognized by the preset model based on the initial sample image;

[0073] S305: Determine sample loss information based on the difference between the initial sample category and the initial sample object category;

[0074] S307: Adjusting model parameters of the preset model based on the sample loss information until a training end condition is met, and determining the preset model at the end of training as the initial object category recognition model.

[0075] In an embodiment of the present specification, the initial sample image and the newly added sample image are different images corresponding to the same sample object, and the sample objects included belong to different categories. The preset model can be a classification model or classification network, for example, it can include but is not limited to: AlexNet (an image classification model based on deep learning, which can be composed of 5 convolutional layers and 3 fully connected layers), VGG network (a network using multiple consecutive 3×3 convolution kernels, mainly composed of repeated convolutional layers and pooling layers stacked), ResNet (a neural network model containing a residual module), etc. The initial sample image can be input into the preset model for image feature extraction to obtain the sample initial image features, and then the sample initial image features are processed for initial object category recognition to obtain the initial sample category; then, based on the difference between the initial sample category and the initial sample object category, sample loss information is calculated; and the model parameters of the preset model are adjusted according to the sample loss information until the training end condition is met. Specifically, the model parameters of the preset model can be adjusted in the direction of reducing the sample loss information. The training end condition may include but is not limited to the sample loss information being less than a preset loss threshold, the sample loss information being less than a preset loss threshold, and the number of model training iterations reaching the target number. Finally, the preset model at the end of training is determined as the initial object category recognition model.

[0076] In an embodiment of the present specification, a preset model can be supervised trained based on the initial sample image and the initial sample object category annotated therewith, so as to quickly and accurately obtain an initial object category recognition model, so that when there is a new sample image, the model can be updated based on the initial object category recognition model, thereby improving the efficiency of the model update. It is understandable that the initial sample object category annotated by the initial sample image and the aforementioned new object category belong to the same type of data, and the training performed on the preset model is supervised training. During the supervised training process, the preset model can optimize its own parameters based on the difference between the initial sample category it identifies and the annotated initial sample object category, so that the initial sample category it subsequently identifies and the initial sample object category are the same type of data, and the identified initial sample category is the same or similar to the initial sample object category, thereby making the initial object category recognition model and the teacher model obtained based on the preset model able to identify the output object category during subsequent object category processing, and the object category it identifies and the annotated object category belong to the same type of data.

[0077] S205: Input the newly added sample image into the teacher model to perform first object category recognition processing to obtain a first sample category.

[0078] In an embodiment of the present specification, the newly added sample image can be input into the teacher model, the image features of the sample image can be extracted based on the teacher model, and the first object category recognition processing can be performed based on the extracted image features to obtain the first sample category; the first sample category and the newly added object category are the same type of data, and the same type here refers to the same data format, such as the first sample category and the newly added object category are both text format data, or both are floating point format data, etc.

[0079] In an embodiment of the present specification, after constructing a teacher model and a student model based on the initial object category recognition model, newly added sample images can be input into the teacher model for first object category recognition processing to obtain a first sample category; thereby, the student model is trained according to the output result of the teacher model (i.e., the first sample category).

[0080] S207: Perform fusion processing on the first sample category and the newly added object category to obtain a fused category.

[0081] In the embodiment of this specification, the first sample category and the newly added object category may be fused, and the fusion process is shown in the following formula (1):

[0082] Among them, the newly added object categories include one-hot annotation labels of K categories, where K is a positive integer; kis the one-hot annotation label corresponding to the kth category, k∈[1,K]; the first sample category includes the one-hot annotation labels of J categories, y j is the one-hot annotation label corresponding to the j-th category, j∈[1,J]; The teacher model predicts the category of the sample object in the newly added sample image as the score of the kth category, The teacher model predicts the category of the sample object in the newly added sample image as the score of the jth category, and τ is the smoothing temperature. The fused label It will contain the decision boundary information of the existing model while correcting the errors of the teacher model.

[0083] S209: Input the newly added sample image into the student model to perform second object category recognition processing to obtain a second sample category.

[0084] In an embodiment of the present specification, a new sample image may be input into the student model, image features of the new sample image may be extracted based on the student model, and a second object category recognition process may be performed based on the extracted image features to obtain a second sample category.

[0085] S2011: Based on the difference between the second sample category and the fusion category, the student model is trained to obtain a final object category recognition model.

[0086] In an embodiment of the present specification, target loss information can be constructed based on the difference between the second sample category and the fusion category, and the student model can be trained using the target loss information, so that the trained student model can be used as the final object category recognition model, or the final object category recognition model can be further obtained based on the trained model.

[0087] In an exemplary embodiment, after acquiring the newly added sample image of the sample object, the method further includes:

[0088] Performing data pollution processing on the newly added sample images to obtain sample processing data, wherein the pollution processing is used to change the distribution of the newly added sample images;

[0089] The sample processing data is input into the teacher model and the student model respectively to obtain first loss information.

[0090] In the embodiment of this specification, during the training process of the student model, considering that historical data cannot be obtained and new data is relatively scarce, the boundary information retained by formula (1) (such as the category indicated by the newly added object category) is limited. This embodiment further proposes a method of polluting the input space to perform more decision boundary information distillation. Data pollution processing is used to expand the sample data. The data pollution processing may include but is not limited to adding Gaussian noise, other noise or data transformation processing to the newly added sample images to obtain sample processing data; the sample processing data does not have a label, and the first loss information is determined through unsupervised training.

[0091] For example, strong Gaussian noise is randomly added to newly added sample images to redistribute the newly added sample images in the model's decision space, distributing as many samples as possible around the decision area. This allows knowledge distillation to be performed on the contaminated data, achieving a complete boundary distillation effect. Gaussian noise refers to a type of noise whose probability density function follows a Gaussian distribution Q (i.e., a normal distribution). Common Gaussian noise types include fluctuation noise, cosmic noise, thermal noise, and shot noise. In addition to commonly used noise suppression methods, mathematical statistics are often used to suppress Gaussian noise. If a noise has an amplitude distribution that follows a Gaussian distribution and a uniform power spectral density, it is called Gaussian white noise. The second-order moments of Gaussian white noise are uncorrelated, while the first-order moments are constant, which refers to the temporal correlation between successive signals. Gaussian noise includes thermal noise and shot noise. In communication channel testing and modeling, Gaussian noise is used as additive white noise to generate additive white Gaussian noise. The input space is polluted in the following way: X′ is the sample image after the pollution, norm(X) represents the normalization operation of the sample image, and G(μ,δ) is the Gaussian noise with mean μ and variance δ. X′=norm(X)+G(μ,δ) (2)

[0092] In the embodiment of this specification, as shown in FIG4 , the sample processing data is input into the teacher model and the student model respectively to obtain the first loss information, including:

[0093] S401: Inputting the sample processed data into the teacher model to perform first object category recognition processing to obtain a third sample category;

[0094] S403: Inputting the sample processed data into the student model to perform second object category recognition processing to obtain a fourth sample category;

[0095] S405: Determine first loss information based on the difference between the fourth sample category and the third sample category.

[0096] In the embodiments of this specification, sample processing data can be input into the teacher model and the student model respectively to obtain a third sample category and a fourth sample category; unsupervised training is then performed based on the third sample category and the fourth sample category, and the first loss information is determined based on the difference between the fourth sample category and the third sample category. This method allows the first loss information to be determined even when the sample processing data does not have labels, thereby reducing the number of labels required during model training and thus saving processing resources (such as storage space) occupied by the labels.

[0097] Exemplarily, as shown in FIG5 , the training of the student model based on the difference between the second sample category and the fusion category includes:

[0098] S20111: determining second loss information based on a difference between the second sample category and the fusion category;

[0099] In the embodiment of the present specification, supervised training may be performed based on the second sample category and the fusion category, and the second loss information may be obtained by calculating the difference between the second sample category and the fusion category.

[0100] S20113: Determine target loss information based on the first loss information and the second loss information;

[0101] In the embodiments of this specification, the sum of the first loss information and the second loss information can be calculated to obtain the target loss information; alternatively, the weights corresponding to the first loss information and the second loss information can be set separately, and the weighted sum of the first loss information and the second loss information can be calculated to obtain the target loss information.

[0102] S20115: Adjust the model parameters of the student model based on the target loss information.

[0103] In the embodiment of this specification, the entire decision boundary distillation process, that is, the loss function corresponding to the target loss information is shown in formula (3). On the newly added sample images, the boundary distillation and new knowledge learning effects are achieved simultaneously by using the fusion label. On the contaminated data X', the boundary distillation is mainly performed. In this way, when the model parameters of the student model are adjusted based on the target loss information, the student model can learn more knowledge, thereby improving the accuracy of the model parameter adjustment of the student model. Formula (3) is used to train the student model, N is the number of newly added sample images, y f (x)log p s (x) is the second loss function corresponding to the second loss information, p t (x′)log p s(x′) is the first loss function corresponding to the first loss information, and λ is the weight of the first loss function.

[0104] In some embodiments, the model parameters of the student model can be adjusted based on the target loss information until the training end conditions are met, and the student model at the end of training can be used as the final object category recognition model; the model parameters of the student model at the end of training can also be used to perform knowledge precipitation (KC) on the teacher model, and the final teacher model can be used as the final object category recognition model.

[0105] In the embodiment of this specification, as shown in FIG6 , the method further includes:

[0106] S601: Based on the target loss information, adjust the model parameters of the student model until the training end condition is met, and use the student model at the end of training as the updated student model;

[0107] S603: Update the teacher model based on the updated model parameters of the student model to obtain a final object category recognition model.

[0108] In an embodiment of the present specification, the model parameters of the student model can be adjusted according to the target loss information until the training end condition is met. The training end condition can include that the target loss information is less than a preset threshold or the target loss information is less than a preset threshold and the number of iterations reaches a preset number; then the student model at the end of training is used as the updated student model.

[0109] In the embodiment of this specification, the updating of the teacher model based on the model parameters of the updated student model to obtain the final object category recognition model includes:

[0110] Updating the teacher model based on the model parameters of the updated student model to obtain an updated teacher model;

[0111] The updated student model and the updated teacher model are iteratively updated to obtain a final object category recognition model.

[0112] In the embodiment of this specification, as shown in FIG7 , the updating of the teacher model based on the model parameters of the updated student model to obtain the final object category recognition model includes:

[0113] S6031: Get and update the current model parameters of the student model;

[0114] S6033: Process the current model parameters based on the exponential moving average algorithm to obtain the target model parameters;

[0115] S6035: Update the teacher model based on the target model parameters to obtain the final object category recognition model.

[0116] In the embodiments of this specification, the exponential moving average (EMA), also known as the weighted moving average, is an averaging method that gives more weight to recent data. The exponential moving average algorithm can be used to process the current model parameters of the updated student model to obtain the target model parameters. The target model parameters are then used to replace the model parameters in the teacher model to update the teacher model and obtain the final object category recognition model.

[0117] In some embodiments, the method further comprises:

[0118] After the number of training times reaches the first target number of times, the teacher model is updated based on the model parameters of the student model to obtain the final object category recognition model, and the number of training times refers to the number of times the student model is trained using the newly added sample images. The process of training the student model once using the newly added sample images can refer to the relevant description of the aforementioned steps S20111-S20115. Each time the student model is trained once, the model parameters of the student model are adjusted once. It can be seen that in this way, after the student model is fully trained and learns new knowledge based on the first target number of training times, the teacher model is updated based on the model parameters of the student model. This not only improves the updating accuracy of the teacher model, but also avoids the frequent updating of the teacher model, so that the teacher model can better precipitate knowledge, thereby improving the performance of the teacher model and saving the processing resources consumed by the frequent updating of the teacher model.

[0119] Exemplarily, after the number of training times reaches the first target number of times, the teacher model is updated based on the model parameters of the student model to obtain a final object category recognition model, including:

[0120] After the number of training times reaches a first target number, updating the teacher model based on the model parameters of the student model to obtain an updated teacher model;

[0121] The updated teacher model is used as the final object category recognition model, or during the training process after the training number of times reaches the first target number of times, the updated teacher model is iteratively updated based on the model parameters of the student model to obtain the final object category recognition model. The iterative update refers to: updating the updated teacher model multiple times, and each update is performed on the updated teacher model obtained by the previous update.

[0122] In an embodiment of the present specification, after the number of training times reaches a first target number of times, the teacher model can be updated according to the model parameters of the student model to obtain an updated teacher model; and during the training process after the number of training times reaches the first target number of times, the updated teacher model can be iteratively updated according to the model parameters of the student model, so that the final object category recognition model can be quickly trained, and the model performance of the final object category recognition model can be improved by iteratively updating the updated teacher model.

[0123] In some embodiments, during the training process after the number of training times reaches the first target number of times, iteratively updating the updated teacher model based on the model parameters of the student model to obtain a final object category recognition model includes:

[0124] After the number of training times reaches the first target number of times, each time a second target number of training times passes, the final updated teacher model is updated based on the model parameters of the student model;

[0125] After the training is completed, the updated teacher model is determined as the final object category recognition model.

[0126] It can be seen that the above method can not only realize the continuous alternation of updating the model parameters of the learning model and the teacher model during the training process, thereby improving the accuracy of the object category recognition model, but also realize the update frequency of the teacher model based on the second target number to constrain the update frequency of the teacher model. In this way, it can be avoided that after the student model is trained for the first target number of times, the teacher model is updated every time the student model is trained. In this way, the updated teacher model can better accumulate knowledge within an update cycle (that is, the cycle corresponding to the second target number of times), thereby improving the performance of the updated teacher model and saving processing resources consumed by frequent updates of the teacher model.

[0127] In the embodiment of this specification, the updating of the teacher model based on the model parameters of the student model after each second target number of trainings includes:

[0128] After each second target number of training times, obtain the current model parameters of the student model;

[0129] Processing the current model parameters based on an exponential moving average algorithm to obtain target model parameters;

[0130] The updated teacher model is updated based on the target model parameters.

[0131] In the embodiments of this specification, the first target number can be set according to actual conditions, for example, the first target number can be set to 10 epochs; the second target number can be set to be less than the first target number, for example, the second target number can be set to 5 epochs; after the number of training times reaches the first target number, the updated teacher model is updated based on the model parameters of the student model every time the second target number of training times passes; thereby, the final object category recognition model with higher accuracy can be quickly trained.

[0132] Since the exponential moving average is an averaging method that gives higher weight to recent data, the above method processes the current model parameters based on this method, which can make the processing process pay more attention to recent data with higher weight, thereby obtaining target model parameters that are more consistent with recent data. This can improve the realism of the target model parameters, and then improve the model performance of the final object category recognition model obtained by updating the updated teacher model based on the target model parameters.

[0133] In some embodiments, the processing of the current model parameters based on the exponential moving average algorithm to obtain the target model parameters includes:

[0134] Determining a first weight of the teacher model and a second weight of the student model based on the number of training times corresponding to the current model parameters; wherein the first weight decreases as the number of training times increases, and the second weight increases as the number of training times increases;

[0135] The current model parameters are processed based on the first weight, the second weight, and the exponential moving average algorithm to obtain the target model parameters.

[0136] In the embodiments of this specification, the ordinary model EMA operation does not have the ability to precipitate knowledge. It performs exponential averaging of model parameters at each step of model iteration, resulting in rapid homogenization of the teacher model and the student model, thereby failing to distill the decision boundary, causing the model to be severely forgotten and overfit to new data. Therefore, this embodiment proposes a dynamic knowledge precipitation KC-EMA operation suitable for knowledge precipitation. It sets a freezing window that only trains the student model to ensure that the EMA of the model parameters only occurs after the student model is fully trained and learns new knowledge.

[0137] For example, the time window is set to 10 epochs, where an epoch is the process of a deep learning model completing a training cycle using all training data. That is, KC-EMA begins after the student model has trained for 10 epochs. The execution process is to execute KC-EMA once every 5 epochs, and the EMA update momentum (α) is set to a dynamic value that decreases as the number of training epochs increases, as shown in formula (4):

[0138] Among them, α is the first weight, α0 is the initial value of α, e is an epoch, e w refers to the initial epoch. Formula (4) will accelerate the process of knowledge accumulation from the student model to the teacher model as the student model continues to improve.

[0139] The update formula (5) corresponding to EMA is as follows, θ t are the parameters of the teacher model, θ s is the parameter of the learning model; the second weight ɑ acts on the teacher model; during the model update process, α becomes smaller and smaller; 1-α is the second weight, which acts on the student model, so the weight value of the student model becomes larger and larger. θ t ←ɑθ t +(1-ɑ)θ s (5)

[0140] In the method proposed in this scheme, although the student model and the teacher model are both obtained by copying the initial object category recognition model, they are exactly the same models at the beginning. However, based on the above description, it can be seen that the student model S actually plays the role of quickly learning new data, while the teacher model T is a slow learner and a balance stabilizer of new and old knowledge. That is, the model parameter update frequency of the student model and the model parameter update frequency of the teacher model are completely different. With the update of different frequencies, the teacher model and the student model can gradually become two models with different model parameters. Moreover, since the model parameter update frequency of the teacher model is slower than that of the student model, it can better balance the new and old knowledge learned during the model training process, thereby having better model performance.

[0141] Exemplarily, as shown in FIG8 , adjusting the model parameters of the student model based on the target loss information until the training end condition is met, and using the student model at the end of the training as the updated student model includes:

[0142] S6011: Inputting the newly added sample image into the student model for training once to obtain target loss information, and determining the target loss information as initial loss information;

[0143] S6013: Adjust the model parameters of the student model based on the initial loss information until the number of training times reaches the first target number, and use the student model at the end of training as the updated student model.

[0144] In the embodiments of this specification, the first target number can be set according to actual conditions, for example, the first target number can be set to 10 epochs; after one training session, the model parameters of the student model can be adjusted according to the initial loss information until the number of training sessions reaches the first target number, and the student model at the end of the training is used as the updated student model.

[0145] Exemplarily, the updated student model can be trained based on the newly added sample images, and the model parameters of the current student model are obtained every time the number of training times in the training process reaches the second target number; then, based on the model parameters of the current student model, the updated teacher model is updated to obtain the final object category recognition model.

[0146] In the embodiment of this specification, as shown in FIG9 , the iterative updating of the student model and the teacher model to obtain the final object category recognition model includes:

[0147] S901: The updated student model is used as the current student model, and the updated teacher model is used as the current teacher model;

[0148] S903: Training the current student model based on the newly added sample image until the number of training times reaches a second target number of times;

[0149] S905: Re-using the current student model at the end of training as the current student model, and obtaining model parameters of the current student model;

[0150] S907: Based on the model parameters of the current student model, the current teacher model is updated, and the updated current teacher model is used as the current teacher model again;

[0151] S909: Repeatedly training the current student model based on the newly added sample images until the number of training times reaches a second target number, updating the current teacher model based on the model parameters of the current student model, and re-using the updated current teacher model as the current teacher model until the update end condition is met;

[0152] S9011: Determine the current teacher model when the update end condition is met as the final object category recognition model.

[0153] In an embodiment of the present specification, the second target number can be set to be less than the first target number. For example, the second target number can be set to 5 epochs; the updated student model is used as the current student model, and the updated teacher model is used as the current teacher model; the current student model is trained based on the newly added sample image until the number of training reaches the second target number; the current student model at the end of training is used again as the current student model, and the model parameters of the current student model are obtained; based on the model parameters of the current student model, the current teacher model is updated, and the updated current teacher model is used again as the current teacher model; then jump to the step of training the current student model based on the newly added sample image until the number of training reaches the second target number and repeat the subsequent steps, and when the number of training in the training process reaches the second target number, the model parameters of the current student model are obtained; based on the model parameters of the current student model, the updated teacher model is updated to obtain the object category recognition model, so that the model parameters of the learning model and the teacher model are continuously and alternately updated during the training process, thereby improving the accuracy of the final object category recognition model.

[0154] In the embodiments of this specification, some historical data can also be added for training. The practical application of artificial intelligence (AI) is largely limited by the complex update process of the model or the response speed to newly added difficult data. Since deep learning models usually take a long time to train, the operating costs remain high and it is difficult to respond quickly to problems encountered in the business. These problems make it difficult for the deployment of AI systems to gain trust and recognition. Moreover, deep learning models usually need to retain all data for training. In scenarios where data is missing due to factors such as confidentiality or where data storage costs are limited, model updates and upgrades are difficult to complete, further limiting the application of AI models. The instance incremental learning method proposed in this method can greatly speed up the process of iterative updating of the model (reduce costs) while maintaining a knowledge accumulation effect (increase efficiency) similar to that of the full data model. The method of this embodiment can learn new knowledge without forgetting old knowledge in the absence of historical data.

[0155] For example, as shown in FIG10 , FIG10 is a performance test curve obtained by updating and training multiple models based on multiple batches of new data, wherein FIG10 (A) is a performance test curve corresponding to the model obtained by training based on the full amount of data, the model obtained by fine-tuning the model, and the object category recognition model obtained in this embodiment. The test curve represents the corresponding relationship between the task identifier and the accuracy rate. Multiple task identifiers correspond to the same type of task, and different task identifiers correspond to different new data. The curve includes a forgetting area and an improving area. The forgetting area refers to the accuracy rate result area obtained by training the model without using the historical data before the new data, and the improving area refers to the accuracy rate result area obtained by considering the historical data before the new data during the model training process. Curve a is the performance test curve corresponding to the model obtained by training based on the full amount of data, curve b is the performance test curve corresponding to the model obtained by fine-tuning the model, and curve c is the performance test curve corresponding to the object category recognition model obtained in this embodiment. FIG10 (B) is the model flow M and data flow D corresponding to each model in FIG10 (A). In the initial stage, sufficient data D(0) is generally collected to train a version of the initial model M0. As the business progresses, new data of the same type but slightly different in form will continue to appear. For example, on the production line, we will generally focus on mining bad cases to iterate the model and improve the model capabilities. Assume that the first batch of new data D n (1) Arrival, since historical data cannot be obtained, usually only new data D is used n (1) Iteratively train the model M0. The trained model M0 will be updated to M1. And so on, forming a continuous and rapidly updated model stream. In the process of model update, it is possible to achieve rapid accumulation of knowledge just like the model obtained by training with full data. It can be seen that the accuracy of the model obtained by each update training in this embodiment is much higher than the accuracy of the model obtained by fine-tuning the model, and slightly lower than the model obtained by training with full data; however, the training time of the model obtained by training with full data is much longer than the training time of the model in this embodiment.

[0156] As shown in Figure 11, Figure 11 is a comparison diagram of the model training framework of this embodiment and the prior art; Figure 11 (a) is a framework diagram of the teacher model (T) and the student model (S) based on knowledge distillation training in the prior art; Figure 11 (b) is a framework diagram of the teacher model (T) and the student model (S) trained by the decision boundary distillation method with knowledge precipitation in this embodiment; the prior art inputs historical data into the teacher model and the student model at the same time, determines the first loss data based on the prediction results output by the two models, obtains an updated student model based on the first loss data, and then uses only the new data (new data) to update the updated learning model, that is, inputs the new data into the updated student model, determines the second loss data based on the third prediction result output by the updated student model and the object category label corresponding to the new data, and obtains the application model based on the second loss data; while this embodiment inputs new data (newly added sample images) into the teacher model and the learning model at the same time, trains the learning model based on the output of the teacher model, and then uses the trained student model to precipitate knowledge of the teacher model, and finally uses the teacher model as the application model. The specific training process is as follows: the initial sample images are input into the teacher model and the student model respectively to obtain the first historical category and the second historical category, and the sample loss information is calculated; when there are new sample images, the new sample images are input into the teacher model and the student model obtained in the previous training respectively, and the first loss information is determined according to the output results of the teacher model and the student model; according to the first sample category output by the teacher model and the new object category corresponding to the new sample image, the fusion category is obtained, and then according to the second sample category and the fusion category output by the student model, the second loss information is obtained; finally, the target loss information is determined according to the first loss information and the second loss information; then the parameters of the student model are adjusted according to the target loss information, and the knowledge of the teacher model is precipitated through the trained student model, and finally the teacher model is used as the application model (the final object category recognition model).

[0157] The method of this embodiment can perform model iteration without historical data, improve the model's capabilities, and not forget the learned capabilities, which not only saves time but also improves the efficiency of responding to difficult data in business scenarios. In general, this technical solution can mainly solve the following problems:

[0158] 1) Reduce training time and the consumption of graphics processing units (GPUs), and improve the response speed and efficiency to difficult data in application scenarios: Only new data is used to iteratively train the existing model. Therefore, it saves time and requires fewer GPUs compared to training from scratch with all the data. In business scenarios, after the difficult data (badcases) are transmitted back, all the data is generally used in combination with historical data to re-iterate the model. In addition to being time-consuming, this method may also result in a low proportion of difficult data in all data. After training is completed, the model's perception of this part of new data is not greatly improved. This method only involves new difficult data and can effectively avoid this problem.

[0159] 2) Knowledge accumulation and forgetting: Traditional model fine-tuning uses only newly acquired data, which can easily lead to forgetting past knowledge. This method helps alleviate this forgetting problem by distilling the decision boundary. Furthermore, by expanding the boundary and accumulating knowledge, it achieves knowledge accumulation for the first time, building on distillation.

[0160] 3) Model update and upgrade issues when there is no historical data.

[0161] 4) This method can be easily embedded in the training of any model and has broad prospects for promotion and application.

[0162] It can be seen from the technical solutions provided by the above embodiments of this specification that the embodiments of this specification disclose a training method for an object category recognition model, including: obtaining a new sample image of a sample object; the new sample image is annotated with a new object category; based on the initial object category recognition model, a teacher model and a student model are constructed; the initial object category recognition model is obtained by inputting the initial sample image of the sample object into a preset model for object category recognition training; the initial sample image is annotated with an initial sample object category; the initial sample object category is different from the new object category; the new sample image is input into the teacher model for a first object category recognition process to obtain a first sample category; the first sample category and the new object category are fused to obtain a fused category; the new sample image is input into the student model for a second object category recognition process to obtain a second sample category; based on the difference between the second sample category and the fused category, the student model is trained to obtain a final object category recognition model. In the process of updating and training the initial object category recognition model, the present application does not need to obtain the initial sample images, and only uses the newly added sample images to train the initial object category recognition model. It can learn new data to improve model performance while not forgetting historical knowledge, achieving a knowledge accumulation effect similar to that of the full data model. Compared with the training process of the full data, the model training method of the present application greatly shortens the training time and improves the model update efficiency.

[0163] The embodiment of this specification also provides an object category identification method, as shown in FIG12 , the method comprising:

[0164] S1201: Acquire an image of an object to be identified;

[0165] In the embodiments of this specification, the object to be identified and the sample object are objects in the same field, the image to be identified can be an image obtained by photographing the object to be identified, or an image extracted from the video to be identified corresponding to the object to be identified, and one object to be identified can correspond to one or more images to be identified.

[0166] S1203: Inputting the image to be identified into an object category recognition model to perform object category recognition processing to obtain a target object category of the object to be identified;

[0167] The object category recognition model is trained based on the above training method.

[0168] In the embodiment of the present specification, when the object to be identified corresponds to multiple images to be identified, the target object category of the object to be identified can be determined based on the category prediction results of the multiple images to be identified.

[0169] In the embodiments of this specification, the Cifar-100 and ImageNet-100 datasets are used to perform performance tests on the models of the prior art and the models trained by this solution; Cifar, which is the abbreviation of the "Canadian Institute for Advanced Research" developed by Canadian computer scientists Alex Krizhevsky, Vinod Nair and Geoffrey Hinton, is a dataset commonly used for image recognition. The CIFAR-100 dataset is a subset of the CIFAR dataset, with a total of 100 categories, each category containing 600 images. Among them, it contains 50,000 training images and 10,000 test images. Each image is 32x32 in size and is marked with the category it belongs to. ImageNet-100 is a subset of ImageNet ILSVRC2012, and its training set is randomly selected from the original training set and consists of 500 instances of 100 categories. The validation set is the corresponding 100 categories of the original validation set. The ILSXXS-2012 dataset is an image recognition dataset used in the ImageNet Large Scale Vision Recognition Challenge (ILSVRC). The ILSXXS-2012 dataset is part of the ImageNet project, which aims to advance the field of computer vision. Released in 2012, the dataset contains 1.2 million accurately annotated images from 1,000 different categories. Each category has approximately 1,000 training images and 50 validation images for model training and evaluation. The ILSXXS-2012 dataset covers a wide range of image categories, including people, animals, plants, objects, and scenes. The images also vary in resolution, with some images being very clear while others are blurry. Furthermore, the dataset includes challenging images, such as those with occlusion, rotation, scaling, and complex backgrounds.

[0170] As shown in Figures 13-14, Figure 13 is a test curve chart using the Cifar-100 dataset to test the performance of the existing technology model and the model trained by this solution; Figure 14 is a test curve chart using the ImageNet-100 dataset to test the performance of the existing technology model and the model trained by this solution. The horizontal axis is the task identifier, and different task identifiers correspond to different newly added sample images. The vertical axis is the percentage improvement in model accuracy. The meanings of the various curve identifiers in Figures 13 and 14 are as follows:

[0171] Curve 1: Full-data, model trained with full data;

[0172] Curve 2: LwF: learning without forgetting, no forgetting learning model;

[0173] Curve 3: Pooled outputs distillation for small-tasks incremental learning, referred to as PODNet, a pooled output distillation model for small-task incremental learning;

[0174] Curve 4: Dynamically expandable representation for class incremental learning, a dynamically expandable representation model for class incremental learning;

[0175] Curve 5: Balancing between forgetting and acquisition in incremental subpopulation learning, the balance model between forgetting and acquisition in incremental subpopulation learning;

[0176] Curve 6: Fine-tune, fine-tuning model;

[0177] Curve 7: Incremental classifier and representation learning, abbreviated as iCarl; incremental classifier and representation learning model;

[0178] Curve 8: Incremental learning in online scenario, referred to as Online_learning, is an incremental learning model in online scenarios.

[0179] Curve 9: Online prototype learning for online continual learning, referred to as OnPro, an online prototype learning model for online continuous learning;

[0180] Curve 10: object category recognition model of this embodiment.

[0181] As shown in Figures 13 and 14, using only new data, only the object category recognition model in this example can achieve stable model improvement on Cifar-100 and ImageNet-100. This method can be generalized and applied to any update or upgrade of an already deployed model.

[0182] The present embodiment further provides a training device for an object category recognition model, as shown in FIG15 , wherein the device includes:

[0183] A new sample image acquisition module 1510 is configured to acquire new sample images of sample objects; the new sample images are annotated with new object categories, and the new object categories are used to represent the categories of the sample objects;

[0184] The student model construction module 1520 is used to construct a teacher model and a student model based on the initial object category recognition model; the teacher model and the student model are respectively obtained by copying the initial object category recognition model;

[0185] A first sample result determination module 1530 is configured to input the newly added sample image into the teacher model to perform first object category recognition processing to obtain a first sample category, where the first sample category and the newly added object category are the same type of data;

[0186] A sample category fusion module 1540 is configured to fuse the first sample category and the newly added object category to obtain a fused category;

[0187] A second sample result determination module 1550 is configured to input the newly added sample image into the student model to perform second object category recognition processing to obtain a second sample category;

[0188] The student model training module 1560 is used to train the student model based on the difference between the second sample category and the fusion category to obtain a final object category recognition model.

[0189] In an exemplary embodiment, the apparatus further comprises:

[0190] a sample processing module, configured to perform data contamination processing on the newly added sample images to obtain sample processing data, wherein the contamination processing is used to change the distribution of the newly added sample images;

[0191] A first loss determination module, configured to input the sample processing data into the teacher model and the student model respectively to obtain first loss information;

[0192] The student model training module includes:

[0193] a second loss determining unit, configured to determine second loss information based on a difference between the second sample category and the fusion category;

[0194] a target loss determining unit, configured to determine target loss information based on the first loss information and the second loss information;

[0195] A parameter adjustment unit is used to adjust the model parameters of the student model based on the target loss information.

[0196] In an exemplary embodiment, the first loss determination module includes:

[0197] a third sample result determination unit, configured to input the sample processing data into the teacher model to perform first object category recognition processing to obtain a third sample category;

[0198] a fourth sample result determination unit, configured to input the sample processing data into the student model to perform second object category recognition processing to obtain a fourth sample category;

[0199] The first loss determining unit is configured to determine the first loss information based on a difference between the fourth sample category and the third sample category.

[0200] In an exemplary embodiment, the apparatus further comprises:

[0201] An updating module is used to update the teacher model based on the model parameters of the student model after the number of training times reaches a first target number to obtain a final object category recognition model. The number of training times refers to the number of times the student model is trained using the newly added sample images.

[0202] In an exemplary embodiment, the update module includes:

[0203] a teacher model updating unit, configured to update the teacher model based on the model parameters of the student model after the number of training times reaches a first target number, to obtain an updated teacher model;

[0204] An iterative updating unit is used to iteratively update the updated teacher model based on the model parameters of the student model during the training process after the number of training times reaches the first target number, so as to obtain a final object category recognition model.

[0205] In an exemplary embodiment, the iterative updating unit includes:

[0206] a model updating subunit, configured to update the updated teacher model based on the model parameters of the student model every time a second target number of training times has passed after the number of training times reaches the first target number of training times;

[0207] The model determination subunit is used to determine the updated teacher model as the final object category recognition model after the training is completed.

[0208] In an exemplary embodiment, the model updating subunit includes:

[0209] a current parameter acquisition subunit, configured to acquire current model parameters of the student model after each second target number of training times;

[0210] a target parameter determination subunit, configured to process the current model parameters based on an exponential moving average algorithm to obtain target model parameters;

[0211] An updating subunit is used to update the updated teacher model based on the target model parameters.

[0212] In an exemplary embodiment, the target parameter determination subunit includes:

[0213] a weight determination subunit, configured to determine a first weight of the teacher model and a second weight of the student model based on the number of training times corresponding to the current model parameters; wherein the first weight decreases as the number of training times increases, and the second weight increases as the number of training times increases;

[0214] A parameter processing subunit is used to process the current model parameters based on the first weight, the second weight and the exponential moving average algorithm to obtain the target model parameters.

[0215] In an exemplary embodiment, the apparatus further comprises:

[0216] an initial image acquisition module, configured to acquire the initial sample image of the sample object;

[0217] An initial result determination module, configured to input the initial sample image into the preset model for initial object category recognition processing to obtain an initial sample category;

[0218] a sample loss determination module, configured to determine sample loss information based on a difference between the initial sample category and the initial sample object category;

[0219] An initial model determination module is used to adjust the model parameters of the preset model based on the sample loss information until the training end condition is met, and determine the preset model at the end of training as the initial object category recognition model.

[0220] The embodiments of this specification also provide an object category recognition device, as shown in FIG16 , the device comprising:

[0221] The image acquisition module 1610 is used to acquire an image of an object to be identified.

[0222] A target object category determination module 1620 is configured to input the image to be identified into an object category recognition model to perform object category recognition processing to obtain a target object category of the object to be identified;

[0223] The object category recognition model is trained based on the above training method.

[0224] The device and method embodiments in the device embodiments are based on the same inventive concept.

[0225] An embodiment of the present specification provides an electronic device, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the object category recognition model training method or object category recognition method provided in the above method embodiment.

[0226] An embodiment of the present application also provides a computer storage medium, which can be set in a terminal to store at least one instruction or at least one program related to a training method for an object category recognition model in a method embodiment. The at least one instruction or at least one program is loaded and executed by the processor to implement the training method for the object category recognition model or the object category recognition method provided in the above method embodiment.

[0227] Embodiments of the present application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the object category recognition model training method or object category recognition method provided in the above method embodiment.

[0228] Optionally, in the embodiments of this specification, the storage medium may be located in at least one of the multiple network servers of the computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media that can store program code, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0229] The memory described in the embodiments of this specification can be used to store software programs and modules, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory may mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required for functions, etc.; the data storage area can store data created according to the use of the device, etc. In addition, the memory may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory may also include a memory controller to provide the processor with access to the memory.

[0230] The training method embodiment of the object category recognition model provided in the embodiments of this specification can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Taking running on a server as an example, Figure 17 is a hardware structure block diagram of a server for a training method of an object category recognition model provided in the embodiments of this specification. As shown in Figure 17, the server 1700 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPUs) 1710 (the central processing unit 1710 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 1730 for storing data, and one or more storage media 1720 (such as one or more mass storage devices) for storing application programs 1723 or data 1722. Among them, the memory 1730 and the storage medium 1720 can be temporary storage or permanent storage. The program stored in the storage medium 1720 may include one or more modules, each of which may include a series of instruction operations on the server. Furthermore, the central processing unit 1710 may be configured to communicate with the storage medium 1720 and execute a series of instruction operations in the storage medium 1720 on the server 1700. The server 1700 may also include one or more power supplies 1760, one or more wired or wireless network interfaces 1750, one or more input and output interfaces 1740, and / or one or more operating systems 1721, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0231] The input / output interface 1740 can be used to receive or send data via a network. Specific examples of the aforementioned network may include a wireless network provided by the communication provider of the server 1700. In one embodiment, the input / output interface 1740 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the input / output interface 1740 can be a radio frequency (RF) module for wireless communication with the Internet.

[0232] Those skilled in the art will appreciate that the structure shown in FIG17 is merely illustrative and does not limit the structure of the electronic device. For example, the server 1700 may include more or fewer components than shown in FIG17 or have a configuration different from that shown in FIG17.

[0233] From the embodiments of the training method, apparatus, device or storage medium of the object category recognition model provided by the above-mentioned present application, it can be seen that the present application discloses a training method for an object category recognition model, including: obtaining a new sample image of a sample object; the new sample image is annotated with a new object category; based on the initial object category recognition model, constructing a teacher model and a student model; inputting the new sample image into the teacher model for a first object category recognition process to obtain a first sample category; fusing the first sample category and the new object category to obtain a fusion category; inputting the new sample image into the student model for a second object category recognition process to obtain a second sample category; based on the difference between the second sample category and the fusion category, training the student model to obtain a final object category recognition model. In the process of model update training of the initial object category recognition model, the present application can train the initial object category recognition model using only the new sample image without obtaining the initial sample image, and can improve the model performance while learning new data without forgetting historical knowledge, thereby achieving a knowledge accumulation effect similar to that of the full data model, and compared with the training process of the full data, the model training method of the present application greatly shortens the training time and improves the model update efficiency.

[0234] It should be noted that the order in which the embodiments of this specification are presented is for illustrative purposes only and does not represent the superiority or inferiority of the embodiments. Furthermore, the foregoing descriptions are of specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in an order different from that in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0235] The various embodiments in this specification are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the device, equipment, and storage medium embodiments are generally similar to the method embodiments, so their descriptions are relatively simplified. For relevant portions, refer to the descriptions of the method embodiments.

[0236] Those skilled in the art will understand that all or part of the steps of implementing the above embodiments may be accomplished by hardware, or by a program instructing the relevant hardware to accomplish the steps. The program may be stored in a computer storage medium, and the above-mentioned storage medium may be a read-only memory, a disk, or an optical disk, etc.

[0237] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A training method for an object category recognition model, characterized in that: The method comprises: Acquire a new sample image of the sample object; the new sample image is annotated with a new object category, and the new object category is used to represent the category of the sample object; Based on the initial object category recognition model, a teacher model and a student model are constructed; the teacher model and the student model are respectively obtained by copying the initial object category recognition model; Inputting the newly added sample image into the teacher model to perform first object category recognition processing to obtain a first sample category, where the first sample category and the newly added object category are the same type of data; Performing a fusion process on the first sample category and the newly added object category to obtain a fusion category; Inputting the newly added sample image into the student model to perform second object category recognition processing to obtain a second sample category; Based on the difference between the second sample category and the fusion category, the student model is trained to obtain a final object category recognition model.

2. The method according to claim 1, characterized in that After acquiring the newly added sample image of the sample object, the method further includes: Performing data pollution processing on the newly added sample images to obtain sample processing data, wherein the pollution processing is used to change the distribution of the newly added sample images; Inputting the sample processing data into the teacher model and the student model respectively to obtain first loss information; The training of the student model based on the difference between the second sample category and the fusion category includes: determining second loss information based on a difference between the second sample category and the fusion category; determining target loss information based on the first loss information and the second loss information; Based on the target loss information, model parameters of the student model are adjusted.

3. The method according to claim 1 or 2, characterized in that The step of inputting the sample processing data into the teacher model and the student model respectively to obtain first loss information includes: Inputting the sample processed data into the teacher model to perform first object category recognition processing to obtain a third sample category; Inputting the sample processing data into the student model to perform second object category recognition processing to obtain a fourth sample category; The first loss information is determined based on a difference between the fourth sample category and the third sample category.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: After the number of training times reaches the first target number, the teacher model is updated based on the model parameters of the student model to obtain the final object category recognition model. The number of training times refers to the number of times the student model is trained using the newly added sample images.

5. The method according to any one of claims 1 to 4, characterized in that After the number of training times reaches the first target number of times, the teacher model is updated based on the model parameters of the student model to obtain a final object category recognition model, including: After the number of training times reaches a first target number of times, updating the teacher model based on the model parameters of the student model to obtain an updated teacher model; During the training process after the number of training times reaches the first target number of times, the updated teacher model is iteratively updated based on the model parameters of the student model to obtain a final object category recognition model.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: iteratively updating the updated teacher model based on the model parameters of the student model during the training process after the number of training times reaches the first target number of times to obtain a final object category recognition model, including: After the number of training times reaches the first target number of times, the updated teacher model is updated based on the model parameters of the student model every time a second target number of training times passes; After the training is completed, the updated teacher model is determined as the final object category recognition model.

7. The method according to any one of claims 1 to 6, characterized in that The updating of the teacher model based on the model parameters of the student model after each second target number of training times includes: After each second target number of training times, obtain the current model parameters of the student model; Processing the current model parameters based on an exponential moving average algorithm to obtain target model parameters; The updated teacher model is updated based on the target model parameters.

8. The method according to any one of claims 1 to 7, characterized in that The processing of the current model parameters based on the exponential moving average algorithm to obtain target model parameters includes: Determining a first weight of the teacher model and a second weight of the student model based on the number of training times corresponding to the current model parameters; wherein the first weight decreases as the number of training times increases, and the second weight increases as the number of training times increases; The current model parameters are processed based on the first weight, the second weight, and the exponential moving average algorithm to obtain the target model parameters.

9. The method according to any one of claims 1 to 8, characterized in that The initial object category recognition model is obtained by performing object category recognition training using the initial sample images of the sample objects; the initial sample images are annotated with the initial sample object categories; The initial sample object category is different from the newly added object category; Methods for training an initial object category recognition model include: Acquiring the initial sample image of the sample object; Inputting the initial sample image into a preset model to perform initial object category recognition processing to obtain an initial sample category; determining sample loss information based on a difference between the initial sample category and the initial sample object category; The model parameters of the preset model are adjusted based on the sample loss information until a training end condition is met, and the preset model at the end of training is determined as the initial object category recognition model.

10. A method for object category recognition, characterized in that: The method comprises: Obtain an image of an object to be identified; Inputting the image to be identified into an object category recognition model to perform object category recognition processing to obtain a target object category of the object to be identified; Wherein, the object category recognition model is trained based on the training method described in any one of claims 1-9.

11. A training device for an object category recognition model, characterized in that: The device comprises: A new sample image acquisition module is used to acquire a new sample image of a sample object; the new sample image is annotated with a new object category, and the new object category is used to represent the category of the sample object; A student model construction module is used to construct a teacher model and a student model based on the initial object category recognition model; the teacher model and the student model are respectively obtained by copying the initial object category recognition model; A first sample result determination module is configured to input the newly added sample image into the teacher model to perform first object category recognition processing to obtain a first sample category, where the first sample category and the newly added object category are the same type of data; a sample category fusion module, configured to fuse the first sample category and the newly added object category to obtain a fused category; A second sample result determination module is configured to input the newly added sample image into the student model to perform second object category recognition processing to obtain a second sample category; The student model training module is used to train the student model based on the difference between the second sample category and the fusion category to obtain a final object category recognition model.

12. An object category recognition device, characterized in that: The device comprises: An image acquisition module for obtaining an image of an object to be identified, used for obtaining an image of an object to be identified; a target object category determination module, configured to input the image to be identified into an object category recognition model for object category recognition processing to obtain a target object category of the object to be identified; Wherein, the object category recognition model is trained based on the training method described in any one of claims 1-9.

13. An electronic device, characterized in that: The device includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the training method according to any one of claims 1 to 9 or the object category recognition method according to claim 10.

14. A computer storage medium, characterized in that The computer storage medium stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the training method according to any one of claims 1 to 9 or the object category recognition method according to claim 10.

15. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the training method according to any one of claims 1 to 9 or the object category recognition method according to claim 10 is implemented.

Images