Model Training Method, Object Recognition Method, Device, Medium and Electronic Device

Abstract

The present disclosure relates to the field of computer technology, and specifically to a model training method, an object recognition method, a model training device, an object recognition device, a computer-readable storage medium, and an electronic device. The model training method in the embodiment of the present disclosure is used to perform additional training on a pre-trained model including a feature extraction layer and a classification layer, and the method includes: constructing an additional sample classification model based on the feature extraction layer of the pre-trained model; training the additional sample classification model using additional samples labeled with additional types to determine the additional classification weights of the additional types; and adding the additional classification weights to the classification layer of the pre-trained model. This method does not need to repeatedly train the original samples, and can continuously add new data types while ensuring the overall accuracy of the model, and can greatly save the model training cost and improve the model training efficiency.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, and particularly to a model training method, an object recognition method, a model training apparatus, an object recognition apparatus, a computer-readable storage medium, and an electronic device. Background Art

[0002] Currently, with the development of artificial intelligence, some automated devices have gradually replaced humans and played an important role. In the settlement system of daily consumption, the automatic settlement desk system for goods based on deep learning has been able to achieve full automation settlement completely through a real-time camera acquisition system and can obtain high accuracy. In such a full-automation settlement system, first, goods are detected through a real-time camera, and the feature information of a single commodity image is extracted, and then a query is made in a query database to obtain the final commodity type. The establishment of the query database is an important part of it, which contains the feature information of commodity images and commodity labels. Since there are a wide variety of daily commodities, and over time, some commodities will be taken off the shelves and new commodities will continue to increase, an important problem that follows is to establish a new image data query library. Therefore, it is necessary to perform database warehousing operations on new commodity images to meet the actual operation and maintenance requirements.

[0003] In the related art, a deep learning model can be used for automatic recognition and classification of goods. During the initial model training process of deep learning, a large number of image data samples often need to be collected to train a high accuracy. Since the types of new commodities are constantly increasing, the data acquisition level needs to reach the level of the old data set, which requires a large amount of manpower and material resources to collect and label the data, and the data labeling also needs to be performed by specialized technical personnel, and the overall cost is relatively high, which is not conducive to long-term development. And if the data acquisition volume of new commodities is too low, it will affect the final training accuracy. In addition, whenever a batch of new commodities is warehoused, the initial model needs to be retrained. As the number of types increases, the time and resources consumed by training are more, and new commodities will continuously disperse the recognition rate of old commodities. Therefore, as the number of types increases, the overall accuracy often continuously decreases.

[0004] It should be noted that the information disclosed in the above background art section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0005] The purpose of the present disclosure is to provide a model training method, an object recognition method, a model training apparatus, an object recognition apparatus, a computer-readable storage medium, and an electronic device, so as to at least to some extent overcome the technical problems such as high training cost and poor training accuracy existing during the additional training of the model due to the limitations of the related art.

[0006] According to one aspect of the present disclosure, there is provided a model training method for additional training of a pre-trained model including a feature extraction layer and a classification layer, characterized in that the method includes:

[0007] Construct an additional sample classification model based on the feature extraction layer of the pre-trained model;

[0008] Train the additional sample classification model with additional samples labeled with additional types to determine the additional classification weights for the additional types;

[0009] Add the additional classification weights to the classification layer of the pre-trained model.

[0010] In an exemplary embodiment of the present disclosure, the additional sample classification model includes an additional sample feature generator and an additional sample weight generator; the constructing an additional sample classification model based on the feature extraction layer of the pre-trained model includes:

[0011] Fix the parameters of the feature extraction layer of the pre-trained model to form the additional sample feature generator;

[0012] Obtain the original classification weights from the classification layer of the pre-trained model and construct the additional sample weight generator based on the original classification weights.

[0013] In an exemplary embodiment of the present disclosure, the constructing the additional sample weight generator based on the original classification weights includes:

[0014] Use the additional sample feature generator to extract features from the additional samples to obtain the feature vectors of the additional samples;

[0015] Calculate the mean vector corresponding to the additional type of the additional samples according to the feature vectors;

[0016] Construct the additional sample weight generator with the mean vector and the original classification weights of the pre-trained model as input parameters and the additional classification weights as output parameters.

[0017] In an exemplary embodiment of the present disclosure, the mean vector is calculated based on the following formula:

[0018]

[0019] where, F j is the mean vector of the additional type, m is the number of additional samples corresponding to the additional type, and f i is the feature vector of the additional sample.

[0020] In an exemplary embodiment of the present disclosure, the additional sample weight generator is constructed based on the following formula:

[0021]

[0022] Where W n is the additional classification weight of the additional type, n is the number of additional types, base is the number of original types of the pre-trained model, attention() is the attention model function, F j is the mean vector of the additional type, l b is the attention allocation parameter to be learned, and W b is the original classification weight of the pre-trained model.

[0023] According to one aspect of the present disclosure, there is provided an object recognition method, characterized in that it includes:

[0024] Obtain an image including an object to be recognized;

[0025] Input the image into a pre-trained object recognition model to obtain a recognition result of the object to be recognized;

[0026] Wherein, the object recognition model is trained by the model training method in any one of the above exemplary embodiments.

[0027] According to one aspect of the present disclosure, there is provided a model training device for performing additional training on a pre-trained model including a feature extraction layer and a classification layer, characterized in that the device includes:

[0028] A model construction module configured to construct an additional sample classification model based on the feature extraction layer of the pre-trained model;

[0029] A model training module configured to train the additional sample classification model with additional samples labeled with additional types to determine the additional classification weights of the additional types;

[0030] A weight addition module configured to add the additional classification weights to the classification layer of the pre-trained model.

[0031] According to one aspect of the present disclosure, there is provided an object recognition device, characterized in that it includes:

[0032] An image acquisition module configured to acquire an image including an object to be recognized;

[0033] An image recognition module configured to input the image into a pre-trained object recognition model to obtain a recognition result of the object to be recognized;

[0034] Among them, the object recognition model is trained by the model training method in any one of the above exemplary embodiments.

[0035] According to one aspect of the present disclosure, there is provided a computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the above-mentioned method is implemented.

[0036] According to one aspect of the present disclosure, there is provided an electronic device, characterized in that it includes a processor and a memory; wherein, the memory is used to store executable instructions of the processor, and the processor is configured to execute the above-mentioned method by executing the executable instructions.

[0037] In the model training method, object recognition method, model training device, object recognition device, computer-readable storage medium and electronic device provided by the exemplary embodiments of the present disclosure, by using the characteristics related to the underlying basic features of data samples, a new additional sample classification model is constructed on the basis of the feature extraction layer of the pre-trained model, and then the classification layer of the pre-trained model is updated by using the training results of the additional sample classification model. This additional training method does not need to repeat the training of the original samples. In addition, the training results of the additional samples mainly determine the recognition and classification accuracy of the additional types, and will not affect the recognition and classification accuracy of the original types. Therefore, new data types can be continuously added while ensuring the overall accuracy of the model. In addition, the data volume level of the additional samples does not have to be consistent with that of the original samples, which can avoid a large amount of complicated data collection and annotation work, occupy less resources, greatly save the model training cost, and improve the model training efficiency.

[0038] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.

[0040] Figure 1 Schematically shows a flowchart of the steps of a model training method in an exemplary embodiment of the present disclosure.

[0041] Figure 2 Schematically shows a partial flowchart of the steps of a model training method in another exemplary embodiment of the present disclosure.

[0042] Figure 3 Schematically shows a partial step flowchart of the model training method in another exemplary embodiment of the present disclosure.

[0043] Figure 4 Schematically shows a step flowchart of the object recognition method in an exemplary embodiment of the present disclosure.

[0044] Figure 5 Shows a schematic diagram of the principle of the model training method of the present disclosure in an application scenario.

[0045] Figure 6 Schematically shows a block diagram of the composition of the model training device in an exemplary embodiment of the present disclosure.

[0046] Figure 7 Schematically shows a block diagram of the composition of the object recognition device in an exemplary embodiment of the present disclosure.

[0047] Figure 8 Schematically shows a schematic diagram of a program product in an exemplary embodiment of the present disclosure.

[0048] Figure 9 Schematically shows a module diagram of an electronic device in an exemplary embodiment of the present disclosure. Detailed implementation manners

[0049] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more complete and comprehensive, and will fully convey the concept of the example embodiments to those skilled in the art. The features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments.

[0050] In addition, the drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repeated description will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0051] In an exemplary embodiment of the present disclosure, a model training method is first provided for additional training of a pre-trained model including a feature extraction layer and a classification layer. The pre-trained model is a weight model trained using original samples, and the model can at least include a feature extraction layer for extracting features from the original samples and a classification layer for outputting a classification result based on the extracted features. Taking the commodity automatic settlement desk system based on deep learning as an example, the pre-trained model in this exemplary embodiment can classify commodity images based on a classification model. The input data of the model is the commodity image, and the output data of the model is the corresponding commodity category. The pre-trained model in this exemplary embodiment can be a Convolutional Neural Networks (CNN) model or any other form of machine learning model. When the pre-trained model is a CNN model, the feature extraction layer can correspond to the convolutional layer and the fully connected layer in the CNN model, and the classification layer can use a softmax classifier. The corresponding loss function can be the CrossEntropy Loss function. Additionally, the classification layer can also use an SVM classifier or any other classification function model, and this exemplary embodiment does not make special limitations on this.

[0052] As Figure 1 shown, the model training method provided in this exemplary embodiment mainly includes the following steps:

[0053] Step S110. Construct an additional sample classification model based on the feature extraction layer of the pre-trained model.

[0054] In this step, the pre-trained pre-trained model is structurally separated, and an additional sample classification model is constructed based on its feature extraction layer. The additional sample classification model is a new weight model different from the pre-trained model and is used for additional training with additional samples as input data. Still taking the CNN model applied to commodity classification as an example, the convolutional layer of the CNN model, as the basic image feature extraction part, can extract the underlying features of the image. Whether it is the original samples used to train the pre-trained model or the additional samples used to train the additional sample classification model, most of the underlying feature information such as the shape, color, and texture of the commodities in the commodity images has a certain correlation. Therefore, in this step, after the CNN model is trained, its convolutional layer can be fixed as the feature extraction operator for the additional samples.

[0055] Step S120. Train the additional sample classification model with additional samples labeled with additional types to determine the additional classification weights of the additional types.

[0056] The pre-trained model is obtained by training with the original samples. In this step, it is necessary to expand the samples by appending additional samples on the basis of the original samples. The additional samples can mainly be new data samples that belong to different sample types from the original samples, so that new classification labels and classification weights corresponding to the additional samples can be added on the basis of the original classification results of the pre-trained model. After each additional sample is labeled with the additional type in this step, it is used as input data to train the additional sample classification model. After the training is completed, the additional classification weights of the additional type can be determined. Since the training process of the additional sample classification model is independent of the original pre-trained model, the data volume of the additional samples can be different from that of the original samples. Moreover, even when the data volume of the additional samples is very small, it will not affect the overall recognition accuracy, and thus new data categories can be added while ensuring the recognition accuracy of the original samples.

[0057] Step S130. Add the additional classification weights to the classification layer of the pre-trained model.

[0058] After the training process of step S120, the additional classification weights of the additional type can be obtained from the trained additional sample classification model. This step can add this part of the additional classification weights to the classification layer of the pre-trained model to complete the additional training of the pre-trained model, realizing the addition of new additional types and additional classification weights on the basis of the original type and original classification weights of the pre-trained model.

[0059] In the model training method provided in this exemplary embodiment, by utilizing the characteristics related to the underlying basic features of the data samples, a new additional sample classification model is constructed on the basis of the feature extraction layer of the pre-trained model, and then the classification layer of the pre-trained model is updated using the training results of the additional sample classification model. This method of additional training does not require repeated training of the original samples. In addition, the training results of the additional samples mainly determine the recognition and classification accuracy of the additional types, and will not affect the recognition and classification accuracy of the original types. Therefore, new data types can be continuously added while ensuring the overall accuracy of the model. In addition, the data volume of the additional samples does not have to be the same as that of the original samples, which can avoid a large amount of complicated data collection and annotation work, occupy less resources, and can greatly save the model training cost and improve the model training efficiency.

[0060] In another exemplary embodiment of the present disclosure, the additional sample classification model constructed by step S110 may include an additional sample feature generator and an additional sample weight generator. Among them, the additional sample feature generator is used to extract features from the additional samples, and the additional sample weight generator is used to generate the additional classification weights of the additional type according to the feature extraction results of the additional sample feature generator.

[0061] On this basis, as Figure 2As shown, step S110. Constructing an additional sample classification model based on the feature extraction layer of a pre-trained model may further include the following steps:

[0062] Step S210. Fix the parameters of the feature extraction layer of the pre-trained model to form an additional sample feature generator.

[0063] Taking the pre-trained CNN model as an example, in this step, the parameters of the convolutional layer of the CNN model can be fixed, while the fully connected layer needs to be connected to the network for training to obtain feature vectors adapted to the new additional types. The part used for feature extraction of the additional samples here constitutes the additional sample feature generator of the additional sample classification model.

[0064] Step S220. Obtain the original classification weights from the classification layer of the pre-trained model and construct an additional sample weight generator based on the original classification weights.

[0065] In this exemplary embodiment, the original classification weights corresponding to the original types can be obtained from the classification layer of the pre-trained model, and then the original classification weights are incorporated into the construction of the additional sample weight generator. For example, an additional sample weight generator can be generated using the attention mechanism, so that the additional classification weights of the additional types can be integrated using the original classification weights of the original types.

[0066] It should be noted that in other exemplary embodiments of the present disclosure, other forms of additional sample classification models can also be constructed, and the present disclosure does not make special limitations on this.

[0067] Based on the above exemplary embodiments, constructing an additional sample weight generator based on the original classification weights in step S220 may further include the following steps as shown Figure 3 shown:

[0068] Step S310. Use the additional sample feature generator to extract features from the additional samples to obtain the feature vectors of the additional samples.

[0069] The additional sample feature generator formed in step S210 can be used to extract features from the additional samples to obtain the feature vectors corresponding to the underlying features of the additional samples.

[0070] Step S320. Calculate the mean vector corresponding to the additional type of the additional samples according to the feature vectors.

[0071] The additional samples used in this exemplary embodiment can correspond to multiple different additional types. In this step, the feature vectors of the corresponding additional samples can be processed according to the different additional types to obtain the mean vectors of various additional types. For example, the mean vector in this step can be calculated based on the following formula:

[0072]

[0073] Among them, F j is the mean vector of the j-th additional type, m is the number of additional samples corresponding to this additional type, and f i is the feature vector of the i-th additional sample in this additional type.

[0074] Step S330. Use the mean vector and the original classification weights of the pre-trained model as input parameters and the additional classification weights as output parameters to construct an additional sample weight generator.

[0075] The additional sample weight generator constructed in this step uses the mean vector calculated in step S320 and the original classification weights of the pre-trained model as input parameters, and the additional classification weights of the additional type as output parameters. Specifically, it can perform branch calculations of the attention model on the mean vector and the original classification weights to obtain the weight branches of each additional type. For example, the additional sample weight generator in this step can be constructed based on the following formula:

[0076]

[0077] Among them, W n is the additional classification weight of the additional type in this training process, n is the number of additional types in this training process, base is the number of original types of the pre-trained model, attention() is the attention model function, and F j is the mean vector of the j-th additional type, l b is the attention allocation parameter to be learned, and W b is the original classification weight of the pre-trained model. The attention allocation parameter l b is learned through subsequent training processes.

[0078] The attention model function used in this exemplary embodiment can be a function based on cosine distance, Euclidean distance, or any other function representing vector distance. This exemplary embodiment does not make special limitations on this.

[0079] The present disclosure also provides an object recognition method based on the model training method in the above exemplary embodiment. This method can be applied to any artificial intelligence application scenarios such as commodity classification and product sorting based on image recognition. As Figure 4 shown, this method mainly includes the following steps:

[0080] Step S410. Obtain an image containing the object to be recognized.

[0081] In this step, an image acquisition device such as a camera or a video camera can be used to obtain an image containing the object to be recognized. For example, in an automatic checkout counter system for goods, a camera can be used to collect images of the goods to be settled in real time.

[0082] Step S420. Input the image into a pre-trained object recognition model to obtain the recognition result of the object to be recognized.

[0083] Input the image obtained in step S410 into the pre-trained object recognition model, and the recognition result of the object to be recognized in the image can be output. For example, it can be the product label of a specific product. Subsequently, the cost information corresponding to the product label can be queried from the database to complete the automatic settlement of the product. The object recognition model used in this step is trained by the model training method provided by any one of the above exemplary embodiments.

[0084] The following describes in detail the model training method provided in the exemplary embodiments of the present disclosure in conjunction with a specific application scenario.

[0085] As Figure 5 shown, this application scenario involves a model training method for storing a small number of product samples in an automatic checkout counter for goods. This method can perform training operations on additional samples based on a pre-trained model. An additional sample feature generator and an additional sample weight generator are mainly added to the original CNN model. The additional sample feature generator is used to analyze and extract additional sample features, and the additional sample weight generator is used to construct weights for new additional types in the classification layer. At this time, the classification layer can obtain a new classification result based on the original label. When training the model for newly stored products, the additional sample feature generator and the additional sample weight generator are mainly trained to meet the feature extraction of additional samples and the classification training between additional samples and original samples. This training method does not need to retrain the original samples. Only the parameters of the fully connected layer and the classification layer in the original samples are used, and training is combined with additional sample data, which will not affect the recognition rate of the original samples and can also recognize newly added additional samples.

[0086] The principle of the model training method in this application scenario can be specifically embodied in the following process:

[0087] Step 1: Establish an initial deep learning model, train it with the original sample data, and obtain the trained weight model. Extract features from the data in the feature library to be extracted (which stores the original samples) through the trained model to establish the original commodity feature library and index. The main purpose of this step is to establish a pre-trained model and the original image feature library. The training of the pre-trained model is an essential process in the deep learning settlement system. Generally, this model is based on a classification model to classify commodity images. The input is the sample image, the output is the corresponding category, and the loss function is the cross-entropy loss function. Use the trained model to extract features from the image search library data, and use the fully connected layer as the image feature layer to represent image information and establish the original image feature library.

[0088] Step 2: After obtaining the pre-trained model, separate the convolutional layer, fully connected layer, and classification layer of the model, and construct a new weight model. Fix the convolutional layer, and the fully connected layer needs to be connected to the network for training to obtain feature vectors adapted to the new additional types. This part is the process of determining the additional sample feature generator. In this step, the CNN model obtained from training in Step 1 is separated. The convolutional layer is used as the basic image feature extraction. It can be considered that the convolutional layer extracts the underlying features of the image, and there is a certain correlation between the shape, color, texture, etc. of most commodities in the commodity image. After the initial model is trained, fix the convolutional layer as the initial feature extraction operator for additional samples.

[0089] Step 3: Introduce the feature vectors of the additional samples obtained in Step 2 into the additional sample weight generator, and make corresponding adjustments to the classification layer. Combine the relevant parameters of the classification layer in the original samples, and introduce the calculation of the attention model parameters to generate the newly added classification weights for the additional samples.

[0090] Step 4: Jointly train the additional sample feature generator and the additional sample weight generator. The input is the additional sample commodity image, and the output is the additional sample category label. After training, use the feature extraction layer of the model as the data extraction layer for the additional samples, which can be used as the data features for the additional samples to be stored in the library. After establishing the additional sample feature generator and the additional sample weight generator through Step 2 and Step 3 respectively, this step can construct the input and output data, and use the gradient descent method to train the additional sample classification model. During training, the parameters of the additional sample feature generator and the additional sample weight generator will be updated according to the classification loss. The classification loss function is the cross-entropy loss function. After obtaining the model through training, send the additional sample query library data into the initial CNN model to obtain the preliminary feature extraction vector, and then obtain the final sample feature vector through the additional sample feature generator as the data for storage in the library.

[0091] As can be seen from the model training process in the above application scenarios, the model training method provided by the exemplary embodiments of the present disclosure can solve the problem of how to store and train a small amount of newly added commodity data samples after the automatic settlement system is established. Using this method for storage can avoid a large amount of complicated data collection and annotation work. Moreover, the data volume of the additional sample corresponding to each additional type in this method can often be controlled within 10 images, without the need to be consistent with the data volume of the original samples, and it will not affect the overall recognition accuracy. In addition, using this method can ensure that new commodity data categories are continuously added on the premise that the original commodity recognition rate does not decrease, and the resources occupied by training are less, which can solve the long-term storage problem of the current automatic settlement desk system based on deep learning.

[0092] It should be noted that although the above exemplary embodiments describe the steps of the method in the present disclosure in a specific order, this does not require or imply that these steps must be executed in this specific order, or that all steps must be executed to achieve the desired result. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step for execution, and / or one step may be decomposed into multiple steps for execution, etc.

[0093] In another exemplary embodiment of the present disclosure, there is also provided a model training apparatus based on the model training method in the above embodiments. The model training apparatus is used to perform additional training on a pre-trained model including a feature extraction layer and a classification layer. As Figure 6 shown, the model training apparatus 600 mainly may include: a model construction module 610, a model training module 620, and a weight addition module 630.

[0094] Among them, the model construction module 610 is configured to construct an additional sample classification model based on the feature extraction layer of the pre-trained model. The model construction module 610 can separate the structure of the pre-trained model obtained by pre-training, and construct an additional sample classification model on the basis of its feature extraction layer. The additional sample classification model is a new weight model different from the pre-trained model, and is used to perform additional training with the additional samples as input data.

[0095] The model training module 620 is configured to train the additional sample classification model using additional samples of the annotation append type to determine the additional classification weights of the append type. The pre-trained model is trained using the original samples. In this step, it is necessary to expand the additional samples based on the original samples. The additional samples can mainly be new data samples of different sample types from the original samples, so that new classification labels and classification weights corresponding to the additional samples can be added to the original classification results of the pre-trained model. The model training module 620 performs annotation of the append type on each additional sample and uses it as input data to train the additional sample classification model. After the training is completed, the additional classification weights of the append type can be determined.

[0096] The weight append module 630 is configured to add the additional classification weights to the classification layer of the pre-trained model. After being trained by the model training module 620, the additional classification weights of the append type can be obtained from the trained additional sample classification model. The weight append module 630 can then add this part of the additional classification weights to the classification layer of the pre-trained model to complete the additional training of the pre-trained model, realizing the addition of new append types and additional classification weights on the basis of the original type and original classification weights of the pre-trained model.

[0097] In another exemplary embodiment of the present disclosure, the model construction module 610 may further include: a feature generator construction module and a weight generator construction module.

[0098] Among them, the feature generator construction module is configured to fix the parameters of the feature extraction layer of the pre-trained model to form an additional sample feature generator. Taking the pre-trained CNN model as an example, the feature generator construction module can fix the parameters of the convolutional layer of the CNN model, while the fully connected layer needs to be connected to the network for training to obtain feature vectors adapted to the new append type. The part used for feature extraction of the additional samples here constitutes the additional sample feature generator of the additional sample classification model.

[0099] The weight generator construction module is configured to obtain the original classification weights from the classification layer of the pre-trained model and construct an additional sample weight generator based on the original classification weights. The weight generator construction module can obtain the original classification weights corresponding to the original type from the classification layer of the pre-trained model, and then incorporate the original classification weights into the construction of the additional sample weight generator. For example, an attention mechanism can be used to generate the additional sample weight generator, so that the additional classification weights of the append type can be integrated using the original classification weights of the original type.

[0100] In another exemplary embodiment of the present disclosure, the weight generator construction module may further include: a feature vector extraction unit, a mean vector calculation unit, and a generator construction unit.

[0101] Among them, the feature vector extraction unit is configured to use the additional sample feature generator to extract features from the additional samples to obtain the feature vectors of the additional samples. The additional sample feature generator constructed by using the feature generator construction module can extract features from the additional samples to obtain the feature vectors corresponding to the underlying features of the additional samples.

[0102] The mean vector calculation unit is configured to calculate the mean vector of the additional type corresponding to the additional samples according to the feature vectors. The additional samples used in this exemplary embodiment can correspond to multiple different additional types. The mean vector calculation unit can process the feature vectors of the corresponding additional samples according to different additional types to obtain the mean vectors of various additional types.

[0103] The generator construction unit is configured to use the mean vector and the original classification weights of the pre-trained model as input parameters and the additional classification weights as output parameters to construct an additional sample weight generator. The additional sample weight generator constructed by the generator construction unit uses the mean vector calculated by the mean vector calculation unit and the original classification weights of the pre-trained model as input parameters, and the additional classification weights of the additional type as output parameters. Specifically, it can perform branch calculations of the attention model on the mean vector and the original classification weights to obtain the weight branches of each additional type.

[0104] In another exemplary embodiment of the present disclosure, an object recognition device based on the object recognition method in the above embodiments is further provided. As Figure 7 shown, the object recognition device 700 mainly may include: an image acquisition module 710 and an image recognition module 720.

[0105] Among them, the image acquisition module 710 is configured to acquire an image including the object to be recognized. The image acquisition module 710 can use any image acquisition device such as a camera or a video camera to acquire an image including the object to be recognized. For example, in a commodity automatic settlement desk system, a camera can be used to collect images of commodities that need to be settled in real time.

[0106] The image recognition module 720 is configured to input the image into a pre-trained object recognition model to obtain the recognition result of the object to be recognized. The image recognition module 720 inputs the image acquired by the image acquisition module 710 into the pre-trained object recognition model and can output the recognition result of the object to be recognized in the image. For example, it can be the commodity label of a specific commodity, and then query the cost information corresponding to the commodity label from the database to complete the automatic settlement of the commodity. The object recognition model used by the image recognition module 720 is trained by the model training method provided in any one of the above exemplary embodiments.

[0107] Other specific details of the above model training device and object recognition device have been described in detail in the corresponding model training method and object recognition method, and thus will not be elaborated here.

[0108] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, such division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more of the above-described modules or units can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0109] In an exemplary embodiment of the present disclosure, there is also provided a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the above methods of the present disclosure can be implemented. In some possible embodiments, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program code; the program product can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network; when the program product runs on a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.), the program code is used to cause the computing device to execute the method steps in the above various exemplary embodiments of the present disclosure.

[0110] See Figure 8 As shown, the program product 800 for implementing the above method according to the embodiments of the present disclosure can adopt a portable compact disc read-only memory (CD-ROM) and include program code, and can run on a computing device (such as a personal computer, a server, a terminal device, or a network device, etc.). However, the program product of the present disclosure is not limited thereto. In this exemplary embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, apparatus, or device.

[0111] The program product can adopt any combination of one or more readable media. The readable media can be a readable signal medium or a readable storage medium.

[0112] A readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0113] A readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which the readable program code is carried. Such a propagated data signal may take many forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable signal medium may also be any readable medium other than the readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.

[0114] The program code contained on the readable medium may be transmitted with any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the above.

[0115] The program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as the C language or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, executed as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user's computing device through any type of network (including a local area network (LAN) or a wide area network (WAN), etc.); or, it may be connected to an external computing device, for example, by connecting through the Internet using an Internet service provider.

[0116] In an exemplary embodiment of the present disclosure, an electronic device is further provided, which includes at least one processor and at least one memory for storing executable instructions of the processor; wherein, the processor is configured to execute the method steps in the above exemplary embodiments of the present disclosure by executing the executable instructions.

[0117] The following is combined with Figure 9A description is given of the electronic device 900 in this exemplary embodiment. The electronic device 900 is merely an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.

[0118] Refer to Figure 9 As shown, the electronic device 900 is presented in the form of a general-purpose computing device. The components of the electronic device 900 may include, but are not limited to: at least one processing unit 910, at least one storage unit 920, a bus 930 connecting different system components (including the processing unit 910 and the storage unit 920), and a display unit 940.

[0119] Among them, the storage unit 920 stores program code, and the program code can be executed by the processing unit 910, so that the processing unit 910 executes the method steps in the above various exemplary embodiments of the present disclosure.

[0120] The storage unit 920 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit 921 (RAM) and / or a cache storage unit 922, and may further include a read-only storage unit 923 (ROM).

[0121] The storage unit 920 may also include a program / utilities 924 having a set (at least one) of program modules 925. Such program modules include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. Implementations of a network environment may be included in each or some combination of these examples.

[0122] The bus 930 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus structures.

[0123] The electronic device 900 can also communicate with one or more external devices 1000 (such as a keyboard, a pointing device, a Bluetooth device, etc.), can also communicate with one or more devices that enable a user to interact with the electronic device 900, and / or can communicate with any device that enables the electronic device 900 to communicate with one or more other computing devices (such as a router, a modem, etc.). Such communication can be carried out through an input / output (I / O) interface 950. And, the electronic device 900 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 960. As Figure 9As shown, network adapter 960 can communicate with other modules of electronic device 900 via bus 930. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0124] Those skilled in the art can understand that various aspects of the present disclosure can be implemented as a system, method, or program product. Therefore, various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software, which can be collectively referred to as "circuit", "module", or "system" here.

[0125] After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily conceive of other embodiments of the present disclosure. This application is intended to cover any variations, uses, or adaptations of the present disclosure, which follow the general principles of the present disclosure and include well-known knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and embodiments are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the appended claims.

[0126] The features, structures, or characteristics described above can be combined in any suitable manner in one or more embodiments. If possible, the features discussed in each embodiment are interchangeable. In the above description, many specific details are provided to give a full understanding of the embodiments of the present disclosure. However, those skilled in the art will realize that the technical solutions of the present disclosure can be practiced without one or more of the specific details, or other methods, components, materials, etc. can be used. In other cases, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

Claims

1. A model training method for additional training of a pre-trained model including a feature extraction layer and a classification layer, characterized in that, the pre-trained model includes a commodity classification model, the input data of the commodity classification model is a commodity image, and the output data is the corresponding commodity category; the method includes: Constructing an additional sample classification model based on the feature extraction layer of the pre-trained model; Training the additional sample classification model with additional samples labeled with additional types to determine the additional classification weights of the additional types; Adding the additional classification weights to the classification layer of the pre-trained model; wherein, the additional sample classification model includes an additional sample feature generator and an additional sample weight generator; constructing the additional sample classification model based on the feature extraction layer of the pre-trained model includes: Fixing the parameters of the feature extraction layer of the pre-trained model to form the additional sample feature generator; Obtaining the original classification weights from the classification layer of the pre-trained model, using the additional sample feature generator to extract features from the additional samples to obtain the feature vectors of the additional samples; calculating the mean vectors corresponding to the additional types of the additional samples according to the feature vectors; using the mean vectors and the original classification weights of the pre-trained model as input parameters and the additional classification weights as output parameters to construct the additional sample weight generator; wherein, the additional sample weight generator is constructed based on the following formula: Among them, W n is the additional classification weight of the additional type, n is the number of the additional types, base is the number of the original types of the pre-trained model, attention() is the attention model function, F j is the mean vector of the additional type, l b is the attention allocation parameter to be learned, W b is the original classification weight of the pre-trained model.

2. The model training method according to claim 1, characterized in that, the pre-trained model includes a CNN model; fixing the parameters of the feature extraction layer of the pre-trained model includes: Fixing the parameters of the convolutional layer of the CNN model.

3. The model training method according to claim 2, characterized in that, When training the additional sample classification model, the fully connected layer of the CNN model is also trained.

4. The model training method according to claim 1, characterized in that, The mean vector is calculated based on the following formula: Among them, F j is the mean vector of the said additional type, m is the number of additional samples corresponding to the said additional type, f i is the feature vector of the said additional sample.

5. The model training method according to claim 1, characterized in that, The attention model function is a function based on the distance of the representation vectors.

6. An object recognition method, characterized in that, including: Obtaining an image containing the object to be recognized; Inputting the image into a pre-trained object recognition model to obtain the recognition result of the object to be recognized; wherein, the object recognition model is trained by the model training method according to any one of claims 1-5.

7. A model training device for additional training of a pre-trained model including a feature extraction layer and a classification layer, characterized in that, the pre-trained model includes a commodity classification model, the input data of the commodity classification model is a commodity image, and the output data is the corresponding commodity category; the device includes: A model construction module configured to construct an additional sample classification model based on the feature extraction layer of the pre-trained model; A model training module, configured to train the additional sample classification model using additional samples of an annotation addition type to determine additional classification weights for the addition type; A weight addition module, configured to add the additional classification weights to a classification layer of the pre-trained model; Wherein, the additional sample classification model includes an additional sample feature generator and an additional sample weight generator; constructing the additional sample classification model based on a feature extraction layer of the pre-trained model includes: Fixing parameters of the feature extraction layer of the pre-trained model to form the additional sample feature generator; Obtaining original classification weights from the classification layer of the pre-trained model, using the additional sample feature generator to extract features from additional samples to obtain feature vectors of the additional samples; calculating a mean vector corresponding to the addition type of the additional samples according to the feature vectors; using the mean vector and the original classification weights of the pre-trained model as input parameters and using the additional classification weights as output parameters to construct the additional sample weight generator; Wherein, the additional sample weight generator is constructed based on the following formula: Among them, W n is the additional classification weight of the additional type, n is the number of the additional types, base is the number of the original types of the pre-trained model, attention() is the attention model function, F j is the mean vector of the additional type, l b is the attention allocation parameter to be learned, W b is the original classification weight of the pre-trained model.

8. An object recognition device, Characterized in that, Comprising: An image acquisition module, configured to acquire an image including an object to be recognized; An image recognition module, configured to input the image into a pre-trained object recognition model to obtain a recognition result of the object to be recognized; Wherein, the object recognition model is trained by the model training method according to any one of claims 1-5.

9. A computer-readable storage medium, having a computer program stored thereon, Characterized in that, The computer program, when executed by a processor, implements the method according to any one of claims 1-6.

10. An electronic device, Characterized in that, Comprising: A processor; A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the method according to any one of claims 1-6 by executing the executable instructions.

Images