Training of object classification model, object classification method, device and storage medium

Abstract

The application discloses a kind of training of object classification model, object classification method, device and storage medium, can be applied to cloud technology, artificial intelligence, wisdom traffic, Internet of Vehicles and various scenes, the method comprises: obtaining the sample image of sample object;Sample image is input into teacher model, and the sample label graph of sample image is determined based on the label graph extraction network of teacher model;Sample image is cropped, and sample cropped image is obtained;Based on the first coordinate information of sample cropped image in sample image, sample label graph is cropped, and sample sub-label graph is obtained;Sample cropped image is input into student model and is carried out category label prediction processing, and sample label prediction result is obtained;Based on the first difference of sample label prediction result and sample category label and the second difference of sample label prediction result and cropped sub-label, student model is trained, and object classification model is obtained.The application improves the accuracy of object classification model.

Description

Technical Field

[0001] This application relates to the field of Internet technology, and in particular to the training of an object classification model, an object classification method, an apparatus, and a storage medium. Background Technology

[0002] Industrial defect inspection refers to the quality inspection of industrial products during the manufacturing process. Traditionally, industrial quality inspection is carried out manually by quality control workers. In recent years, with the rise of AI technology, AI-based quality inspection based on machine vision can significantly improve the accuracy of quality inspection and save labor costs. The input of this technology is an image of the surface of the industrial product, and the output is the confidence level of the defect.

[0003] Traditional machine vision-based industrial defect inspection algorithms extract features manually from the input image, including gradient and texture features. Then, based on these features, an SVM classifier (or a tree-based classifier) ​​is trained to classify the image as defective. This approach has two problems: first, the extracted manual features have poor generalization ability, often extracting harmful features that confuse the subsequent classifier for diverse video data; second, feature extraction and classifier training are performed independently, resulting in high training costs. Other techniques train the model by labeling categories, extracting features first and then predicting categories. However, industrial defect inspection is not a simple binary classification; many defective images are relatively minor (even classifiable as OK images). Simple binary labels may be highly subjective, leading to noisy manual labeling. Training the model with such noisy labels negatively impacts accuracy. Summary of the Invention

[0004] This application provides a training method, apparatus, and storage medium for an object classification model, which can improve the training accuracy of the object classification model.

[0005] On the one hand, this application provides a method for training an object classification model, the method comprising: Obtain sample images of the sample objects; the sample images are labeled with sample category tags; The sample image is input into the teacher model, and the sample label map of the sample image is determined based on the label map extraction network of the teacher model. The sample image is cropped to obtain a cropped sample image; Based on the first coordinate information of the sample cropped image in the sample image, the sample label image is cropped to obtain a sample sub-label image; The cropped sample image is input into the student model for category label prediction processing to obtain the sample label prediction result; Based on the first difference between the sample label prediction result and the sample category label, and the second difference between the sample label prediction result and the cropped sub-label, the student model is trained to obtain an object classification model; the cropped sub-label is obtained by performing category label prediction processing on the sample sub-label map.

[0006] On the other hand, an object classification method is provided, the method comprising: Obtain the image of the object to be identified; The image to be identified is input into an object classification model for category prediction processing to obtain the target category of the image to be identified; the target category represents the category of the object to be identified. The object classification model is trained using the training method described above.

[0007] On the other hand, a training device for an object classification model is provided, the device comprising: The sample image acquisition module is used to acquire sample images of sample objects; the sample images are labeled with sample category tags. The sample label map determination module is used to input the sample image into the teacher model and determine the sample label map of the sample image based on the label map extraction network of the teacher model; The sample cropping image determination module is used to crop the sample image to obtain a sample cropping image; The sample sub-label image determination module is used to perform cropping processing on the sample label image based on the first coordinate information of the sample cropped image in the sample image to obtain the sample sub-label image; The sample result determination module is used to input the cropped sample image into the student model for category label prediction processing to obtain the sample label prediction result. The model training module is used to train the student model based on the first difference between the sample label prediction result and the sample category label and the second difference between the sample label prediction result and the cropped sub-label to obtain an object classification model; the cropped sub-label is obtained by performing category label prediction processing on the sample sub-label map.

[0008] In one exemplary embodiment, there are multiple sample images, each corresponding to a cropped sample image, and the apparatus further includes: The label determination module is used to determine the sample category label and cropping sub-label corresponding to each cropped image; An abnormal image determination module is used to determine any cropped image as an abnormal image if the similarity between the sample category label and the cropping sub-label corresponding to any cropped image is less than a preset threshold. The sample image selection module is used to remove abnormal images from a plurality of sample images to obtain a selected sample image. The sample cropping image determination module is used to determine the sample cropping image corresponding to the sample image being selected as the sample cropping image.

[0009] In one exemplary embodiment, the model training module includes: The model training submodule is used to train the student model based on the first difference between the sample label prediction result corresponding to the cropped image of the selected sample and the sample category label, and the second difference between the sample label prediction result corresponding to the cropped image of the selected sample and the cropping sub-label.

[0010] In one exemplary embodiment, the label determination module includes: The first determining submodule is used to determine the sample category label of the sample image corresponding to each sample cropped image; The second determining submodule is used to determine the sample category label of the sample image corresponding to each sample cropped image as the sample category label of each sample cropped image; The cropping sub-label acquisition submodule is used to determine the cropping sub-label of the sample sub-label image corresponding to each sample cropped image; The cropping sub-label determination submodule is used to determine the cropping sub-label of the sample sub-label image corresponding to each sample cropped image as the cropping sub-label of each sample cropped image.

[0011] In one exemplary embodiment, the sample sub-label map determination module includes: The first coordinate determination submodule is used to determine the first coordinate information of the sample cropped image in the sample image; The image size determination submodule is used to determine the image size corresponding to the sample image and the sample label image, respectively. The sample sub-label image determination submodule is used to perform cropping processing on the sample label image based on the first coordinate information of the sample cropped image in the sample image, and the image size corresponding to the sample image and the sample label image respectively, to obtain the sample sub-label image.

[0012] In one exemplary embodiment, the sample sub-label map determination submodule includes: The second coordinate determination unit is used to determine the first coordinate information of the cropped sample image in the sample image as the second coordinate information of the sample sub-label image in the sample label image if the sample image and the sample label image have the same image size. The sub-label image determination unit is used to perform cropping processing on the sample label image based on the second coordinate information to obtain the sample sub-label image.

[0013] In one exemplary embodiment, the apparatus further includes: The update module is used to adjust the size of the sample label image to the size of the sample image if the sample image and the sample label image have different corresponding image sizes, thereby obtaining an updated sample label image; The second coordinate determination module is used to determine the second coordinate information of the updated sample sub-label image in the updated sample label image based on the first coordinate information of the sample cropped image in the sample image; The image determination module is used to adjust the size of the updated sample label image based on the ratio of the corresponding sizes of the updated sample sub-label image and the sample label image, so as to obtain the sample label image.

[0014] In one exemplary embodiment, the model training module includes: The first information determination submodule is used to determine first loss information based on the first difference between the sample label prediction result and the sample category label; The second information determination submodule is used to determine second loss information based on the second difference between the sample label prediction result and the cropped sublabel; The target information determination submodule is used to determine target loss information based on the first loss information and the second loss information; The parameter adjustment submodule is used to adjust the model parameters of the student model based on the target loss information until the training termination condition is met, and to determine the student model at the end of training as the object classification model.

[0015] In one exemplary embodiment, the apparatus further includes: The training feature map determination module is used to input the sample image into the teacher model to be trained, and perform feature extraction processing on the sample image based on the initial feature extraction network of the teacher model to be trained to obtain the training feature map of the sample image. The training label map determination module is used to perform label map extraction processing on the training feature map based on the initial label map extraction network of the teacher model to be trained, so as to obtain the training label map. The training label prediction module is used to perform category label prediction processing on the training label map based on the initial category label prediction network of the teacher model to be trained, so as to obtain the training label prediction result. The teacher model determination module is used to train the teacher model to be trained based on the difference between the training label prediction result and the sample category label, so as to obtain the teacher model.

[0016] In one exemplary embodiment, the apparatus further includes: The sample category feature determination module is used to perform global pooling on the sample sub-label map to obtain sample category features; The normalization processing module is used to normalize the sample category features to obtain sample normalized features; The cropped sub-label determination module is used to perform category label prediction processing on the normalized features of the sample to obtain the cropped sub-labels corresponding to the sample sub-label map.

[0017] On the other hand, an object classification device is provided, the device comprising: The image acquisition module is used to acquire the image of the object to be identified. The target category determination module is used to input the image to be identified into an object classification model, perform category prediction processing, and obtain the target category of the image to be identified; the target category represents the category of the object to be identified. The object classification model is trained using the training method described above.

[0018] On the other hand, an electronic device is provided, the device including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the training method or object classification method of the object classification model as described above.

[0019] On the other hand, a computer storage medium is provided, which stores at least one instruction or at least one program, which is loaded and executed by a processor to implement the training method or object classification method of the object classification model as described above.

[0020] On the other hand, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform a training method or object classification method for implementing the object classification model as described above.

[0021] The object classification model training, object classification method, apparatus, and storage medium provided in this application have the following technical effects: This application obtains sample images of sample objects; the sample images are labeled with sample category labels; the sample images are input into a teacher model, and a label extraction network based on the teacher model determines the sample label map of the sample images; the sample images are cropped to obtain cropped sample images; based on the first coordinate information of the cropped sample image in the sample image, the sample label map is cropped to obtain sample sub-label maps; thus, the sample sub-label maps can be used as additional supervision signals in the training process of the learning model; the cropped sample images are then input into a student model for category label prediction processing to obtain sample label prediction results; based on the first difference between the sample label prediction results and the sample category labels, and the second difference between the sample label prediction results and the cropped sub-labels, the student model is trained to obtain an object classification model; the cropped sub-labels are obtained by performing category label prediction processing on the sample sub-label maps. This application determines the cropped sub-labels through the sample sub-label maps, thereby training the student model based on the first difference between the sample label prediction results and the sample category labels, and the second difference between the sample label prediction results and the cropped sub-labels; this application introduces sample sub-label maps as additional supervision signals in the training process of the object classification model, improving the accuracy of the object classification model. Attached Figure Description

[0022] To more clearly illustrate the technical solutions and advantages in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of a training system for an object classification model provided in an embodiment of this specification; Figure 2 This is a flowchart illustrating a training method for an object classification model provided in an embodiment of this specification. Figure 3 This is a flowchart illustrating a training method for a teacher model provided in an embodiment of this specification. Figure 4 This is a schematic diagram illustrating the features of a teacher model training process provided in the embodiments of this specification; Figure 5 This is a flowchart illustrating a method for cropping a sample label image to obtain a sample sub-label image, as provided in an embodiment of this specification. Figure 6This is a flowchart illustrating a method for cropping a sample label image to obtain a sample sub-label image based on the first coordinate information of the cropped sample image in the sample image and the image size corresponding to the sample image and the sample label image, as provided in an embodiment of this specification. Figure 7 This is a flowchart illustrating a method for determining a sample label image provided in an embodiment of this specification; Figure 8 This is a flowchart illustrating a method for training the student model to obtain an object classification model based on the first difference between the above-mentioned sample label prediction results and the above-mentioned sample category labels, and the second difference between the above-mentioned sample label prediction results and the cropped sub-labels, provided in an embodiment of this specification. Figure 9 This is a flowchart illustrating a method for determining cropped images for screening samples, as provided in the embodiments of this specification. Figure 10 This is a flowchart illustrating a method for determining the sample category label and cropping sub-label corresponding to each sample cropped image, as provided in the embodiments of this specification. Figure 11 This is a schematic diagram illustrating the features of a teacher-student model training process provided in the embodiments of this specification; Figure 12 This is a flowchart illustrating an object classification method provided in an embodiment of this specification; Figure 13 This is a schematic diagram of an object classification method provided in the embodiments of this specification; Figure 14 This specification provides an example of an image to be identified and its corresponding category prediction result. Figure 15 This is another image to be identified and its corresponding category prediction result provided in the embodiments of this specification; Figure 16 This is a schematic diagram of the structure of a training device for an object classification model provided in an embodiment of this specification; Figure 17 This is a schematic diagram of the structure of an object classification device provided in the embodiments of this specification; Figure 18 This is a schematic diagram of the structure of a server provided in the embodiments of this specification. Detailed Implementation

[0024] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] First, some of the nouns or terms that appear in the description of the embodiments in this specification are explained as follows: Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0026] Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in recognizing and measuring targets, and then performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0027] Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0028] Deep Learning (DL) is a branch of machine learning that attempts to perform high-level abstractions of data using multiple processing layers with complex structures or multiple nonlinear transformations. Deep learning learns the inherent patterns and hierarchical representations of training sample data; the information gained during this learning process greatly aids in interpreting data such as text, images, and sound. The ultimate goal of deep learning is to enable machines to possess analytical and learning capabilities similar to humans, capable of recognizing data such as text, images, and sound. Deep learning is a complex machine learning algorithm, and its performance in speech and image recognition far surpasses previous related technologies.

[0029] Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. Cloud technology is a collective term for network technologies, information technologies, integration technologies, management platform technologies, and application technologies applied to cloud computing business models. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing.

[0030] Big data refers to data sets that cannot be captured, managed, and processed within a certain timeframe using conventional software tools. It represents massive, rapidly growing, and diverse information assets that require new processing models to achieve stronger decision-making, insightful discovery, and process optimization capabilities. With the advent of the cloud era, big data has attracted increasing attention. Big data requires specialized technologies to effectively process large amounts of data within a tolerable timeframe. Technologies suitable for big data include massively parallel processing databases, data mining, distributed file systems, distributed databases, cloud computing platforms, the Internet, and scalable storage systems. The image to be identified in the embodiments of this application can be one type of big data.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0032] Please see Figure 1 , Figure 1 This is a schematic diagram of a training system for an object classification model provided in an embodiment of this specification, such as... Figure 1 As shown, the training system for this object classification model can include at least server 01 and client 02.

[0033] Specifically, in the embodiments of this specification, server 01 may include a standalone server, a distributed server, or a server cluster composed of multiple servers. It may also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Server 01 may include a network communication unit, a processor, and a memory, etc. Specifically, server 01 can be used to train a teacher model, and based on the teacher model and student model, train an object classification model; and receive the image of the object to be identified sent by client 02, input the image into the object classification model, and obtain the target category of the object to be identified.

[0034] Specifically, in the embodiments of this specification, the client 02 may include physical devices such as smartphones, desktop computers, tablets, laptops, digital assistants, smart wearable devices, smart speakers, in-vehicle terminals, and smart TVs. It may also include software running on the physical device, such as web pages provided to users by service providers, or applications provided to users by these service providers. Specifically, the client 02 can be used to display the target category of the object to be identified; in the field of industrial product inspection, the object is an industrial product, and the target category can characterize whether the industrial product is a qualified product or a defective product.

[0035] The following describes a training method for an object classification model proposed in this application. Figure 2 This is a flowchart illustrating a training method for an object classification model provided in an embodiment of this specification. This specification provides the operational steps of the method described in the embodiment or flowchart, but based on conventional or non-inventive labor, more or fewer operational steps may be included. The order of steps listed in the embodiment is merely one possible execution order among many and does not represent the only possible execution order. In actual system or server products, the method can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment) as shown in the embodiment or accompanying drawings. Specifically, as... Figure 2 As shown, the above method may include: S201: Obtain the sample image of the sample object; the sample image is labeled with the sample category label. In the embodiments of this specification, the sample image can be an image obtained by taking a picture of the sample object, or an image extracted from the sample video corresponding to the sample object. One sample object can correspond to one or more sample images. The sample object can include positive sample objects and negative sample objects. The positive sample image corresponding to the positive sample object is labeled with a positive sample category label, and the negative sample image corresponding to the negative sample object is labeled with a negative sample category label.

[0036] The method in this embodiment can be applied to the conformity inspection of industrial products, and can identify defective products by recognizing defective images. In the field of industrial product inspection, the sample object can be an industrial product, and the sample image can be an image of an industrial product; the positive sample object can be a qualified product, and the negative sample object can be a product with defects.

[0037] The method of this embodiment can also be applied to other scenarios, and the sample objects will be different in different application scenarios. For example, the sample objects can be animals or plants in nature, and this embodiment can classify animals or plants in nature; the sample objects can also be electronic products, and electronic products can be classified, etc.

[0038] S203: Input the above sample images into the teacher model, and determine the sample label images of the above sample images based on the label image extraction network of the above teacher model.

[0039] In the embodiments of this specification, a teacher network can be pre-trained to obtain a teacher model. The teacher network is a first deep model, and the number of parameters of the model is greater than a first parameter threshold. The first parameter threshold can be set according to the actual situation.

[0040] In some embodiments, such as Figure 3 As shown, the training methods for the above teacher model include: S301: Input the above sample image into the teacher model to be trained, and perform feature extraction processing on the above sample image based on the initial feature extraction network of the teacher model to be trained to obtain the training feature map of the above sample image. In the embodiments of this specification, the teacher model to be trained may include an initial feature extraction network, which can be used to perform feature extraction processing on the sample images to obtain training feature maps of the sample images; in an exemplary embodiment, as shown... Figure 4 As shown, Figure 4 This is a feature diagram illustrating the training process of a teacher model. Sample data (x, y) can be input into the initial feature extraction network, where x represents the sample image and y represents the sample class label of the sample image. The training feature map extracted by the initial feature extraction network is as follows:

[0041] in, This represents the weight parameters of the initial feature extraction network. express Shape feature map, three dimensions These represent the height, width, and number of channels, respectively.

[0042] S303: Based on the initial label map extraction network of the above-mentioned teacher model to be trained, the above-mentioned training feature map is processed to extract the label map to obtain the training label map; In the embodiments of this specification, the teacher model to be trained may include an initial label map extraction network, which may be a convolutional layer; the initial label map extraction network can extract label maps of training feature maps to obtain training label maps; in an exemplary embodiment, as shown... Figure 4 As shown, Figure 4 This is a feature illustration of the teacher model training process; the training label map (predicted label map) extracted through the convolutional layer is as follows:

[0043] in, This represents the convolution operation of a convolutional layer. This represents the weight parameters of the convolutional layer. Represent a Feature map of shape, This indicates the total number of category labels.

[0044] S305: Based on the initial category label prediction network of the above-mentioned teacher model to be trained, the above-mentioned training label map is processed for category label prediction to obtain the training label prediction result. In the embodiments of this specification, the teacher model to be trained may include an initial class label prediction network, which may include a global pooling layer and a normalization layer; the normalization layer may use a normalization exponential function (softmax) for probability normalization; the training label map can be globally pooled by the global pooling layer to obtain training probability features; the training probability features are then input into the normalization layer to obtain a prediction probability map; the prediction probability map can represent the training label prediction result; in an exemplary embodiment, such as Figure 4 As shown, Figure 4 This is a feature diagram illustrating the training process of a teacher model, and the resulting prediction probability map is as follows:

[0045] in, This indicates a global pooling operation, which performs global pooling based on the spatial dimension, and uses... The function is probability normalized, and finally a value of is obtained. Predicted probability plot .

[0046] S307: Based on the difference between the above training label prediction results and the above sample category labels, the above teacher model to be trained is trained to obtain the above teacher model.

[0047] In the embodiments of this specification, loss information can be determined based on the difference between the training label prediction results and the aforementioned sample category labels; the parameters of the teacher model to be trained can be adjusted based on the loss information until the training termination condition is met; and the teacher model to be trained at the end of training is determined as the teacher model. The training termination condition can be set according to actual circumstances; for example, it can be determined based on the loss information or the number of training iterations.

[0048] In an exemplary embodiment, the damage function during model training can be the cross-entropy loss function, which can be based on the predicted probability graph. The loss information is calculated using the labels corresponding to the input data, and the calculation formula is as follows:

[0049] in, This represents the cross-entropy loss function, which is then used to calculate the loss. Gradient descent is used to iteratively update the model parameters, resulting in the weight values ​​of the trained model. and .

[0050] In the embodiments of this specification, a structure with a large number of model parameters is used as the teacher model. The main consideration is that larger models have stronger learning and fitting capabilities, and these models mainly play the role of providing additional supervision signals in the subsequent secondary training stage. The teacher model is not directly deployed in the final deployment stage; therefore, the problem of increased time consumption caused by large models is not involved.

[0051] S205: Crop the above sample image to obtain a cropped sample image.

[0052] In the embodiments of this specification, random cropping can be used to perform data augmentation on sample images to obtain cropped sample images; the size of the cropped sample images can be the same or different for different sample images.

[0053] S207: Based on the first coordinate information of the above sample cropped image in the above sample image, the above sample label image is cropped to obtain a sample sub-label image.

[0054] In the embodiments of this specification, the sample label image can be cropped according to the first coordinate information of the cropped sample image in the sample image to obtain a sample sub-label image; the proportion of the sample sub-label image in the sample label image is the same as the proportion of the cropped sample image in the sample image.

[0055] In the embodiments described in this specification, such as Figure 5 As shown, based on the first coordinate information of the cropped sample image in the sample image, the sample label image is cropped to obtain a sample sub-label image, including: S2071: Determine the first coordinate information of the above-mentioned sample cropped image in the above-mentioned sample image; In the embodiments of this specification, after image cropping, the first coordinate information of the cropped sample image in the above sample image can be determined. The first coordinate information may include the cropping start coordinate and the cropping end coordinate of the cropped sample image. In some embodiments, the cropped sample image is a rectangle. The cropping start coordinate can be the upper left corner coordinate, and the cropping end coordinate can be the lower right corner coordinate.

[0056] S2073: Determine the image size corresponding to the above sample image and the above sample label image respectively; In the embodiments of this specification, the image size corresponding to the sample image and the sample label image determined based on the sample image may be the same or different.

[0057] S2075: Based on the first coordinate information of the cropped sample image in the sample image, and the image size corresponding to the sample image and the sample label image respectively, the sample label image is cropped to obtain the sample sub-label image.

[0058] In the embodiments of this specification, the sample sub-label image can be quickly determined based on the first coordinate information of the sample cropped image in the above sample image and the image size of the sample image and the sample label image.

[0059] In the embodiments described in this specification, such as Figure 6 As shown, based on the first coordinate information of the cropped sample image in the sample image, and the image sizes corresponding to the sample image and the sample label image respectively, the sample label image is cropped to obtain the sample sub-label image, including: S20751: If the sample image and the sample label image have the same image size, the first coordinate information of the cropped sample image in the sample image shall be determined as the second coordinate information of the sample sub-label image in the sample label image. In the embodiments of this specification, if the sample image and the sample label image have the same image size, when the sample image and the sample cropped image are superimposed, it can be determined that the position coordinates of the sample sub-label image in the sample label image are the same as the position coordinates of the sample cropped image in the sample image.

[0060] In some embodiments, the sample cropped image is rectangular; the cropping start coordinates can be the top-left corner coordinates, and the cropping end coordinates can be the bottom-right corner coordinates; that is, the first coordinate information (start and end coordinates) can be... If the sample image and the sample label image have the same image size, the coordinates of the sample sub-label image within the sample label image can be determined as follows: .

[0061] S20753: Based on the second coordinate information mentioned above, the sample label image is cropped to obtain the sample sub-label image.

[0062] In one exemplary embodiment, the second coordinate information includes the cropping start coordinates and cropping end coordinates of the sample sub-label image, wherein the sample sub-label image is:

[0063] in, For sample label images, This is the clipping function. This is a sample sub-label image.

[0064] In the embodiments described in this specification, such as Figure 7 As shown, the above method also includes: S207501: If the image size corresponding to the above sample image and the above sample label image is different, adjust the size of the above sample label image to the size of the above sample image to obtain an updated sample label image. In the embodiments of this specification, when the sample image and the sample label image have different corresponding image sizes, the two images can be adjusted to the same size; the size of the sample label image can be adjusted to the size of the sample image to obtain an updated sample label image; or the size of the sample image can be adjusted to the size of the sample label image.

[0065] S207503: Based on the first coordinate information of the cropped sample image in the sample image, determine the second coordinate information of the updated sample sub-label image in the updated sample label image; In the embodiments of this specification, the second coordinate information of the updated sample sub-label image in the updated sample label image can be determined based on the first coordinate information of the sample cropped image in the sample image; the sample image of the same size and the updated sample sub-label image are overlapped, and a coordinate system is constructed, and the first coordinate information of the sample cropped image in the sample image is determined as the second coordinate information of the updated sample sub-label image in the updated sample label image.

[0066] S207505: Based on the ratio of the corresponding dimensions of the updated sample sub-label image and the sample label image, the updated sample label image is resized to obtain the sample label image.

[0067] In the embodiments of this specification, the updated sample label image can be resized according to the ratio of the corresponding sizes of the updated sample sub-label image and the sample label image to obtain a sample label image; that is, the updated sample sub-label image is resized back to its original size, and the size of the updated sample label image is adjusted simultaneously to obtain a sample label image.

[0068] S209: Input the cropped sample image above into the student model for category label prediction to obtain the sample label prediction result.

[0069] In the embodiments of this specification, the number of parameters in the student model is less than the number of parameters in the teacher model, the number of parameters in the student model is less than the second parameter threshold, and the second parameter threshold is less than the first parameter threshold.

[0070] S2011: Based on the first difference between the above sample label prediction results and the above sample category labels, and the second difference between the above sample label prediction results and the cropped sub-labels, the above student model is trained to obtain an object classification model; the above cropped sub-labels are obtained by performing category label prediction processing on the above sample sub-label map.

[0071] In the embodiments of this specification, a first difference between the sample label prediction result and the above-mentioned sample category label can be determined, and a second difference between the sample label prediction result and the cropped sub-label can be determined; then, based on the first difference and the second difference, the parameters of the student model are adjusted by backpropagation until the training termination condition is met; and the student model at the end of training is determined as the object classification model.

[0072] In some embodiments, the above method further includes: The above sample sub-label images are processed by category label prediction to obtain the cropped sub-labels corresponding to the above sample sub-label images.

[0073] In the embodiments of this specification, the cropped sub-labels corresponding to the sample sub-labels can be obtained by performing category label prediction processing on the sample sub-label images, and the cropped sub-labels can be used as supervision signals in the training process of the learning model, thereby improving the model accuracy.

[0074] In the embodiments of this specification, category label prediction processing is performed on the above-mentioned sample sub-label image to obtain the cropped sub-labels corresponding to the above-mentioned sample sub-label image, including: Global pooling is performed on the above sample sub-label images to obtain sample category features; In the embodiments described in this specification, global pooling reduces dimensionality from 3 dimensions to 1 dimension, outputting one response operation for each feature map. Sample category features can be obtained by performing global pooling on the sample sub-label maps.

[0075] The above sample category features are normalized to obtain sample normalized features; In the embodiments of this specification, the sample category features can be normalized using the normalization exponential function (softmax) to obtain normalized sample features. The normalization exponential function, or softmax function, is a generalization of the logistic function. It can "compress" a K-dimensional vector z containing arbitrary real numbers into another K-dimensional real vector σ(z), such that each element is in the range (0, 1), and the sum of all elements is 1. This function is often used in multi-class classification problems.

[0076] The normalized features of the above samples are processed for category label prediction to obtain the cropped sub-labels corresponding to the above sample sub-label images.

[0077] In the embodiments of this specification, the normalized features of the samples can be processed for category label prediction to obtain the cropped sub-labels corresponding to the above sample sub-label images; in an exemplary embodiment, the calculation formula for the cropped sub-labels is as follows:

[0078] in, This is the sample sub-label map, GlobalPooling is the global pooling function, and softmax is the normalization exponential function.

[0079] In the embodiments described in this specification, such as Figure 8 As shown, based on the first difference between the above sample label prediction results and the above sample category labels, and the second difference between the above sample label prediction results and the cropping sub-labels, the above student model is trained to obtain an object classification model, including: S20111: Based on the first difference between the above sample label prediction results and the above sample category labels, determine the first loss information; In the embodiments described in this specification, the first loss information The calculation formula is as follows:

[0080] Among them, the loss function corresponding to the first loss information is the pixel-level cross entropy loss (CE), which examines each pixel one by one and compares the prediction result (probability distribution vector) for each pixel category with the hot-coded label vector. This represents the sample label prediction result of the student model, where y is the sample category label.

[0081] S20113: Based on the second difference between the above sample label prediction results and the above cropped sub-labels, determine the second loss information; In the embodiments described in this specification, the second loss information The calculation formula is as follows:

[0082] Among them, the loss function corresponding to the second loss information is the cross entropy loss (CE); This represents the sample label prediction results of the student model. This indicates the cropping sub-label.

[0083] S20115: Based on the first loss information and the second loss information mentioned above, determine the target loss information; In an exemplary embodiment, the sum of the first loss information and the second loss information described above can be calculated to obtain the target loss information.

[0084] In the embodiments of this specification, a weighted sum of the first loss information and the second loss information can also be calculated to obtain the target loss information; the formula for calculating the target loss information is as follows:

[0085] in, , For the second loss information The weights can be used to adjust the proportion of the two losses.

[0086] S20117: Based on the above target loss information, adjust the model parameters of the above student model until the training termination condition is met, and determine the student model at the end of training as the above object classification model.

[0087] In the embodiments of this specification, the loss function The parameters of the student model will be updated through gradient calculation and backpropagation. The training termination condition can be either that the target loss information is less than a preset loss threshold, or that the number of iterations reaches a target number. The student model at the end of training can be defined as an object classification model.

[0088] In the embodiments described in this specification, there are multiple sample images, and each sample image corresponds to a cropped sample image, such as... Figure 9 As shown, the above method also includes: S901: Determine the sample category label and cropping sub-label corresponding to each cropped image; In some embodiments, such as Figure 10 As shown, the above determination of the sample category label and cropping sub-label corresponding to each sample cropped image includes: S9011: Determine the sample category label of the sample image corresponding to each cropped sample image; In the embodiments of this specification, the correspondence between the cropped sample image and the sample image is defined by determining the sample category label corresponding to each sample image as the sample category label of the cropped sample image corresponding to that sample image.

[0089] S9013: Determine the sample category label of the sample image corresponding to each sample cropped image as the sample category label of each sample cropped image mentioned above; In the embodiments of this specification, the sample image corresponding to each sample cropping image can be determined first based on the correspondence between the sample cropping image and the sample image, and then the sample category label of the sample image can be obtained and the sample category label of the sample image can be determined as the sample category label of the sample cropping image.

[0090] S9015: Determine the cropping sub-label of the sample sub-label image corresponding to each sample cropped image; In the embodiments of this specification, for any sample cropped image, the sample sub-label image corresponding to the sample cropped image can be determined first, and then the cropped sub-label of the sample sub-label image can be obtained.

[0091] S9017: Determine the cropping sub-label of the sample sub-label image corresponding to each sample cropped image as the cropping sub-label of each sample cropped image.

[0092] In the embodiments of this specification, for any sample cropped image, the sample sub-label image corresponding to the sample cropped image can be determined first, and then the cropping sub-label of the sample sub-label image can be determined as the cropping sub-label of the sample cropped image; thus obtaining the cropping sub-label of the sample cropped image.

[0093] S903: If the similarity between the sample category label and the cropping sub-label corresponding to any cropped sample image is less than a preset threshold, the above cropped sample image is determined to be an abnormal image. In the embodiments of this specification, if the similarity between the sample category label and the cropping sub-label corresponding to any sample cropped image is less than a preset threshold, it indicates that the difference between the prediction result (cropping sub-label) of the teacher model and the labeled sample category label is large, and the two are inconsistent; it indicates that the label of the sample image is a noise label, and the sample data is noise data, which needs to be deleted from the sample dataset and not participate in the training of the model, thereby reducing the impact of noise label samples on the model and avoiding affecting the accuracy of the model.

[0094] In the embodiments of this specification, the similarity between the sample category label and the cropping sub-label corresponding to any sample cropped image is less than a preset threshold, i.e., it satisfies:

[0095] in, The distance measurement function can be represented by KL divergence, etc. It is a hyperparameter used to control the preset threshold of the distance between two distributions.

[0096] S905: Remove the abnormal images from the above sample images to obtain the filtered sample images; In the embodiments described in this specification, abnormal images can be identified as noise sample data, which need to be filtered out and are not included in the updating of model parameters.

[0097] S907: The cropped image corresponding to the above-mentioned selected sample image is determined as the selected sample cropped image.

[0098] In the embodiments of this specification, sample images with a similarity greater than a preset threshold between the sample category label and the cropping sub-label can also be selected from multiple sample images to serve as the selected sample cropping images. In some embodiments, the student model is trained based on a first difference between the predicted sample label and the predicted sample category label, and a second difference between the predicted sample label and the cropped sub-label, including: Based on the first difference between the predicted sample label and the sample category label of the cropped image of the selected sample, and the second difference between the predicted sample label and the cropped sub-label of the cropped image of the selected sample, the student model is trained.

[0099] In the embodiments of this specification, data that is not identified as noise samples will be used to calculate the loss and update the parameters of the student model, while data that is identified as noise samples will not be used to calculate the loss and update the parameters of the student model, thereby reducing the impact of noise-labeled samples on the model and avoiding affecting the accuracy of the model.

[0100] In one exemplary embodiment, such as Figure 11 As shown, Figure 11 This is a feature diagram illustrating the training process of a teacher model and a student model. Deep model 1 is the teacher model, and deep model 2 is the student model. The student model and the teacher model can be deep network models of the same type and structure, such as convolutional neural networks (CNN), residual networks (ResNet, RN), etc. The teacher model includes a feature extraction network and a label extraction network. Training data (x, y) is input into the feature extraction network, where x represents a sample image and y represents the sample category label. The feature extraction network extracts sample feature maps. The label extraction network extracts label maps from the sample feature maps, resulting in sample label maps. Spatial local pooling and normalization are then performed on the sample label maps to obtain cropped sub-labels corresponding to the sample sub-labels. These cropped sub-labels are used as distillation probability supervision signals for training the student model. After cropping the training data (sample images), it is input into the student model for category label prediction, yielding sample label prediction results. Sample images are filtered based on the difference between the sample category label and the cropped sub-label corresponding to the same sample image, removing those with significant differences between the sample category label and the cropped sub-label, thus excluding them from the student model's training. By filtering out noisy data and removing it from the training data, the model can avoid interference from noise, enhancing the robustness and accuracy of the student model.

[0101] For industrial defect detection tasks, the method in this embodiment can accurately detect the degree of defects in the input image. The teacher model pre-stores the sample label map, and combined with the random cropping data augmentation method in the training stage of the student model, the sub-label map of the teacher model is used as an additional supervision signal and compared with the original labeled label. This filters out noisy data and removes it from the training data, so that the model can avoid the interference of noisy data and enhance the robustness and accuracy of the student model.

[0102] As can be seen from the technical solutions provided in the embodiments of this specification above, the embodiments of this specification obtain sample images of sample objects; the sample images are labeled with sample category labels; the sample images are input into a teacher model, and the sample label map of the sample image is determined based on the label map extraction network of the teacher model; the sample images are cropped to obtain cropped sample images; based on the first coordinate information of the cropped sample image in the sample image, the sample label map is cropped to obtain sample sub-label maps; thus, the sample sub-label maps can be used as additional supervision signals in the training process of the learning model; the cropped sample images are then input into a student model for category label prediction processing to obtain sample label prediction results; based on the first difference between the sample label prediction results and the sample category labels and the second difference between the sample label prediction results and the cropped sub-labels, the student model is trained to obtain an object classification model; the cropped sub-labels are obtained by performing category label prediction processing on the sample sub-label maps. This application determines the clipping sub-labels through the sample sub-label map, thereby training the student model based on the first difference between the sample label prediction result and the sample category label, and the second difference between the sample label prediction result and the clipping sub-label. In the training process of the object classification model, this application introduces the sample sub-label map as an additional supervision signal, which improves the accuracy of the object classification model.

[0103] This specification also provides an object classification method, such as... Figure 12 As shown, the above method includes: S1201: Obtain the image of the object to be identified; In the embodiments of this specification, the object to be identified and the sample object are objects in the same field. The image to be identified can be an image obtained by taking a picture of the object to be identified, or an image extracted from the video to be identified corresponding to the object to be identified. One object to be identified can correspond to one or more images to be identified.

[0104] S1203: Input the above image to be identified into the object classification model, perform category prediction processing, and obtain the target category of the above image to be identified; the above target category represents the category of the above object to be identified. The object classification model described above was trained using the same training method described above.

[0105] In the embodiments of this specification, when the object to be identified corresponds to multiple images to be identified, the category result of the object to be identified can be determined based on the category prediction results of the multiple images to be identified. For example, if any image to be identified is a defective image, the object to be identified can be determined to be an abnormal object; if multiple images to be identified are all normal images, the object to be identified can be determined to be a normal object.

[0106] In an exemplary embodiment, in a product testing scenario for industrial products, such as Figure 13 As shown, Figure 13 This is a schematic diagram of an object classification method, where the test data is the image of the object to be identified. The image is input into the object classification model (deep model 2), and the output is the probability value p of the image to be identified as a defective image, thus obtaining the corresponding target category result. The target category includes a first category and a second category. The first category represents the image to be identified as a defective image (p>0.5), and the second category represents the image to be identified as a normal image (p≤0.5).

[0107] In one exemplary embodiment, such as Figure 14-15 As shown, Figure 14 This is a prediction result for an image to be identified and its corresponding category. The image to be identified is a normal image. In addition to outputting the prediction result, it can also output the defect confidence data of 0.05. Figure 15 For another image to be identified and its corresponding category prediction result, the image to be identified is a defect image. In addition to outputting the prediction result, the defect confidence data of 0.95 can also be output.

[0108] This specification also provides a training device for an object classification model, such as... Figure 16 As shown, the above-mentioned device includes: The sample image acquisition module 1610 is used to acquire sample images of sample objects; the sample images are labeled with sample category tags. The sample label map determination module 1620 is used to input the above sample image into the teacher model and determine the sample label map of the above sample image based on the label map extraction network of the above teacher model. The sample cropping image determination module 1630 is used to perform cropping processing on the above sample image to obtain a sample cropping image; The sample sub-label image determination module 1640 is used to perform cropping processing on the sample label image based on the first coordinate information of the sample cropped image in the sample image to obtain the sample sub-label image. The sample result determination module 1650 is used to input the above-mentioned cropped sample image into the student model for category label prediction processing to obtain the sample label prediction result. The model training module 1660 is used to train the student model based on the first difference between the above sample label prediction results and the above sample category labels and the second difference between the above sample label prediction results and the cropped sub-labels, to obtain an object classification model; the above cropped sub-labels are obtained by performing category label prediction processing on the above sample sub-label map.

[0109] In one exemplary embodiment, there are multiple sample images, each corresponding to a cropped sample image, and the apparatus further includes: The label determination module is used to determine the sample category label and cropping sub-label corresponding to each cropped image; An abnormal image determination module is used to determine any sample cropped image as an abnormal image if the similarity between the sample category label corresponding to any sample cropped image and the cropping sub-label is less than a preset threshold. The sample image selection module is used to remove the abnormal images from multiple sample images to obtain the selected sample images. The sample cropping image determination module is used to determine the sample cropping image corresponding to the above-mentioned sample cropping image as the sample cropping image.

[0110] In one exemplary embodiment, the model training module includes: The model training submodule is used to train the student model based on the first difference between the sample label prediction result and the sample category label corresponding to the cropped image of the selected sample, and the second difference between the sample label prediction result and the cropping sublabel corresponding to the cropped image of the selected sample.

[0111] In one exemplary embodiment, the label determination module includes: The first determining submodule is used to determine the sample category label of the sample image corresponding to each sample cropped image; The second determining submodule is used to determine the sample category label of the sample image corresponding to each sample cropped image as the sample category label of each sample cropped image. The cropping sub-label acquisition submodule is used to determine the cropping sub-label of the sample sub-label image corresponding to each sample cropped image; The cropping sub-label determination submodule is used to determine the cropping sub-label of the sample sub-label image corresponding to each sample cropped image as the cropping sub-label of each sample cropped image.

[0112] In one exemplary embodiment, the sample sub-label map determination module includes: The first coordinate determination submodule is used to determine the first coordinate information of the above-mentioned sample cropped image in the above-mentioned sample image; The image size determination submodule is used to determine the image size corresponding to the above sample image and the above sample label image respectively; The sample sub-label image determination submodule is used to perform cropping processing on the sample label image based on the first coordinate information of the cropped sample image in the sample image, and the image size corresponding to the sample image and the sample label image respectively, to obtain the sample sub-label image.

[0113] In one exemplary embodiment, the sample sub-label map determination submodule includes: The second coordinate determination unit is used to determine the first coordinate information of the cropped sample image in the sample image as the second coordinate information of the sample sub-label image in the sample label image if the sample image and the sample label image have the same image size. The sub-label image determination unit is used to perform cropping processing on the sample label image based on the second coordinate information to obtain the sample sub-label image.

[0114] In one exemplary embodiment, the above-described apparatus further includes: The update module is used to adjust the size of the sample label image to the size of the sample image if the image sizes corresponding to the sample image and the sample label image are different, so as to obtain an updated sample label image. The second coordinate determination module is used to determine the second coordinate information of the updated sample sub-label image in the updated sample label image based on the first coordinate information of the cropped sample image in the sample image. The image determination module is used to adjust the size of the updated sample label image based on the ratio of the corresponding sizes of the updated sample sub-label image and the sample label image, so as to obtain the sample label image.

[0115] In one exemplary embodiment, the model training module includes: The first information determination submodule is used to determine the first loss information based on the first difference between the above sample label prediction results and the above sample category labels; The second information determination submodule is used to determine the second loss information based on the second difference between the above sample label prediction results and the above cropped sublabels; The target information determination submodule is used to determine the target loss information based on the first loss information and the second loss information mentioned above. The parameter adjustment submodule is used to adjust the model parameters of the student model based on the target loss information mentioned above until the training termination condition is met, and the student model at the end of training is determined as the object classification model mentioned above.

[0116] In one exemplary embodiment, the above-described apparatus further includes: The training feature map determination module is used to input the above sample images into the teacher model to be trained, and perform feature extraction processing on the above sample images based on the initial feature extraction network of the teacher model to be trained, so as to obtain the training feature map of the above sample images. The training label map determination module is used to extract label maps from the training feature maps based on the initial label map extraction network of the teacher model to be trained, and obtain training label maps. The training label prediction module is used to perform category label prediction processing on the training label map based on the initial category label prediction network of the teacher model to be trained, and to obtain the training label prediction result. The teacher model determination module is used to train the teacher model to be trained based on the difference between the training label prediction results and the sample category labels, so as to obtain the teacher model.

[0117] In one exemplary embodiment, the above-described apparatus further includes: The sample category feature determination module is used to perform global pooling on the above sample sub-label map to obtain sample category features; The normalization module is used to normalize the above sample category features to obtain sample normalized features. The cropped sub-label determination module is used to perform category label prediction processing on the normalized features of the above samples to obtain the cropped sub-labels corresponding to the above sample sub-label maps.

[0118] The apparatus and method embodiments described above are based on the same inventive concept.

[0119] This specification also provides an object classification device, such as... Figure 17 As shown, the above-mentioned device includes: The image acquisition module 1710 is used to acquire the image of the object to be identified. The target category determination module 1720 is used to input the above-mentioned image to be identified into the object classification model, perform category prediction processing, and obtain the target category of the above-mentioned image to be identified; the above-mentioned target category represents the category of the above-mentioned object to be identified. The object classification model described above was trained using the aforementioned training method.

[0120] The apparatus and method embodiments described above are based on the same inventive concept.

[0121] This specification provides an electronic device including a processor and a memory. The memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement the object classification model training method or object classification method provided in the above method embodiments.

[0122] Embodiments of this application also provide a computer storage medium, which can be disposed in a terminal to store at least one instruction or at least one program related to the training method of an object classification model in the method embodiment. The at least one instruction or at least one program is loaded and executed by the processor to implement the training method or object classification method of the object classification model provided in the above method embodiment.

[0123] Embodiments of this application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform a training method or object classification method for the object classification model provided in the above-described method embodiments.

[0124] Optionally, in the embodiments of this specification, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0125] The memory described in the embodiments of this specification can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for the functions, etc.; the data storage area may store data created based on the use of the aforementioned device, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory may also include a memory controller to provide the processor with access to the memory.

[0126] The training method for the object classification model provided in the embodiments of this specification can be executed on a mobile terminal, computer terminal, server, or similar computing device. Taking running on a server as an example, Figure 18This is a hardware structure block diagram of a server for a training method of an object classification model provided in an embodiment of this specification. For example... Figure 18 As shown, the server 1800 can vary significantly due to different configurations or performance. It may include one or more Central Processing Units (CPUs) 1810 (CPUs 1810 may include, but are not limited to, microprocessors (MCUs) or programmable logic devices (FPGAs), a memory 1830 for storing data, and one or more storage media 1820 (e.g., one or more mass storage devices) for storing application programs 1823 or data 1822. The memory 1830 and storage media 1820 may be temporary or persistent storage. The program stored in the storage media 1820 may include one or more modules, each module may include a series of instruction operations on the server. Furthermore, the CPU 1810 may be configured to communicate with the storage media 1820 and execute the series of instruction operations in the storage media 1820 on the server 1800. Server 1800 may also include one or more power supplies 1860, one or more wired or wireless network interfaces 1850, one or more input / output interfaces 1840, and / or one or more operating systems 1821, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0127] The input / output interface 1840 can be used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of server 1800. In one example, input / output interface 1840 includes a network interface controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, input / output interface 1840 may be a radio frequency (RF) module used for wireless communication with the Internet.

[0128] Those skilled in the art will understand that Figure 18 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, server 1800 may also include... Figure 18 The more or fewer components shown, or having the same Figure 18 The different configurations shown.

[0129] As can be seen from the embodiments of the training method, apparatus, device, or storage medium for the object classification model provided in this application, this application obtains sample images of sample objects; the sample images are labeled with sample category labels; the sample images are input into a teacher model, and a sample label map of the sample image is determined based on the label map extraction network of the teacher model; the sample images are cropped to obtain cropped sample images; based on the first coordinate information of the cropped sample image in the sample image, the sample label map is cropped to obtain sample sub-label maps; thus, the sample sub-label maps can be used as additional supervision signals in the training process of the learning model; the cropped sample images are then input into a student model for category label prediction processing to obtain sample label prediction results; based on the first difference between the sample label prediction results and the sample category labels and the second difference between the sample label prediction results and the cropped sub-labels, the student model is trained to obtain an object classification model; the cropped sub-labels are obtained by performing category label prediction processing on the sample sub-label maps. This application determines the clipping sub-labels through the sample sub-label map, thereby training the student model based on the first difference between the sample label prediction result and the sample category label, and the second difference between the sample label prediction result and the clipping sub-label. In the training process of the object classification model, this application introduces the sample sub-label map as an additional supervision signal, which improves the accuracy of the object classification model.

[0130] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments of this specification have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0131] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0132] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer storage medium, such as a read-only memory, a disk, or an optical disk.

[0133] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A training method for an object classification model, characterized in that, The method includes: Obtain sample images of the sample objects; the sample images are labeled with sample category tags; The sample image is input into the teacher model, and the sample label map of the sample image is determined based on the label map extraction network of the teacher model. The sample image is cropped to obtain a cropped sample image; Based on the first coordinate information of the sample cropped image in the sample image, the sample label image is cropped to obtain a sample sub-label image; The sample images are multiple, and each sample image corresponds to a sample cropped image. The sample category label and cropping sub-label corresponding to each sample cropped image are determined. If the similarity between the sample category label and the cropping sub-label of any cropped sample image is less than a preset threshold, the cropped sample image is determined to be an abnormal image. From the multiple sample images, the abnormal images are removed to obtain the filtered sample images; the cropped image corresponding to the filtered sample image is determined as the cropped image of the filtered sample. The cropped images of the selected samples are input into the student model for category label prediction processing to obtain the sample label prediction results; Based on the first difference between the sample label prediction result corresponding to the cropped image of the selected sample and the sample category label, and the second difference between the sample label prediction result corresponding to the cropped image of the selected sample and the cropped sub-label, the student model is trained to obtain an object classification model; the cropped sub-label is obtained by performing category label prediction processing on the sample sub-label image.

2. The method according to claim 1, characterized in that, The process of determining the sample category label and cropping sub-label corresponding to each cropped image includes: Determine the sample category label of the sample image corresponding to each cropped sample image; The sample category label of the sample image corresponding to each cropped sample image is determined as the sample category label of each cropped sample image; Determine the cropping sub-label of the sample sub-label image corresponding to each sample cropped image; The cropping sub-label of the sample sub-label image corresponding to each sample cropped image is determined as the cropping sub-label of each sample cropped image.

3. The method according to claim 1, characterized in that, The step of cropping the sample label image based on the first coordinate information of the cropped sample image in the sample image to obtain a sample sub-label image includes: Determine the first coordinate information of the cropped sample image within the sample image; Determine the image size corresponding to each of the sample image and the sample label image; Based on the first coordinate information of the cropped sample image in the sample image, and the image size corresponding to the sample image and the sample label image respectively, the sample label image is cropped to obtain the sample sub-label image.

4. The method according to claim 3, characterized in that, The step of cropping the sample label image based on the first coordinate information of the cropped sample image in the sample image, and the image sizes corresponding to the sample image and the sample label image respectively, to obtain the sample sub-label image, includes: If the sample image and the sample label image have the same image size, the first coordinate information of the cropped sample image in the sample image is determined as the second coordinate information of the sample sub-label image in the sample label image; Based on the second coordinate information, the sample label image is cropped to obtain the sample sub-label image.

5. The method according to claim 4, characterized in that, The method further includes: If the sample image and the sample label image have different corresponding image sizes, the size of the sample label image is adjusted to the size of the sample image to obtain an updated sample label image. Based on the first coordinate information of the cropped sample image in the sample image, determine the second coordinate information of the updated sample sub-label image in the updated sample label image; Based on the ratio of the corresponding dimensions of the updated sample sub-label image and the sample label image, the updated sample label image is resized to obtain the sample label image.

6. The method according to any one of claims 1-5, characterized in that, The student model is trained based on the first difference between the predicted sample label and the sample category label, and the second difference between the predicted sample label and the cropped sub-label, to obtain an object classification model, including: Based on the first difference between the sample label prediction result and the sample category label, the first loss information is determined; Based on the second difference between the sample label prediction result and the cropped sub-label, the second loss information is determined; Based on the first loss information and the second loss information, the target loss information is determined; Based on the target loss information, the model parameters of the student model are adjusted until the training termination condition is met, and the student model at the end of training is determined as the object classification model.

7. The method according to any one of claims 1-5, characterized in that, The training method for the teacher model includes: The sample image is input into the teacher model to be trained, and the sample image is processed by the initial feature extraction network of the teacher model to be trained to obtain the training feature map of the sample image. Based on the initial label map extraction network of the teacher model to be trained, the training feature map is processed to extract the label map to obtain the training label map; Based on the initial category label prediction network of the teacher model to be trained, the training label map is processed for category label prediction to obtain the training label prediction result; Based on the difference between the training label prediction results and the sample category labels, the teacher model to be trained is trained to obtain the teacher model.

8. The method according to any one of claims 1-5, characterized in that, The method further includes: Global pooling is performed on the sample sub-label map to obtain sample category features; The sample category features are normalized to obtain sample normalized features; The normalized features of the samples are subjected to category label prediction processing to obtain the cropped sub-labels corresponding to the sample sub-label map.

9. An object classification method, characterized in that, The method includes: Obtain the image of the object to be identified; The image to be identified is input into an object classification model for category prediction processing to obtain the target category of the image to be identified; the target category represents the category of the object to be identified. The object classification model is trained based on the training method described in any one of claims 1-8.

10. A training device for an object classification model, characterized in that, The device includes: The sample image acquisition module is used to acquire sample images of sample objects; the sample images are labeled with sample category tags. The sample label map determination module is used to input the sample image into the teacher model and determine the sample label map of the sample image based on the label map extraction network of the teacher model; The sample cropping image determination module is used to crop the sample image to obtain a sample cropping image; The sample sub-label image determination module is used to perform cropping processing on the sample label image based on the first coordinate information of the sample cropped image in the sample image to obtain a sample sub-label image; there are multiple sample images, and each sample image corresponds to a sample cropped image; The label determination module is used to determine the sample category label and cropping sub-label corresponding to each cropped image; An abnormal image determination module is used to determine any cropped image as an abnormal image if the similarity between the sample category label and the cropping sub-label corresponding to any cropped image is less than a preset threshold. The sample image selection module is used to remove abnormal images from a plurality of sample images to obtain a selected sample image. The sample cropping image determination module is used to determine the sample cropping image corresponding to the sample image as the sample cropping image. The sample result determination module is used to input the cropped image of the screened sample into the student model for category label prediction processing to obtain the sample label prediction result. The model training module is used to train the student model based on the first difference between the sample label prediction result corresponding to the cropped image of the selected sample and the sample category label, and the second difference between the sample label prediction result corresponding to the cropped image of the selected sample and the cropped sub-label, to obtain an object classification model; the cropped sub-label is obtained by performing category label prediction processing on the sample sub-label image.

11. The apparatus according to claim 10, characterized in that, The label determination module includes: The first determining submodule is used to determine the sample category label of the sample image corresponding to each sample cropped image; The second determining submodule is used to determine the sample category label of the sample image corresponding to each sample cropped image as the sample category label of each sample cropped image; The cropping sub-label acquisition submodule is used to determine the cropping sub-label of the sample sub-label image corresponding to each sample cropped image; The cropping sub-label determination submodule is used to determine the cropping sub-label of the sample sub-label image corresponding to each sample cropped image as the cropping sub-label of each sample cropped image.

12. The apparatus according to claim 10, characterized in that, The sample sub-label map determination module includes: The first coordinate determination submodule is used to determine the first coordinate information of the sample cropped image in the sample image; The image size determination submodule is used to determine the image size corresponding to the sample image and the sample label image, respectively. The sample sub-label image determination submodule is used to perform cropping processing on the sample label image based on the first coordinate information of the sample cropped image in the sample image, and the image size corresponding to the sample image and the sample label image respectively, to obtain the sample sub-label image.

13. The apparatus according to claim 12, characterized in that, The sample sub-label map determination submodule includes: The second coordinate determination unit is used to determine the first coordinate information of the cropped sample image in the sample image as the second coordinate information of the sample sub-label image in the sample label image if the sample image and the sample label image have the same image size. The sub-label image determination unit is used to perform cropping processing on the sample label image based on the second coordinate information to obtain the sample sub-label image.

14. The apparatus according to claim 13, characterized in that, The device further includes: The update module is used to adjust the size of the sample label image to the size of the sample image if the sample image and the sample label image have different corresponding image sizes, thereby obtaining an updated sample label image; The second coordinate determination module is used to determine the second coordinate information of the updated sample sub-label image in the updated sample label image based on the first coordinate information of the sample cropped image in the sample image; The image determination module is used to adjust the size of the updated sample label image based on the ratio of the corresponding sizes of the updated sample sub-label image and the sample label image, so as to obtain the sample label image.

15. The apparatus according to any one of claims 10-14, characterized in that, The model training module includes: The first information determination submodule is used to determine first loss information based on the first difference between the sample label prediction result and the sample category label; The second information determination submodule is used to determine second loss information based on the second difference between the sample label prediction result and the cropped sublabel; The target information determination submodule is used to determine target loss information based on the first loss information and the second loss information; The parameter adjustment submodule is used to adjust the model parameters of the student model based on the target loss information until the training termination condition is met, and to determine the student model at the end of training as the object classification model.

16. The apparatus according to any one of claims 10-14, characterized in that, The device further includes: The training feature map determination module is used to input the sample image into the teacher model to be trained, and perform feature extraction processing on the sample image based on the initial feature extraction network of the teacher model to be trained to obtain the training feature map of the sample image. The training label map determination module is used to perform label map extraction processing on the training feature map based on the initial label map extraction network of the teacher model to be trained, so as to obtain the training label map. The training label prediction module is used to perform category label prediction processing on the training label map based on the initial category label prediction network of the teacher model to be trained, so as to obtain the training label prediction result. The teacher model determination module is used to train the teacher model to be trained based on the difference between the training label prediction result and the sample category label, so as to obtain the teacher model.

17. The apparatus according to any one of claims 10-14, characterized in that, The device further includes: The sample category feature determination module is used to perform global pooling on the sample sub-label map to obtain sample category features; The normalization processing module is used to normalize the sample category features to obtain sample normalized features; The cropped sub-label determination module is used to perform category label prediction processing on the normalized features of the sample to obtain the cropped sub-labels corresponding to the sample sub-label map.

18. An object classification device, characterized in that, The device includes: The image acquisition module is used to acquire the image of the object to be identified. The target category determination module is used to input the image to be identified into an object classification model, perform category prediction processing, and obtain the target category of the image to be identified; the target category represents the category of the object to be identified. The object classification model is trained based on the training method described in any one of claims 1-8.

19. An electronic device, characterized in that, The device includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, the at least one instruction or at least one program being loaded and executed by the processor to implement the training method of the object classification model as described in any one of claims 1-8 or the object classification method as described in claim 9.

20. A computer storage medium, characterized in that, The computer storage medium stores at least one instruction or at least one program, which is loaded and executed by a processor to implement the training method of the object classification model as described in any one of claims 1-8 or the object classification method as described in claim 9.

21. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the training method of the object classification model according to any one of claims 1-8 or the object classification method according to claim 9.

Images