Method for determining object classification, related device and computer program product

Abstract

The invention provides a method for determining object classification, a related device and a computer program product, and the method comprises the steps: processing a to-be-classified object through employing a first initial classification model, and obtaining a first initial classification result and a first confidence coefficient for the first initial classification result; in response to the fact that the first confidence coefficient is larger than or equal to a first confidence coefficient threshold value, a second initial classification model is used for processing the to-be-classified object, a second initial classification result and a second confidence coefficient for the second initial classification result are obtained, and the first initial classification model and the second initial classification model are obtained through distillation based on the same target classification model; and taking the target classification as a classification result of the to-be-classified object in response to the fact that both the first initial classification result and the second initial classification result indicate the target classification and the second confidence coefficient is greater than or equal to a second confidence coefficient threshold value. Therefore, the classification models are deployed in a distributed manner, so that the deployment, configuration requirements and difficulty of the classification models are reduced, and the classification models can be deployed more conveniently and flexibly.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and device for determining object classification, an electronic device, a computer-readable medium, and a computer program product. Background Art

[0002] With the development of society, computer technology is also improving day by day. In computer technology, the Internet can provide a platform for users to communicate, so that users can communicate, exchange and share content with other users by posting videos and information on the Internet. For example, users can achieve "online communication" with other users by posting information on the "platform".

[0003] Accordingly, in order to facilitate users to obtain content and protect their interests, the "platform" usually expects to classify user content in advance so that users can obtain user content more efficiently and safely. For example, the "platform" usually chooses to use classification models to process content to determine the category of the content. Therefore, how to better and more effectively use classification models and deploy classification models more conveniently and flexibly is worthy of attention and urgent need. Summary of the invention

[0004] Multiple aspects of the present application provide a method, apparatus, electronic device, computer-readable storage medium, and computer program product for determining object classification, which selects to distill a classification model into a model with a smaller scale than its own model, and then uses a model group composed of these smaller models to similarly implement the functions of the classification model. Thus, on the basis of ensuring the "utility" of the classification model, by deploying the classification model in a distributed manner, the deployment and configuration requirements and difficulty of the classification model are reduced, so that the classification model can be deployed more lightly and flexibly.

[0005] In one aspect of the present application, a method for determining object classification is provided, comprising: processing an object to be classified using a first initial classification model to obtain a first initial classification result and a first confidence level for the first initial classification result; in response to the first confidence level being greater than or equal to a first confidence level threshold, processing the object to be classified using a second initial classification model to obtain a second initial classification result and a second confidence level for the second initial classification result, wherein the first initial classification model and the second initial classification model are obtained by distillation based on the same target classification model; in response to the first initial classification result and the second initial classification result both indicating target classification, and the second confidence level being greater than or equal to the second confidence level threshold, taking the target classification as the classification result of the object to be classified.

[0006] In another aspect of the present application, there is provided an apparatus for determining object classification, comprising: a first initial result and confidence determination module, configured to process an object to be classified using a first initial classification model to obtain a first initial classification result and a first confidence for the first initial classification result; a second initial result and confidence determination module, configured to process the object to be classified using at least a second initial classification model in response to the first confidence being greater than or equal to a first confidence threshold, to obtain a second initial classification result and a second confidence for the second initial classification result, wherein the first initial classification model and the second initial classification model are obtained by distillation based on the same target classification model; and a first classification result determination module, configured to take the target classification as the classification result of the object to be classified in response to the first initial classification result and the second initial classification result both indicating target classification and the second confidence being greater than or equal to the second confidence threshold.

[0007] Another aspect of the present application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method for determining object classification as provided above.

[0008] In another aspect of the present application, a computer-readable storage medium is provided, on which computer program instructions are stored. The computer program instructions can be executed by a processor to implement the method for determining object classification as provided above.

[0009] In another aspect of the present application, a computer program product includes a computer program having computer program instructions stored thereon. When the computer program is executed by a processor, the method for determining the classification of an object as provided above can be implemented.

[0010] In the solution provided by the embodiment of the present application, the first initial classification model is used to process the object to be classified, and the first initial classification result and the first confidence for the first initial classification result are obtained; in response to the first confidence being greater than or equal to the first confidence threshold, the second initial classification model is used to process the object to be classified, and the second initial classification result and the second confidence for the second initial classification result are obtained, wherein the first initial classification model and the second initial classification model are obtained by distillation based on the same target classification model; in response to the first initial classification result and the second initial classification result both indicating the target classification, and the second confidence being greater than or equal to the second confidence threshold, the target classification is used as the classification result of the object to be classified. Thus, by deploying the classification model in a distributed manner, the deployment, configuration requirements and difficulty of the classification model are reduced, so that the classification model can be deployed more conveniently and flexibly. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0012] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0013] Figure 1 A schematic diagram of a process for determining object classification provided by an embodiment of the present application;

[0014] Figure 2 A schematic diagram of an architecture for distilling an initial classification model provided in an embodiment of the present application;

[0015] Figure 3 A schematic diagram of an architecture for obtaining classification results provided in an embodiment of the present application;

[0016] Figure 4 A schematic diagram of an architecture for obtaining classification results provided by another embodiment of the present application;

[0017] Figure 5 A schematic diagram of an architecture for obtaining classification results provided in yet another embodiment of the present application;

[0018] Figure 6 A schematic diagram of the structure of an apparatus for determining object classification provided in one embodiment of the present application;

[0019] Figure 7 The present invention is a schematic diagram of the structure of an electronic device suitable for implementing the solution in the embodiment of the present application.

[0020] The same or similar reference numerals in the drawings represent the same or similar components. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0022] In a typical configuration of the present application, the terminal and the equipment of the service network each include one or more processors (CPU), input / output interface, network interface and memory.

[0023] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0024] Computer readable media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer program instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0025] As discussed above, how to better and more effectively utilize classification models and deploy classification models more conveniently and flexibly is worthy of attention and urgent need.

[0026] In some solutions, for the classification model used to implement object classification, the performance of the classification model is improved by expanding the model size and complicating its model structure as much as possible, thereby hoping to use a more powerful classification model to implement object classification work more efficiently and with higher quality.

[0027] However, in this way, as the scale of the classification model increases and the structure becomes more complex, the deployment cost and difficulty of the classification model rise sharply. For example, it may be necessary to use a large number of graphics processing units (GPUs) with very strong computing resources, computing power, and cutting-edge performance to achieve the deployment of the classification model. If the deployment is not carried out in this way, it will lead to the process of using the classification model to classify objects. Due to the poor deployment effect of the classification model or the failure to be actually deployed, the ability to classify the objects is affected or lost, and the classification of the objects cannot be achieved, or the classification effect is not ideal.

[0028] In this regard, an embodiment of the present application provides a method for determining object classification, which uses a first initial classification model to process an object to be classified, obtains a first initial classification result and a first confidence for the first initial classification result; in response to the first confidence being greater than or equal to a first confidence threshold, uses a second initial classification model to process the object to be classified, obtains a second initial classification result and a second confidence for the second initial classification result, wherein the first initial classification model and the second initial classification model are obtained by distillation based on the same target classification model; in response to the first initial classification result and the second initial classification result both indicating the target classification, and the second confidence being greater than or equal to the second confidence threshold, uses the target classification as the classification result of the object to be classified. Thus, by deploying the classification model in a distributed manner, the deployment, configuration requirements and difficulty of the classification model are reduced, so that the classification model can be deployed more conveniently and flexibly.

[0029] This method distills the classification model into an initial classification model with a smaller model size than itself, and then uses the "initial classification model group" composed of these initial classification models with a smaller model size to similarly implement the functions of the classification model. Therefore, on the basis of ensuring the "utility" of the classification model, by distributing the initial classification models to achieve distributed deployment of the classification model, the deployment and configuration requirements and difficulty of the classification model are reduced, so that the classification model can be deployed more lightly and flexibly.

[0030] In actual scenarios, the execution subject of the method can be a user device, or a device formed by integrating a user device and a network device through a network, or an application running on the above device. The user device includes but is not limited to various terminal devices such as computers, mobile phones, tablet computers, smart watches, and bracelets. The network device includes but is not limited to network hosts, single network servers, multiple network server sets, or cloud computing-based computer sets. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing (Cloud Computing), where cloud computing is a type of distributed computing, a virtual computer composed of a group of loosely coupled computer sets.

[0031] When the execution subject is software, it can be installed in the electronic devices listed above. It can be implemented as multiple software or software modules, or as a single software or software module, and no specific limitation is made here.

[0032] Figure 1 A process 100 for determining an object classification provided by an embodiment of the present application is shown. The process 100 includes at least the following processing steps:

[0033] (Step) S101, using a first initial classification model to process an object to be classified, to obtain a first initial classification result and a first confidence level for the first initial classification result.

[0034] In an embodiment of the present application, the execution subject (for example, the server used by the above-mentioned "platform") can obtain the object to be classified based on the communication with the terminal device used by the user. For example, the object to be classified can be user content such as images, videos, audios, and texts uploaded by the user.

[0035] It should be understood that in some scenarios, the object to be classified can also be maintained in advance in the local storage of the execution subject. For example, the object to be classified can be the object indicated by the previously retained task to be classified, and the object to be classified has been maintained in advance in the local storage of the execution subject. Accordingly, in such a case, the execution subject can directly obtain the object to be classified from the local storage in a reading manner.

[0036] In an embodiment of the present application, the execution entity can use the first initial classification model to process the object to be classified to obtain an initial classification result (for the convenience of description, it is referred to as the "first initial classification result") and a confidence level for the first initial classification result (for the convenience of description, it is referred to as the "first confidence level").

[0037] As discussed above, the first initial classification model may be distilled from a target classification model, or in other words, the first initial classification model may be obtained based on the distillation of the target classification model.

[0038] For example, you can choose to use the target classification model to process the sample object to be classified, and after obtaining the corresponding classification result, use the sample object to be classified as input and the classification result as output to train the untrained first initial classification model to achieve "distillation".

[0039] In some embodiments, the target classification model may be the pre-trained "ideal" classification model discussed above that meets the performance and scale requirements, for example, the most preferred classification model with the strongest classification performance. For example, the target classification model is a large language model (LLM for short).

[0040] For example, in an object classification scenario, the target classification model may be a classification model that has satisfactory classification performance for a "multimodal data object" that has multiple modal data types at the same time. In practice, a "modal data type" may be, for example, text, image, audio, etc. Accordingly, a "multimodal data object" may actually be a text-image data object, a text-audio data object, an image-audio data object, a text-image-audio data object, etc.

[0041] In some embodiments, for the "distillation" process, for example, the target classification model can be distilled based on Logit Distillation to obtain, for example, the first initial classification model.

[0042] For example, the target classification model can be used to process sample objects to be classified, and the logical output (i.e., logit) involved in the processing and the reference confidence level of the logical output (used to indicate the "degree of trust" in the output result) are provided to an untrained (first) initial classification model to train the untrained initial classification model so as to "distill" the capabilities of the target classification model to the trained initial classification model.

[0043] Therefore, by using a distillation method such as Logit distillation, the output probability distribution of the target classification model (i.e., logits) can be converted into the training target of the initial classification model, so that the initial classification model after distillation can maintain a high accuracy.

[0044] The reference confidence may be provided by, for example, a classification head configured in the target classification model.

[0045] The classification head can be used to determine the "classification loss" (or classification loss), and then determine the confidence of the output results of the corresponding part in the model. The classification head can be understood as the part used to map the learned feature representation to the category prediction. The classification head can usually include one or more fully connected layers to map features to the category space, and an activation function (such as Softmax) to generate the probability distribution of "whether the classification result is reliable" and determine the corresponding confidence.

[0046] In some embodiments, the "confidence" can also be specifically expressed in the form of a "score". For example, multiple "probability value intervals" can be divided based on the difference in "probability". Then, by assigning values ​​to these "probability value" intervals (for example, assigning corresponding scores), a score embodied as "confidence" can be obtained. Therefore, in this way, the "probability" can be rounded to reduce the difficulty of determining the confidence threshold later.

[0047] For another example, the value of the "specific probability" of the corresponding classification result (eg, the first initial classification result) in the "probability distribution" may be directly used as the "score".

[0048] Accordingly, the execution entity may call the first initial classification model to process the object to be classified, so as to obtain a first initial classification result and a first confidence level for the first initial classification result.

[0049] Similarly, the first initial classification model may also be configured with a classification head to generate a first confidence level for the first initial classification result.

[0050] In this way, the classification head can be directly used to supervise the "classification process" of the first initial classification model, and the corresponding first confidence level can be directly generated based on the supervision result of the "classification process". As a result, the "classification process" of the first initial classification model can be directly evaluated while simplifying the overall system structure configuration, thereby improving the reliability of the generation of the first confidence level.

[0051] S102: In response to the first confidence being greater than or equal to a first confidence threshold, at least a second initial classification model is used to process the object to be classified to obtain a second initial classification result and a second confidence for the second initial classification result.

[0052] In an embodiment of the present application, if the execution entity determines that the first confidence level is greater than or equal to a first confidence threshold (for example, the confidence threshold level may be pre-set based on the confidence level corresponding to the result being considered "reliable and trustworthy"), the execution entity may respond to this by at least choosing to use a second initial classification model to process the object to be classified, and obtain a corresponding initial classification result (for the convenience of description, referred to as the "second initial classification result") and a confidence level for the second initial classification result (for the convenience of description, referred to as the "second confidence level").

[0053] The second initial classification model can also be obtained by distilling the target classification model from which the first initial classification model is distilled. In other words, the first initial classification model and the second initial classification model are obtained by distilling based on the same target classification model.

[0054] For example, the second initial classification model can be based on the method of distilling the first initial model discussed above, using the same sample objects to be classified, the same logical outputs produced by the target classification model in processing the sample objects to be classified, and the reference confidence of the corresponding logical outputs, and also based on the Logit method to distill the target classification model.

[0055] It should be understood that the first initial classification model, the second initial classification model, etc. can actually be built based on the same model architecture. For example, each initial classification model can be built based on a deep residual network (ResNet) architecture.

[0056] In some embodiments, in combination with different deployment requirements, the architectures of the initial classification models may also differ, and this application is not intended to be limiting in this regard.

[0057] Generally, similar to the first initial classification model, the second initial classification model can also directly generate the second confidence by configuring a classification head in the second initial classification model.

[0058] S103: In response to the first initial classification result and the second initial classification result both indicating target classification, and the second confidence level is greater than or equal to a second confidence level threshold, taking the target classification as the classification result of the object to be classified.

[0059] In an embodiment of the present application, after obtaining the first initial classification result and the first confidence based on S101 and obtaining the second initial classification result and the second confidence based on S102, if the first initial classification result and the second initial classification result both indicate the same target classification (for example, both point to classification A), and the second confidence is greater than or equal to a second confidence threshold (which can also be pre-set based on the confidence corresponding to the result being considered "reliable and trustworthy"), the executing entity can choose to use the target classification (i.e., classification A) as the classification result of the object to be classified.

[0060] It should be understood that the second confidence threshold is similar to the first confidence threshold in terms of setting and determination, but may be different from the first confidence threshold in terms of value, and the present application is not intended to be limiting in this regard.

[0061] Therefore, the execution entity can jointly use the first initial classification model and the second initial classification model in a "cascading" manner, and overcome the problem that the initial classification model obtained after distillation may be difficult to achieve all the performance and capabilities of the target classification model alone through a "joint strategy", so as to closely and equally realize the capabilities of the "target classification model".

[0062] In some embodiments, even if the first initial classification model and the second initial classification model may have the same architecture, there may be differences in the scale of the models. Therefore, the first initial classification model and the second initial classification model may be selected and configured on different graphics processors accordingly. For example, after respectively determining the graphics processors that can meet the requirements of the first initial classification model and the second initial classification model, the first initial classification model and the second initial classification model may be deployed on different graphics processors accordingly.

[0063] Generally, based on the scale and performance requirements of the initial classification models, the performance of the graphics processors that actually deploy the various initial classification models may also differ. For example, in a cascaded manner, the performance of the graphics processor corresponding to the first initial classification model may be lower than that of the graphics processor of the second initial classification model, thereby enabling the "coarse and fine" two-level classification to be implemented based on the first initial classification model and the second initial classification model, and enabling more flexible "deployment" and balancing the overall accuracy and recall of the initial classification model group composed of the initial classification models.

[0064] For example, in the process of deployment in the form of an "initial classification model group", it is not required to provide two graphics processors that can meet the requirements of the second initial classification model at the same time to complete the overall deployment. When only one processor that meets the second initial classification model is provided, and a graphics processor with weaker capabilities but that can meet the requirements of the first initial classification model is provided, the initial classification model group can still complete the overall deployment and can still provide a close effect of deploying the "target classification model".

[0065] In some embodiments, the model scale of the first initial classification model may be smaller than that of the second initial classification model. For example, the first initial classification model may be ResNet18, and the second initial classification model may be ResNet50.

[0066] In other words, for the two initial classification models distilled from the same target classification, the one with a smaller model size can be used as the first initial classification model for "rough screening", that is, the first initial classification model is used to process the object to be classified before the second initial classification model, and when the first confidence is greater than or equal to the first confidence threshold, the second initial classification model is continued to be used (at least) to process the object to be classified.

[0067] For easier understanding, you can also refer to the following Figure 2-4 The exemplary architecture shown is used to discuss the method of "distilling the initial classification model" and the method of jointly utilizing at least two initial classification models.

[0068] First, for the "distillation" process, you can refer to Figure 2 . Figure 2 A schematic diagram of an architecture 200 for distilling an initial classification model provided in an embodiment of the present application.

[0069] In architecture 200, the target classification model 210 can provide the logical output 215 and reference confidence 217 obtained in the process of processing the input sample object 220 to each untrained initial classification model in the initial classification model group 230, for example, to the untrained initial classification models 231 and 233 (exemplarily, the model scale of the untrained initial classification model 231 is smaller than that of the untrained initial model 233).

[0070] Accordingly, by using the logic output 215 and the reference confidence 217 , the capability of the target classification model 210 can be distilled to the untrained initial classification models 231 and 233 , so as to train and distill the initial classification models 231 ′ and 233 ′.

[0071] Accordingly, at this time, the initial classification model group 230 can also be exemplarily updated to the initial classification model group 230', wherein the initial classification model group 230' can include the distilled, trained initial classification models 231' and 233'. Accordingly, the model scale of the initial classification model 231' is also smaller than that of the initial classification model 233'.

[0072] Accordingly, the initial classification model 231 ′ and the initial classification model 233 ′ may be deployed in two different graphics processors (not shown in the figure).

[0073] Next, for ease of understanding, the process of determining the classification result based on the "cascade" of the first initial classification model and the second initial classification model can be further referred to. Figure 3 .

[0074] Figure 3 A schematic diagram of an architecture 300 for obtaining classification results provided in an embodiment of the present application. For ease of understanding, the architecture 300 can be further implemented based on, for example, an initial classification model distilled from the architecture 200 of the above example. The architecture 300 is specifically as follows:

[0075] The execution subject first uses the initial classification model 231 ′ to process the object to be classified 305 to obtain a corresponding first initial classification result 310 and a first confidence 315 .

[0076] Exemplarily, the first confidence 315 is greater than or equal to the first confidence threshold. Accordingly, the execution subject can respond to this and use the initial classification model 233 ′ to process the object to be classified 305 to obtain the corresponding second initial classification result 320 and the second confidence 325 .

[0077] Exemplarily, the first initial classification result 310 and the second initial classification result 320 both indicate the (same) target classification, and the second confidence 325 is greater than or equal to the second confidence threshold. In response, the execution subject can respond to this and determine (the classification result of) the object to be classified 305 as the target classification.

[0078] Exemplarily, the first initial classification model 231 ′ may include a classification head 317 for generating a first confidence 315 , and the second initial classification model 233 ′ may also include a classification head 319 for generating a second confidence 325 .

[0079] In some embodiments, the "joint strategy" can actually be implemented in the form of "joint voting".

[0080] That is, for the first initial classification model and the second initial classification model, if the first initial classification result and the second initial classification result both indicate the target classification, and the sum of the first confidence and the second confidence is greater than or equal to the third confidence threshold, the execution subject can choose to use the target classification as the classification result of the object to be classified. For example, in such a case, some objects to be classified that the second initial classification model considers to be unable to meet the "reliable and trustworthy" standard (for example, the second confidence is greater than or equal to the second confidence threshold), but the first initial classification model "extremely determines" that they can reach the "reliable and trustworthy" standard, can be classified. In this way, the ability to classify "objects to be classified" is enhanced.

[0081] For the third confidence threshold, it can also be similarly considered that the confidence corresponding to the result “reliable and credible” is preset and configured.

[0082] Next, for ease of understanding, the process of determining the classification result based on the "joint voting" of the first initial classification model and the second initial classification model can be further referred to Figure 4 .

[0083] Figure 4 A schematic diagram of an architecture 400 for obtaining classification results provided by another embodiment of the present application. For ease of understanding, the architecture 400 can be further implemented based on, for example, the initial classification model distilled by the architecture 200 of the above example. The architecture 400 is specifically as follows:

[0084] For the object to be classified 405, the execution entity first uses the initial classification model 231' to process the object to be classified 405 to obtain the corresponding first initial classification result 410 and the first confidence 415. At the same time, the execution entity uses the classification model 233' to process the object to be classified 405 to obtain the corresponding second initial classification result 420 and the second confidence 425.

[0085] Exemplarily, the first initial classification result 410 and the second initial classification result 420 both indicate the (same) target classification. Accordingly, the execution subject may choose to add the first confidence 415 and the second confidence 425 in response to this.

[0086] Exemplarily, the sum of the first confidence 415 and the second confidence 425 is greater than or equal to the third confidence threshold. Accordingly, the execution subject may respond to this and select to determine that the object to be classified 405 is determined as the target classification.

[0087] In some embodiments, the above-mentioned "cascade" and "joint voting" methods can also be used simultaneously to use a combined strategy that has the advantages of both strategies.

[0088] For example, in the process of distilling the target classification model (for example, distilling the first initial classification model and the second initial classification model), a third initial classification model can also be distilled. Similarly, the third initial classification model can process the object to be classified to obtain a corresponding third initial classification result and a third confidence level for the third initial classification result.

[0089] Usually, the model scale of the third initial classification model can be larger than the first initial classification model and smaller than or equal to the second initial classification model, so that the "first initial classification model" can still be used as the "rough processing" link in the cascade. Then, the execution subject determines whether to use the subsequent second initial classification model and the third initial classification model to perform "fine processing" and realize "cascade" according to the first confidence of the first initial classification model for the output first initial classification result.

[0090] For example, in the process of "detailed processing", the second initial classification model and the third initial classification model can determine the classification result of the object to be classified through the above-mentioned "joint voting".

[0091] Therefore, by combining the two strategies of "cascading" and "joint voting", the classification accuracy and recall rate are improved to provide processing performance that is closer to the target classification model.

[0092] For example, we can first refer to Figure 2 .exist Figure 2 In the illustrated architecture 200, the initial classification model group 230 may also include an untrained initial classification model 235. Accordingly, after “distillation” using the target classification model 210, the untrained initial classification model 235 may be correspondingly trained and distilled into an initial classification model 235′ in the initial classification model group 230′.

[0093] Similar to the above discussion, the third initial classification model may also be deployed in a graphics processor different from the first initial classification model and the second initial classification model, similar to the deployment of the first initial classification model and the second initial classification model.

[0094] In some embodiments, when there is a third initial classification model, the third initial classification model may also include a classification head for generating a third confidence level.

[0095] Accordingly, for the combined “cascade” and “joint voting” methods discussed above, the execution subject may first choose to use the first initial classification model to process the object to be classified to obtain a first initial classification result and a first confidence level.

[0096] If the first confidence is greater than or equal to the first confidence threshold, the execution entity may, in addition to using the second initial classification model to process the object to be classified to obtain a second initial classification result and a second confidence for the second initial classification result as discussed above, also simultaneously use the third initial classification model to process the object to be classified to obtain a third initial classification result and a third confidence for the third initial classification result.

[0097] That is, in the process of executing S102, the executing entity may choose to "use the second initial classification model to process the object to be classified, obtain a second initial classification result and a second confidence level for the second initial classification result, and use the third initial classification model to process the object to be classified, obtain a third initial classification result and a third confidence level for the third initial classification result" as an alternative or substitute.

[0098] Next, if the first initial classification result, the second initial classification result and the third initial classification result all indicate target classification, and the sum of the second confidence and the third confidence is greater than or equal to the fourth confidence threshold, the execution entity can respond to this and use the target classification as the classification result of the object to be classified.

[0099] Thus, after the first initial classification model, the “joint voting” of the second initial classification model and the third initial classification model is achieved.

[0100] For easier understanding, you can also refer to Figure 5 . Figure 5 A schematic diagram of an architecture 500 for obtaining classification results provided in another embodiment of the present application. The architecture 500 can be further implemented based on, for example, the initial classification model distilled from the architecture 200 of the above example. The architecture 500 is specifically as follows:

[0101] The execution subject first uses the initial classification model 231 ′ to process the object to be classified 505 to obtain a corresponding first initial classification result 510 and a first confidence level 515 .

[0102] Then, if the first confidence 515 is greater than or equal to the first confidence threshold, the execution entity continues to use the initial classification models 233' and 235' to process the object to be classified 305, and obtains the second initial classification result 520 and the second confidence 525 corresponding to the initial classification model 233', and the third initial classification result 530 and the third confidence 535 corresponding to the initial classification model 235'.

[0103] Exemplarily, the first initial classification result 510, the second initial classification result 520, and the third initial classification result 530 all indicate the (same) target classification, and the sum of the second confidence 525 and the third confidence 535 is greater than or equal to the fourth confidence threshold. Accordingly, the execution subject can respond to this and determine that the object to be classified 505 is the target classification.

[0104] Then, the method for determining object classification provided by the present application uses a first initial classification model to process the object to be classified, and obtains a first initial classification result and a first confidence for the first initial classification result; in response to the first confidence being greater than or equal to the first confidence threshold, uses a second initial classification model to process the object to be classified, and obtains a second initial classification result and a second confidence for the second initial classification result, wherein the first initial classification model and the second initial classification model are obtained by distillation based on the same target classification model; in response to the first initial classification result and the second initial classification result both indicating the target classification, and the second confidence being greater than or equal to the second confidence threshold, the target classification is used as the classification result of the object to be classified. Thus, by deploying the classification model in a distributed manner, the deployment, configuration requirements and difficulty of the classification model are reduced, so that the classification model can be deployed more conveniently and flexibly.

[0105] In some embodiments, considering the joint use of multiple initial classification models (or, "initial classification model group"), it may not be possible to ultimately achieve performance that is completely consistent with the "target classification model".

[0106] Therefore, the "initial classification model group" can be selected as a "backup" or "redundancy" of the target classification model. In other words, such an initial classification model group can be maintained in parallel with the target classification model.

[0107] In this way, the resource consumption of the target classification model can be reduced while saving the graphics processors required by the target classification model. For example, if the target classification model is limited in capacity due to insufficient graphics processors of the target model, or if it is unable to handle a "large amount" of work at the same time, the "initial classification model group" can be deployed at the same time to use the "initial classification model group" to undertake part of the work originally assigned to the target classification model.

[0108] Accordingly, when receiving the object to be classified, the execution subject may also choose to first estimate the amount of resources required by the target classification model to process the object to be classified, and obtain a resource estimation result.

[0109] If the resource estimation result is less than the resource amount that the target classification model can currently provide, the execution subject can accordingly use the initial classification model group to process the objects to be classified and obtain classification results to alleviate the processing pressure of the target classification model.

[0110] Therefore, by combining the initial classification model group and the target classification model, the deployment flexibility requirements and processing capacity requirements can be taken into account.

[0111] In some embodiments, when a target classification model and an initial classification model group are deployed at the same time, the execution entity can also determine whether to use the initial classification model group or the target classification model to process the object to be classified based on indicators such as the classification accuracy requirements and processing efficiency of the object to be classified.

[0112] For example, for classification tasks with classification accuracy requirements higher than the target standard, the object to be classified is provided to the target classification model for more accurate and quality classification. For classification tasks with classification accuracy requirements equal to or lower than the target standard, the initial classification model group is selected to process the object to be classified.

[0113] In some embodiments, the scales of the classification models distilled from the same target classification model may be comparable. In such a case, the execution entity may use any one of them as the first initial classification model for “rough screening”.

[0114] In such a case, in some embodiments, for the "second initial classification model", if the first confidence is not greater than or equal to the first confidence threshold, the execution subject may also choose to use the second initial classification model to process the object to be classified. For example, directly choose to temporarily "ignore" the numerical value of the first confidence, and directly determine the second confidence threshold after determining the first confidence, or at the same time as the first confidence. Alternatively, although the first confidence is referred to, when it is determined that the first confidence is greater than or equal to another confidence threshold lower than the first confidence threshold, choose to use the second initial classification model to process the object to be classified.

[0115] Then, the execution entity can judge whether the sum of the first confidence level and the second confidence level is greater than or equal to the third confidence threshold (it should be understood that the value of the third confidence threshold in this case should be less than the sum of the first confidence threshold and the second confidence threshold) by, for example, the above-mentioned "voting" method, to determine whether to use the target classification pointed to by both as the classification result of the object to be classified.

[0116] Thus, in the case where, for example, the "individual capabilities" of two models of the same scale (for example, such a scale may be the maximum scale allowed for deployment) may not independently meet the "reliable and trustworthy" processing capability requirements, "classification" is performed by combining the processing results of the two initial classification models. Thus, the applicable scenarios of the process of determining object classification are improved, so as to enhance the ability to determine object classification.

[0117] The present application also provides a device for determining object classification, the structure of which is as follows: Figure 6 The device 600 shown. The device 600 includes: a first initial result and confidence determination module 610, configured to process the object to be classified using the first initial classification model to obtain a first initial classification result and a first confidence for the first initial classification result; a second initial result and confidence determination module 620, configured to process the object to be classified using at least the second initial classification model in response to the first confidence being greater than or equal to the first confidence threshold, to obtain a second initial classification result and a second confidence for the second initial classification result, wherein the first initial classification model and the second initial classification model are obtained by distillation based on the same target classification model; a first classification result determination module 630, configured to take the target classification as the classification result of the object to be classified in response to the first initial classification result and the second initial classification result both indicating the target classification and the second confidence being greater than or equal to the second confidence threshold.

[0118] In some embodiments, the device 600 also includes: a second classification result determination module, configured to, in response to the first initial classification result and the second initial classification result both indicating target classification, and the sum of the first confidence level and the second confidence level is greater than or equal to a third confidence level threshold, use the target classification as the classification result of the object to be classified.

[0119] In some embodiments, it also includes a third initial classification model distilled from the target model, the model scale of the third initial classification model is larger than the first initial classification model and less than or equal to the second initial classification model, the second initial result and confidence determination module 620 is further configured to use the second initial classification model to process the object to be classified to obtain the second initial classification result and the second confidence for the second initial classification result, and use the third initial classification model to process the object to be classified to obtain the third initial classification result and the third confidence for the third initial classification result; the third classification result determination module is configured to, in response to the first initial classification result, the second initial classification result and the third initial classification result all indicating the target classification, and the sum of the second confidence and the third confidence is greater than or equal to the fourth confidence threshold, take the target classification as the classification result of the object to be classified.

[0120] In some embodiments, the first initial classification model, the second initial classification model, and the third initial classification model are independently deployed on different graphics processors.

[0121] In some embodiments, the first initial classification model includes a first classification head for generating a first confidence, the second initial classification model includes a second classification head for generating a second confidence; and in response to also including a third initial classification model, the third initial classification model includes a third classification head for generating a third confidence.

[0122] In some embodiments, the first initial classification model and the second initial classification model are obtained by distilling the logical outputs and reference confidences of the corresponding logical outputs produced by the target classification model in the process of processing the sample objects to be classified using the sample objects to be classified.

[0123] In some embodiments, it also includes: a resource quantity estimation module, which is configured to estimate the resource quantity required by the target classification model to process the object to be classified, and obtain a resource quantity estimation result; and a first initial result and confidence determination module, which is further configured to, in response to the resource quantity estimation result being less than the resource quantity that the target classification model can currently provide, use the first initial classification model to process the model to be classified, and obtain a first initial classification result and a first confidence level for the first initial classification result.

[0124] Based on the same inventive concept, an electronic device, a readable storage medium and a computer program product are also provided in an embodiment of the present application. The method corresponding to the electronic device may be the method for determining the classification of an object in the aforementioned embodiment, and its principle of solving the problem is similar to that of the method. The electronic device provided in an embodiment of the present application includes: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the methods and / or technical solutions of the aforementioned multiple embodiments of the present application.

[0125] The electronic device may be a user device, or a device formed by integrating a user device and a network device through a network, or an application running on the above device. The user device includes but is not limited to various terminal devices such as computers, mobile phones, tablet computers, smart watches, and bracelets. The network device includes but is not limited to network hosts, single network servers, multiple network server sets, or cloud computing-based computer sets, which can be used to implement some processing functions when setting an alarm. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing, where cloud computing is a type of distributed computing, a virtual computer composed of a group of loosely coupled computer sets.

[0126] Figure 7 The structure of an electronic device suitable for implementing the method and / or technical solution in the embodiment of the present application is shown, and the electronic device 700 includes a central processing unit (CPU, Central Processing Unit) 701, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM, Read Only Memory) 702 or the program loaded from the storage part 708 to the random access memory (RAM, Random Access Memory) 703. In RAM 703, various programs and data required for system operation are also stored. CPU 701, ROM 702 and RAM 703 are connected to each other through bus 705. Input / output (I / O, Input / Output) interface 704 is also connected to bus 705.

[0127] The following components are connected to the I / O interface 704: an input section 706 including a keyboard, a mouse, a touch screen, a microphone, an infrared sensor, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), an LED display, an OLED display, etc., and a speaker, etc.; a storage section 708 including one or more computer-readable media such as a hard disk, an optical disk, a magnetic disk, a semiconductor memory, etc.; and a communication section 709 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet.

[0128] In particular, the methods and / or embodiments in the embodiments of the present application may be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. When the computer program is executed by the central processing unit (CPU) 701, the above functions defined in the method of the present application are executed.

[0129] Another embodiment of the present application further provides a computer-readable storage medium and a computer program product, on which computer program instructions are stored. The computer program instructions can be executed by a processor to implement the methods and / or technical solutions of any one or more embodiments of the present application.

[0130] Specifically, the present embodiment can adopt any combination of one or more computer-readable media. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, systems, devices or devices including but not limited to electricity, magnetism, light, electromagnetic, infrared, or semiconductors, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, computer-readable storage media can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0131] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than a computer readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0132] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0133] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0134] The flow chart or block diagram in the accompanying drawings shows the possible architecture, function and operation of the equipment, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated system for hardware that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0135] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0136] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules and units is only a logical function division. There may be other division methods in actual implementation. For example, with units as an example, multiple units or page components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0137] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0138] In addition, each functional module or unit in each embodiment of the present application may be integrated into one processing module or unit, or each module or unit may exist physically separately, or two or more units may be integrated into one module or unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules or units.

[0139] The above-mentioned integrated modules and units implemented in the form of software function modules and units can be stored in a computer-readable storage medium. The above-mentioned software function modules and units are stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform some steps of the methods of each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

[0141] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in a device claim can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

Claims

1. A method for determining an object classification, characterized in that include: Processing the object to be classified using a first initial classification model to obtain a first initial classification result and a first confidence level for the first initial classification result; In response to the first confidence being greater than or equal to a first confidence threshold, at least using the second initial classification model to process the object to be classified to obtain a second initial classification result and a second confidence for the second initial classification result, wherein the first initial classification model and the second initial classification model are obtained by distillation based on the same target classification model; In response to the first initial classification result and the second initial classification result both indicating target classification, and the second confidence level being greater than or equal to a second confidence level threshold, the target classification is used as the classification result of the object to be classified.

2. The method according to claim 1, characterized in that Also includes: In response to the first initial classification result and the second initial classification result both indicating target classification, and the sum of the first confidence and the second confidence is greater than or equal to a third confidence threshold, the target classification is used as the classification result of the object to be classified.

3. The method according to claim 1, characterized in that The method further includes a third initial classification model distilled from the target model, wherein the model scale of the third initial classification model is greater than that of the first initial classification model and less than or equal to that of the second initial classification model, and in response to the first confidence being greater than or equal to a first confidence threshold, at least using the second initial classification model to process the object to be classified, to obtain a second initial classification result and a second confidence for the second initial classification result, including: Processing the object to be classified using the second initial classification model to obtain a second initial classification result and a second confidence level for the second initial classification result, and processing the object to be classified using the third initial classification model to obtain a third initial classification result and a third confidence level for the third initial classification result; In response to the first initial classification result, the second initial classification result and the third initial classification result all indicating the target classification, and the sum of the second confidence and the third confidence is greater than or equal to a fourth confidence threshold, the target classification is used as the classification result of the object to be classified.

4. The method according to claim 3, characterized in that The first initial classification model, the second initial classification model, and the third initial classification model are independently deployed on different graphics processors.

5. The method according to any one of claims 2 to 4, characterized in that: The first initial classification model includes a first classification head for generating the first confidence, the second initial classification model includes a second classification head for generating the second confidence; and in response to also including the third initial classification model, the third initial classification model includes a third classification head for generating the third confidence.

6. The method according to claim 1, characterized in that The first initial classification model and the second initial classification model are obtained by distillation using sample objects to be classified, logical outputs generated when the sample objects to be classified are processed by the target classification model, and reference confidences corresponding to the logical outputs.

7. The method according to claim 1, characterized in that Also includes: estimating the amount of resources required by the target classification model to process the object to be classified, and obtaining a resource amount estimation result; as well as The step of processing the model to be classified by using the first initial classification model comprises: In response to the resource quantity estimation result being less than the resource quantity that the target classification model can currently provide, the model to be classified is processed using a first initial classification model to obtain a first initial classification result and a first confidence level for the first initial classification result.

8. A device for determining object classification, characterized in that include: A first initial result and confidence determination module, configured to process the object to be classified using a first initial classification model to obtain a first initial classification result and a first confidence for the first initial classification result; a second initial result and confidence determination module, configured to, in response to the first confidence being greater than or equal to a first confidence threshold, at least process the object to be classified using the second initial classification model to obtain a second initial classification result and a second confidence for the second initial classification result, wherein the first initial classification model and the second initial classification model are obtained by distillation based on the same target classification model; The first classification result determination module is configured to, in response to the first initial classification result and the second initial classification result both indicating target classification and the second confidence being greater than or equal to a second confidence threshold, use the target classification as the classification result of the object to be classified.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A computer-readable medium, characterized in that Computer program instructions are stored thereon, and the computer program instructions can be executed by a processor to implement the method as claimed in any one of claims 1 to 7.

11. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images