Content identification method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a content recognition method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a first object type of a main object according to the main object in a to-be-recognized image; obtaining a corresponding target category recognition model according to the first object category, processing the to-be-recognized image based on the target category recognition model to obtain a second object category and a corresponding prediction confidence coefficient, and obtaining a first recognition result of the to-be-recognized image according to the second object category and the corresponding prediction confidence coefficient, the first recognition result represents a predicted object category of the main object, and the predicted object category is located between a first category level and a second category level. According to the method, the corresponding target category recognition model is selected, the main object in the to-be-recognized objects is subjected to fine-grained classification, and the second object category output by the target category recognition model is corrected based on the prediction confidence, so that the recognition result has fine classification granularity, and meanwhile, the correctness of the recognition result is ensured.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer technology, and in particular to a content recognition method, device, electronic device, and storage medium. Background Art

[0002] Currently, content recognition and classification of media data, such as images and videos, are widely used in various application scenarios. These recognition tasks are typically performed using pre-trained classification models. For example, by inputting an image to be identified into a classification model, the model outputs a description of the image's content.

[0003] In order to meet the user's query needs for subdivided categories, in the existing technology, the classification model is trained by using training samples of more subdivided categories, so that the classification model can distinguish more fine-grained object categories and make the recognition results more accurate.

[0004] However, in actual applications, while the classification model pursues fine-grained classification results, it also leads to the problem of low classification accuracy of the recognition results. Summary of the Invention

[0005] The embodiments of the present disclosure provide a content recognition method, apparatus, electronic device, and storage medium to overcome the problem of low classification accuracy of recognition results.

[0006] In a first aspect, an embodiment of the present disclosure provides a content identification method, including:

[0007] According to the main object in the image to be identified, a first object category of the main object is obtained, and the first object category is located at a first category level; according to the first object category, a corresponding target category recognition model is obtained, and the image to be identified is processed based on the target category recognition model to obtain a second object category and a corresponding prediction confidence, wherein the second object category is located at a second category level, and the second category level is a refined level of the first category level; according to the second object category and the corresponding prediction confidence, a first recognition result of the image to be identified is obtained, wherein the first recognition result represents the predicted object category of the main object, and the predicted object category is located between the first category level and the second category level.

[0008] In a second aspect, an embodiment of the present disclosure provides a data processing device, including:

[0009] A first recognition module is configured to obtain a first object category of a subject object in an image to be recognized, wherein the first object category is located at a first category level;

[0010] a second recognition module, configured to obtain a corresponding target category recognition model based on the first object category, and process the image to be recognized based on the target category recognition model to obtain a second object category and a corresponding prediction confidence, wherein the second object category is at a second category level, which is a refinement level of the first category level;

[0011] A processing module is used to obtain a first recognition result of the image to be recognized based on the second object category and the corresponding prediction confidence, wherein the first recognition result represents the predicted object category of the main object, and the predicted object category is located between the first category level and the second category level.

[0012] In a third aspect, an embodiment of the present disclosure provides an electronic device, including: a processor and a memory;

[0013] The memory stores computer-executable instructions;

[0014] The processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the content identification method described in the first aspect and various possible designs of the first aspect.

[0015] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the content recognition method described in the first aspect and various possible designs of the first aspect is implemented.

[0016] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the content identification method described in the first aspect and various possible designs of the first aspect.

[0017] The content recognition method, apparatus, electronic device, and storage medium provided in this embodiment obtain a first object category of the subject object in the image to be recognized, the first object category being located at a first category level; obtain a corresponding target category recognition model based on the first object category; and process the image to be recognized based on the target category recognition model to obtain a second object category and a corresponding prediction confidence, wherein the second object category is located at a second category level, which is a refinement of the first category level; and obtain a first recognition result for the image to be recognized based on the second object category and the corresponding prediction confidence, wherein the first recognition result represents the predicted object category of the subject object, which is located between the first category level and the second category level. After coarse-grained classification of the image to be recognized, the corresponding target category recognition model is selected, and fine-grained classification of the subject object in the image to be recognized is obtained, the second object category and the corresponding prediction confidence are obtained. The second object category is then corrected based on the prediction confidence to obtain a first recognition result that balances classification accuracy and correctness, thereby ensuring the correctness of the recognition result while maintaining a finer classification granularity. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 An application scenario diagram of the content recognition method provided by the embodiment of the present disclosure;

[0020] Figure 2 Schematic diagram of the process of the content identification method provided in the embodiment of the present disclosure Figure 1 ;

[0021] Figure 3 for Figure 2 A flowchart of a specific implementation method of step S101 in the embodiment shown;

[0022] Figure 4 A schematic diagram of a category hierarchy provided for an embodiment of the present disclosure;

[0023] Figure 5 Schematic diagram of the process of the content identification method provided in the embodiment of the present disclosure Figure 2 ;

[0024] Figure 6A schematic diagram of the structure of a tag tree data provided by an embodiment of the present disclosure;

[0025] Figure 7 for Figure 5 A flowchart of a possible implementation of step S204 in the illustrated embodiment;

[0026] Figure 8 for Figure 7 A flowchart of a specific implementation method of step S2041 in the embodiment shown;

[0027] Figure 9 for Figure 5 A flowchart of another possible implementation of step S204 in the embodiment shown;

[0028] Figure 10 A schematic diagram of a process for verifying an image to be identified based on a verification model provided by an embodiment of the present disclosure;

[0029] Figure 11 A structural block diagram of a content recognition device provided in an embodiment of the present disclosure;

[0030] Figure 12 A schematic structural diagram of an electronic device provided in an embodiment of the present disclosure;

[0031] Figure 13 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0033] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0034] The following explains the application scenarios of the embodiments of the present disclosure:

[0035] Figure 1This is an application scenario diagram of the content recognition method provided by the embodiment of the present disclosure. The content recognition method provided by the embodiment of the present disclosure can be applied to application scenarios such as content search and information recommendation. The execution subject of this embodiment can be a terminal device or server that performs image content recognition and image content classification functions. Taking the server as an example, after receiving the content recognition request sent by the terminal device (the request contains the image to be recognized), the server obtains the name of the main object in the image to be recognized, that is, the recognition result, by executing the method provided by this embodiment, and returns the recognition result to the terminal device side for display. For details, refer to Figure 1 As shown in , the terminal device loads an image to be identified containing a "dog" on the client side, and in response to the user's trigger operation on the trigger control (the control name is "Identify"), sends the image to be identified to the server side for processing. After that, the server returns the text describing the type name of the "dog" in the image to be identified (that is, the recognition result, such as "Golden Retriever") and displays it in the client of the terminal device, thereby completing the image-based content recognition process.

[0036] In the prior art, in order to meet the user's query requirements for subdivided categories in the content recognition function, the recognition model is trained using training samples of more subdivided categories. This allows the recognition model to distinguish more fine-grained object categories and make the recognition results more accurate. For example, the above recognition model can distinguish different categories based on dog breeds (Labrador, Bichon Frise) and dog characteristics (elderly dogs, young dogs), rather than just distinguishing "dog" and "cat". However, in actual application, while pursuing fine-grained classification results, the above recognition model will lead to a decrease in the model's generalization ability, which in turn affects the accuracy of the recognition results output by the model.

[0037] The embodiments of the present disclosure provide a content recognition method that realizes content recognition with a finer classification granularity while ensuring the correctness of the recognition result, thereby solving the above-mentioned problem.

[0038] refer to Figure 2 , Figure 2 Schematic diagram of the process of the content identification method provided in the embodiment of the present disclosure Figure 1 The method of this embodiment can be applied in a server. The content identification method includes:

[0039] Step S101: obtaining a first object category of the main object according to the main object in the image to be identified, where the first object category is located at a first category level.

[0040] For example, refer to Figure 1The application scenario diagram shown uses a server as the execution entity of this method embodiment as an example. The server can obtain the image to be identified by receiving a recognition request from a terminal device. The image to be identified can be included in the recognition request and sent by the terminal device, or stored in a third-party storage device or locally on the server. The specific method for obtaining the image to be identified is not specifically limited. The server then processes the image to be identified and identifies the subject object in the image to be identified. The object in the image refers to a content object, such as a person, object, or animal. The subject object refers to the content object that serves as the main content within the image to be identified. When the image to be identified includes a distinct content object, this content object is the subject object. For example, a single person in a single-person photo is the subject object. When the image to be identified includes multiple content objects, the subject object can be determined based on factors such as the position of each content object, the image area occupied, and the clarity of its outline. For example, the content object located in the center of the image and occupying a large area is the subject object. There can be one or more subject objects. In a specific implementation, the image to be identified is processed by an image recognition model, and the subject object is identified based on pixel features composed of pixel values ​​of each pixel point of the image to be identified.

[0041] Furthermore, after identifying the subject object, the subject object is type-identified to obtain the object category corresponding to the subject object. For example, identifying the subject object as "cat," "Labrador," etc., are all implementations of object classification. In this embodiment, the object category obtained by identifying the subject object is the first object category, i.e., this first object category is located at the first category level. The category level describes the degree of category refinement (generalization). For example, the lower the category level, the higher the category generalization, such as "dog," which is an object category at a low category level. Conversely, the higher the category level, the higher the category refinement, such as "black Labrador Retriever." As described in the subsequent embodiments, the first category level in this embodiment corresponds to a lower category level. The first object category at this first category level has a higher degree of generalization and is a coarse-grained classification result, such as "dog" in the above example. Therefore, the specific implementation of obtaining the first object category in this embodiment can be achieved by processing the image to be processed using a general image recognition model (hereinafter referred to as the general recognition model).

[0042] Furthermore, in a possible implementation, as Figure 3 As shown, the specific implementation of step S101 includes:

[0043] Step S1011: obtaining a preset category corresponding to a preset candidate category recognition model.

[0044] Step S1012: Using the preset category as a detection parameter, perform target detection on the image to be recognized to obtain at least one corresponding target subject belonging to the preset category.

[0045] Step S1013: Obtain a corresponding first object category according to the target subject object.

[0046] For example, in the steps of this embodiment, when identifying the first object category using a pre-trained universal recognition model, several preset categories corresponding to the candidate category recognition models are first obtained. For example, a candidate category recognition model M1 corresponding to "dog," a candidate category recognition model M2 corresponding to "cat," and a candidate category recognition model M3 corresponding to "bird" are selected. The server will subsequently use one of the three candidate category recognition models to achieve fine-grained recognition of the subject object in the image to be identified. Therefore, in the steps of this embodiment, for example, M1, M2, and M3 are input into the universal recognition model to guide the universal recognition model to only recognize objects of the preset categories corresponding to M1, M2, and M3, thereby obtaining the target subject object in the image to be identified. Subsequently, after performing recall sorting and other steps, the one or more identified target subject objects with the highest confidence are determined as the output object, and the object category of the output object (the universal recognition model will synchronize the object category corresponding to the output when identifying the target subject object) is determined to be the first object category.

[0047] In the steps of this embodiment, since the control parameters (preset categories) are input into the universal recognition model, on the one hand, the universal recognition model will only recognize targets of the above-mentioned preset categories during operation, which can reduce the recognition calculation amount of the model. On the other hand, the target subject objects output by the universal recognition model will only include target subject objects of the above-mentioned preset categories, thereby improving the accuracy of the obtained first object similarity and reducing the problem of subsequent incorrect selection of the target category recognition model due to inaccurate first object category.

[0048] Step S102: According to the first object category, a corresponding target category recognition model is obtained, and the image to be recognized is processed based on the target category recognition model to obtain a second object category and a corresponding prediction confidence, wherein the second object category is located at a second category level, and the second category level is a refined level of the first category level.

[0049] Exemplarily, after obtaining the first object category, a corresponding, matching category recognition model, i.e., a target category recognition model, is selected based on the first object category. For example, the first object category can be represented by a category identifier. When the first object category is #1, it indicates that the corresponding object category is "dog." At this time, a model M1 specifically for dog image recognition, i.e., a target category recognition model, is obtained. The image to be recognized, or a processed image generated based on the image to be recognized (e.g., an image generated after cropping or downsampling), is processed to generate a second object category and a corresponding prediction confidence. The second object category is located at the second category level, which is a refined level of the first category level, i.e., the second object category is a refined category of the first object category obtained in the previous step (correspondingly, the first category level is a generalized level of the second category level, and the first object category is a generalized category of the second object category, and the two are relative). More specifically, for example, if the first object category is "dog," using the capabilities of the target category recognition model, the second object category obtained is "old Labrador Retriever."

[0050] Among them, based on the above introduction, the category recognition model can be understood as an image recognition model for processing images of "specific content". The target category recognition model that matches the first object category is used to recognize the image to be recognized, and the target category recognition model is fully utilized to make fine-grained classification capabilities for "specific content" to obtain a more refined recognition result, that is, the second object category. At the same time, similar to other image recognition models, the category recognition model will output the corresponding prediction confidence when outputting the predicted object category, also known as credibility. The greater the prediction confidence, the more credible the prediction result (object category), and vice versa. For example, the second object category obtained by the server in this embodiment can be the object category with the highest prediction confidence output by the target category recognition model after recall and sorting. The above-mentioned category recognition model is generated after training based on the training samples of "specific content". Its specific training process will not be introduced this time.

[0051] Step S103: Obtain a first recognition result of the image to be recognized based on the second object category and the corresponding prediction confidence, wherein the first recognition result represents the predicted object category of the main object, and the predicted object category is between the first category level and the second category level.

[0052] Exemplarily, after obtaining the second object category and the corresponding prediction confidence, the server further evaluates the credibility of the predicted second object category based on the prediction confidence. When the prediction confidence is high, it means that the second object category predicted by the target category recognition model is credible, that is, the target category recognition model selected based on the above steps is capable of accurately identifying a fine-grained object category. At this time, the second object can be directly used as the first recognition result of the final output to achieve the purpose of classification accuracy; when the prediction confidence is low, it means that the second object category predicted by the target category recognition model is unreliable. At this time, an object category that is more generalized than the second object category can be obtained as the predicted object category, and then the first recognition result is generated.

[0053] For example, if the second object category is "Old Labrador Retriever" and its corresponding prediction confidence is lower than the confidence threshold, the second object category is generalized to obtain the predicted object category "Labrador Retriever" and this predicted object category is used as the first recognition result. The category level at which the predicted object category is located is the generalization level of the second category level corresponding to the second object category.

[0054] Furthermore, in one possible implementation, a pre-trained text generalization model can be used to process the second object category to obtain a corresponding predicted object category. That is, the text generalization model can be used to remove at least one limiting feature in the description text corresponding to the second object type. For example, the word "elderly" in "elderly Labrador Retriever" can be removed to generate "Labrador Retriever", thereby achieving category generalization. The specific implementation method of the text generalization model is related to its training method and will not be repeated here.

[0055] Of course, in other possible implementation methods, the generalization of the second object category can also be achieved through preset data that can describe the logical relationship between different category levels, such as label tree data, to obtain the process of predicting the object category. The specific implementation method is introduced in detail in subsequent embodiments and can be set according to needs.

[0056] Figure 4 A schematic diagram of a category hierarchy provided in an embodiment of the present disclosure is shown below in conjunction with Figure 4 The above process is further described. For example, Figure 4As shown, first, the image to be recognized is processed by a general recognition model to obtain a first object category (shown as object category C1 in the figure), whose specific content is, for example, "dog", and the first object category is located at the first category level (shown as category level L1 in the figure); then, the corresponding target category recognition model (shown as model M1 in the figure) is determined by the first object category, and the target category recognition model processes the image to be processed to obtain a second object category (shown as object category C2 in the figure) and a corresponding prediction confidence Q, wherein the second object category, whose specific content is, for example, "black Labrador Retriever", is located at the second category level (shown as category level L4 in the figure); as shown in the figure, category level L4 is a refinement level of category level L1, corresponding to a more refined category. Based on the hierarchical relationship recorded in the preset label tree data, between category level L1 and category level L4, for example, category level L2 corresponds to category object C3, whose specific content is, for example, "retriever," and category level L3 corresponds to category object C4, whose specific content is, for example, "Labrador retriever." Then, based on the specific value of the prediction confidence Q, a predicted object category is selected from category objects C1, C3, and C4, thereby obtaining a first recognition result.

[0057] In this embodiment, a first object category of the subject object in the image to be identified is obtained, the first object category being located at the first category level. A corresponding target category recognition model is obtained based on the first object category, and the image to be identified is processed based on the target category recognition model to obtain a second object category and a corresponding prediction confidence, wherein the second object category is located at the second category level, which is a refinement of the first category level. Based on the second object category and the corresponding prediction confidence, a first recognition result of the image to be identified is obtained, wherein the first recognition result represents the predicted object category of the subject object, which is located between the first category level and the second category level. After coarse-grained classification of the image to be identified is performed, a corresponding target category recognition model is selected, and fine-grained classification of the subject object in the object to be identified is obtained, the second object category and the corresponding prediction confidence are obtained. The second object category is then corrected based on the prediction confidence to obtain a first recognition result that balances classification accuracy and classification correctness, thereby ensuring the correctness of the recognition result while maintaining a finer classification granularity.

[0058] refer to Figure 5 , Figure 5 Schematic diagram of the process of the content identification method provided in the embodiment of the present disclosure Figure 2 In this embodiment Figure 2 Based on the embodiment shown, step S102 is further refined, and a step of reviewing the second object category is added. The content recognition method includes:

[0059] Step S201: obtaining a first object category of the main object according to the main object in the image to be identified, where the first object category is located at a first category level.

[0060] Step S202: Based on the first object category, a corresponding target category recognition model is obtained, and the image to be recognized is processed based on the target category recognition model to obtain a second object category and a corresponding prediction confidence, wherein the second object category is located at a second category level, and the second category level is a refined level of the first category level.

[0061] Step S203: If the prediction confidence is greater than the confidence threshold, the second object category is determined as the first recognition result of the image to be recognized.

[0062] Step S204: If the prediction confidence is less than the confidence threshold, the generalized object category corresponding to the second object category is determined as the first recognition result of the image to be recognized based on the preset label tree data, wherein the label tree data records at least the object categories corresponding to the first category level and the second category level, and the generalized object category is located at the generalization level of the second category level.

[0063] For example, refer to Figure 2 As described in the illustrated embodiment, if the prediction confidence is greater than or equal to the confidence threshold, the second object category can be directly determined as the first recognition result for the image to be recognized. This situation will not be further described. If the prediction confidence is less than or equal to the confidence threshold, a generalized object category corresponding to the second object category is obtained based on the preset label tree data, and the generalized object category is then used as the first recognition result for the image to be recognized. Specifically, the label tree data is data used to record the logical relationship between category hierarchies. In the label tree data, at least the object categories corresponding to the first category hierarchy and the second category hierarchy are recorded.

[0064] Figure 6 A schematic diagram of the structure of a tag tree data provided by the embodiment of the present disclosure, referring to Figure 6As shown, under the first category level, there are relatively coarse-grained category names such as "cats" and "dogs". The specific implementation method of the object category under the first category level can be set as needed, for example, it can also be "land creatures", "aquatic creatures", etc. Further, taking "cats" as an example, at its detailed level, that is, the second category level, the corresponding object categories include "British Shorthair Cat", "Ragdoll Cat", etc. For "British Shorthair Cat", the object categories under the next level of detailed level include "British Shorthair Blue Cat", "British Shorthair Progressive Layer", etc. The label tree data records the logical relationship between different category levels under the same root category through the above data structure. In the label tree data, at least the object categories corresponding to the above-mentioned first category level and second category level are recorded, so that the generalized object category corresponding to the second object category can be obtained based on the label tree data.

[0065] Furthermore, in a possible implementation, as Figure 7 As shown, the specific implementation steps of step S204 include:

[0066] Step S2041: Determine the target category level of the label tree data based on the prediction confidence;

[0067] Step S2042: Obtain the target generalized object category corresponding to the target category level, and determine the target generalized object category as the first recognition result of the image to be recognized.

[0068] Exemplarily, in the steps of this embodiment, the server first determines the target category level that matches the prediction confidence from the label tree structure described by the label tree data based on the prediction confidence obtained in the previous step. Specifically, for example, the higher the prediction confidence (when it is less than the confidence threshold), the higher the corresponding target category level, which means that the classification granularity of the corresponding target generalized object category is finer and the classification is more accurate; conversely, the lower the prediction confidence, the lower the corresponding target category level, which means that the classification granularity of the corresponding target generalized object category is coarser and the classification is more general, but at the same time, its correctness is higher. Exemplarily, there can be a preset mapping relationship between the prediction confidence and the target category level, so that the target category level is determined based on the mapping relationship.

[0069] In another possible implementation, Figure 8 As shown, the specific implementation of step S2041 includes:

[0070] Step S2041A: Obtain the confidence difference and / or confidence ratio between the prediction confidence and the confidence threshold.

[0071] Step S2041B: Determine the corresponding target category level according to the confidence difference and / or confidence ratio value.

[0072] Exemplarily, after obtaining the prediction confidence, the server determines the distance relationship between the prediction confidence and the confidence threshold based on the confidence difference and / or confidence ratio between the two, and then determines the corresponding target category level based on the distance relationship. In the steps of this embodiment, when mapping the target category level based on the prediction confidence, the confidence threshold factor is further referenced to avoid interference caused by different confidence thresholds corresponding to the prediction confidence output by different target category recognition models, further improve the accuracy of the determined target category level, and further improve the accuracy of the recognition result while ensuring the correctness of the recognition result.

[0073] In another possible implementation, Figure 9 As shown, the specific implementation steps of step S204 include:

[0074] Step S2043: Obtaining the image access popularity of the image to be identified;

[0075] Step S2044: determining the target category level of the tag tree data according to the image access popularity;

[0076] Step S2045: Obtain the target generalized object category corresponding to the target category level, and determine the target generalized object category as the first recognition result of the image to be recognized.

[0077] For example, in another possible implementation, for an image to be identified, a target category level is dynamically determined based on the image access popularity of the image to be identified, where the image access popularity of the image to be identified refers to the access frequency of the image to be identified. Specifically, for example, when the image to be identified is a target object with high popularity, that is, a high access frequency, a higher (subdivided) category level is determined as the target category level; otherwise, a lower (generalized) category level is set as the target category level.

[0078] This is because the label tree data is constructed based on image access popularity. For example, for categories with high access popularity such as "cats" and "dogs", there are more sample image groups with more detailed classifications. Therefore, the label tree data generated based on this sample data has better descriptive capabilities for such high-access images and can accurately describe the category name of the image subject in the image at a finer and more refined classification level. Conversely, for categories with lower access popularity such as "microorganisms", the sample data used to construct the label tree data corresponds to a smaller number of sample image groups with more coarse classifications, resulting in the label tree data being unable to describe very fine-grained classification names. Therefore, when processing an image to be identified, based on the image access popularity of the image to be identified, a higher target category level (refinement level) is set for the image to be identified with high image access popularity, thereby improving recognition accuracy and achieving more accurate content search. When processing an image to be identified with low image access popularity, a lower target category level (generalization level) is set to ensure the correctness of the recognition result.

[0079] Optionally, on the other hand, after step S202, the method further includes:

[0080] Step S205: Processing the image to be identified based on the verification model to obtain a third object category, where the third object category is located at a third category level.

[0081] Step S206: If the third category level is not a generalization level of the second category level, a second recognition result is generated, and the second recognition result indicates that the second object category output by the target category recognition model is an incorrect result; if the third category level is a generalization level of the second category level, step S203 or step S204 is executed.

[0082] Exemplarily, on the other hand, in order to further improve the correct probability of the recognition result, after step S202 is completed and the second object category is obtained, the second object category is further reviewed by a preset verification model. Specifically, the image to be recognized is first processed by the verification model. The verification model can be understood as an image recognition model with a lower classification granularity. More specifically, for example, the recognition result output by the general recognition model (such as the first object category) is located in a lower category level L1, the recognition result output by the category recognition model (such as the second object category) is located in a higher category level L3, and the recognition result output by the verification model (such as the third object category) is located in the category level L2 between the two, that is, the third category level referred to above. Afterwards, based on the label tree data, the third category level and the second category The third category level is checked. If the third category level is a generalization level of the second category level, that is, the third object category and the second object category are on the same category branch path, then the result of the positive second object category is likely to be correct. In this case, the subsequent step S203 or step S204 can be continued. On the contrary, if the third category level is not a generalization level of the second category level, that is, the third object category and the second object category are not on the same category branch path, then it means that one of the second object category and the third object category is incorrectly recognized. At the same time, considering that the classification accuracy corresponding to the verification model is lower and the generalization ability is stronger, the credibility of the third object category output by it is higher. Therefore, in this case, a second recognition result is generated, and the second recognition result indicates that the second object category output by the target category recognition model is an incorrect result. Afterwards, the server can further correct the above process based on the second recognition result until the correct result, that is, the first recognition result, is obtained.

[0083] Figure 10 A schematic diagram of a process for verifying an image to be identified based on a verification model provided by an embodiment of the present disclosure is shown as follows: Figure 10As shown, exemplarily, first, based on the image to be identified, the general recognition model is processed to determine the object category P, for example, representing "animal", and then multiple target category recognition models are determined, such as model M1, model M2, and model M3 shown in the figure. Exemplarily, the target category recognition models are category recognition models that further refine the classification of "birds", "dogs", and "cats", respectively. Afterwards, the image to be identified is processed using the above-mentioned models M1, M2, and M3, respectively, to obtain the corresponding second object categories, such as object category P01, object category P12, and object category P23 shown in the figure, which represent, for example, "ostrich", "Labrador retriever", and "orange cat", respectively. On the other hand, the image to be processed is processed by the verification model to obtain the corresponding third object category, such as P10 shown in the figure, which represents, for example, "hound". Next, based on the label tree data and the third object category, the second object category is verified, confirming that object category P12 (Labrador Retriever) is on the same category branch path as object category P10 (Hound). That is, the category hierarchy where object category P10 resides is at the generalization level, and the category hierarchy where object category P12 resides is at the generalization level. Therefore, object category P12 is considered the verified second object category, and subsequent steps are performed.

[0084] Furthermore, in another possible implementation, after step S206, the following steps are further included:

[0085] Step S207: obtaining a corresponding modified category recognition model according to the third object category and the label tree data;

[0086] Step S208: Processing the image to be identified based on the modified category recognition model to obtain a second object category and a corresponding prediction confidence.

[0087] Exemplarily, when the server obtains only one second-category object based on step S202, if a second recognition result is obtained after step S206 (i.e., the target category recognition model recognizes incorrectly), the target category recognition model can be further replaced based on the third object category. For example, the corrected category recognition model M2 corresponding to the third object category is used to replace the currently used target category recognition model M1, and the image to be recognized is further processed based on the corrected category recognition model M2 to obtain an updated second object category and the corresponding prediction confidence. The specific execution process is similar to that of step S202. Afterwards, based on the updated second object category and the corresponding prediction confidence, the server returns to execute step S203 or step S204, and re-uses the prediction confidence for judgment until the first recognition result is obtained.

[0088] In the steps of this embodiment, when the second object category output by the target category recognition model is an erroneous result, the target category recognition model currently in use is corrected based on the third object category output by the verification model to obtain a corrected category recognition model, which is equivalent to reselecting a suitable category recognition model, thereby correcting the problem of incorrect target category recognition model selection caused by inaccurate recognition of the general recognition model, and providing the accuracy of the first recognition result.

[0089] In this embodiment, the implementation of steps S201-S202 is the same as that of the present disclosure. Figure 2 The implementation methods of step S101 and step S102 in the illustrated embodiment are the same and will not be described in detail here.

[0090] Corresponding to the content identification method of the above embodiment, Figure 11 This is a block diagram of the content recognition device provided in an embodiment of the present disclosure. The methods described in the above embodiments can be executed by this content recognition device, which can be implemented using software and / or hardware and integrated into electronic devices with certain data processing capabilities. These electronic devices may include, but are not limited to, mobile terminals with big data processing capabilities, as well as fixed terminals with big data processing capabilities, such as desktop computers and supercomputers.

[0091] For ease of explanation, only the parts related to the embodiments of the present disclosure are shown. Figure 11 , the content identification device 3 includes:

[0092] A first recognition module 31 is configured to obtain a first object category of the subject object according to the subject object in the image to be recognized, where the first object category is located at a first category level;

[0093] a second recognition module 32 for obtaining a corresponding target category recognition model based on the first object category, and processing the image to be recognized based on the target category recognition model to obtain a second object category and a corresponding prediction confidence, wherein the second object category is at a second category level, which is a refinement of the first category level;

[0094] The processing module 33 is used to obtain a first recognition result of the image to be recognized based on the second object category and the corresponding prediction confidence, wherein the first recognition result represents the predicted object category of the main object, and the predicted object category is between the first category level and the second category level.

[0095] According to one or more embodiments of the present disclosure, the first recognition module 31 is specifically used to: obtain a preset category corresponding to a preset alternative category recognition model; perform target detection on the image to be recognized based on the preset category corresponding to the alternative category recognition model to obtain at least one corresponding target subject object belonging to the preset category; and obtain a corresponding first object category based on the target subject object.

[0096] According to one or more embodiments of the present disclosure, the processing module 33 is specifically used to: if the prediction confidence is greater than the confidence threshold, determine the second object category as the first recognition result of the image to be identified; if the prediction confidence is less than the confidence threshold, based on the preset label tree data, determine the generalized object category corresponding to the second object category as the first recognition result of the image to be identified, wherein the label tree data at least records the object categories corresponding to the first category level and the second category level, and the generalized object category is located at the generalization level of the second category level.

[0097] According to one or more embodiments of the present disclosure, when the processing module 33 determines the generalized object category corresponding to the second object category as the first recognition result of the image to be identified based on the preset label tree data, it is specifically used to: determine the target category hierarchy of the label tree data according to the prediction confidence; obtain the target generalized object category corresponding to the target category hierarchy, and determine the target generalized object category as the first recognition result of the image to be identified.

[0098] According to one or more embodiments of the present disclosure, when the processing module 33 determines the target category level of the label tree data based on the prediction confidence, it is specifically used to: obtain the confidence difference and / or confidence ratio value between the prediction confidence and the confidence threshold; and determine the corresponding target category level based on the confidence difference and / or confidence ratio value.

[0099] According to one or more embodiments of the present disclosure, when the processing module 33 determines the generalized object category corresponding to the second object category as the first recognition result of the image to be identified based on the preset label tree data, it is specifically used to: obtain the image access popularity of the image to be identified; determine the target category level of the label tree data according to the image access popularity; obtain the target generalized object category corresponding to the target category level, and determine the target generalized object category as the first recognition result of the image to be identified.

[0100] According to one or more embodiments of the present disclosure, after processing the image to be identified based on the target category recognition model to obtain the second object category and the corresponding prediction confidence, the second recognition module 32 is further used to: process the image to be identified based on the verification model to obtain a third object category, and the third object category is located at the third category level; if the third category level is not a generalization level of the second category level, then generate a second recognition result, and the second recognition result indicates that the second object category output by the target category recognition model is an incorrect result.

[0101] According to one or more embodiments of the present disclosure, after generating the second recognition result, the second recognition module 32 is further used to: obtain a corresponding modified category recognition model based on the third object category and label tree data; process the image to be recognized based on the modified category recognition model to obtain the second object category and the corresponding prediction confidence.

[0102] The first recognition module 31, the second recognition module 32 and the processing module 33 are connected in sequence. The content recognition device 3 provided in this embodiment can implement the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated in this embodiment.

[0103] Figure 12 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure is shown in FIG. Figure 12 As shown, the electronic device 4 includes:

[0104] A processor 41, and a memory 42 communicatively connected to the processor 41;

[0105] Memory 42 stores computer-executable instructions;

[0106] The processor 41 executes the computer execution instructions stored in the memory 42 to implement the following Figure 2-Figure 10 The content identification method in the illustrated embodiment.

[0107] Optionally, the processor 41 and the memory 42 are connected via a bus 43 .

[0108] For related instructions, please refer to Figure 2-Figure 10 The relevant descriptions and effects corresponding to the steps in the corresponding embodiments can be understood, and no further details are given here.

[0109] The present invention provides a computer-readable storage medium that stores computer-executable instructions. When the computer-executable instructions are executed by a processor, the computer-executable instructions are used to implement the present invention. Figure 2-Figure 10 The content identification method provided in any one of the corresponding embodiments.

[0110] The present invention provides a computer program product, including a computer program, which implements the present invention when executed by a processor. Figure 2-Figure 10 The content identification method provided in any one of the corresponding embodiments.

[0111] In order to implement the above embodiment, the embodiment of the present disclosure further provides an electronic device.

[0112] refer to Figure 13 , which shows a schematic structural diagram of an electronic device 900 suitable for implementing the embodiments of the present disclosure. The electronic device 900 may be a terminal device or a server. The terminal device may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 13 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0113] like Figure 13 As shown, the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. Various programs and data required for the operation of the electronic device 900 are also stored in the RAM 903. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0114] Typically, the following devices may be connected to the I / O interface 905: an input device 906 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 908 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 909. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 13The electronic device 900 is illustrated with various means for performing various functions, but it is to be understood that not all of the means illustrated are required to perform the functions. Some of the means can be implemented by software, hardware, or a combination of software and hardware.

[0115] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program comprising program code for performing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from the network via the communication device 909, or installed from the storage device 908, or installed from the ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

[0116] It should be noted that the computer readable medium described above in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination thereof. The computer readable storage medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as part of a carrier wave, in which the computer readable program code is carried. Such a propagated data signal can take any of a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer readable medium can be transmitted by any suitable medium, including but not limited to wire, cable, RF, or any suitable combination thereof.

[0117] The computer readable medium described above can be contained in the electronic device described above; or can exist separately without being assembled into the electronic device.

[0118] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0119] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0120] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0121] The units or modules involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit or module does not, in some cases, limit the unit itself.

[0122] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0123] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0124] In a first aspect, according to one or more embodiments of the present disclosure, a content identification method is provided, including:

[0125] According to the main object in the image to be identified, a first object category of the main object is obtained, and the first object category is located at a first category level; according to the first object category, a corresponding target category recognition model is obtained, and the image to be identified is processed based on the target category recognition model to obtain a second object category and a corresponding prediction confidence, wherein the second object category is located at a second category level, and the second category level is a refined level of the first category level; according to the second object category and the corresponding prediction confidence, a first recognition result of the image to be identified is obtained, wherein the first recognition result represents the predicted object category of the main object, and the predicted object category is located between the first category level and the second category level.

[0126] According to one or more embodiments of the present disclosure, obtaining the first object category of the main object based on the main object in the image to be identified includes: obtaining a preset category corresponding to a preset alternative category recognition model; performing target detection on the image to be identified using the preset category as a detection parameter to obtain at least one corresponding target main object belonging to the preset category; and obtaining the corresponding first object category based on the target main object.

[0127] According to one or more embodiments of the present disclosure, the first recognition result of the image to be recognized is obtained based on the second object category and the corresponding prediction confidence, including: if the prediction confidence is greater than the confidence threshold, the second object category is determined as the first recognition result of the image to be recognized; if the prediction confidence is less than the confidence threshold, based on the preset label tree data, the generalized object category corresponding to the second object category is determined as the first recognition result of the image to be recognized, wherein the label tree data records at least the object categories corresponding to the first category level and the second category level, and the generalized object category is located at the generalization level of the second category level.

[0128] According to one or more embodiments of the present disclosure, the method of determining the generalized object category corresponding to the second object category as the first recognition result of the image to be recognized based on preset label tree data includes: determining the target category hierarchy of the label tree data according to the prediction confidence; obtaining the target generalized object category corresponding to the target category hierarchy, and determining the target generalized object category as the first recognition result of the image to be recognized.

[0129] According to one or more embodiments of the present disclosure, determining the target category level of the label tree data based on the prediction confidence includes: obtaining the confidence difference and / or confidence ratio value between the prediction confidence and the confidence threshold; and determining the corresponding target category level based on the confidence difference and / or the confidence ratio value.

[0130] According to one or more embodiments of the present disclosure, the generalized object category corresponding to the second object category is determined as the first recognition result of the image to be identified based on preset label tree data, including: obtaining the image access popularity of the image to be identified; determining the target category hierarchy of the label tree data based on the image access popularity; obtaining the target generalized object category corresponding to the target category hierarchy, and determining the target generalized object category as the first recognition result of the image to be identified.

[0131] According to one or more embodiments of the present disclosure, after processing the image to be recognized based on the target category recognition model to obtain the second object category and the corresponding prediction confidence, the method further includes: processing the image to be recognized based on a verification model to obtain a third object category, and the third object category is located at a third category level; if the third category level is not a generalization level of the second category level, generating a second recognition result, and the second recognition result indicates that the second object category output by the target category recognition model is an incorrect result.

[0132] According to one or more embodiments of the present disclosure, after generating the second recognition result, it also includes: obtaining a corresponding modified category recognition model based on the third object category and label tree data; processing the image to be recognized based on the modified category recognition model to obtain a second object category and a corresponding prediction confidence.

[0133] In a second aspect, according to one or more embodiments of the present disclosure, a content identification device is provided, including:

[0134] A first recognition module is configured to obtain a first object category of a subject object in an image to be recognized, wherein the first object category is located at a first category level;

[0135] a second recognition module, configured to obtain a corresponding target category recognition model based on the first object category, and process the image to be recognized based on the target category recognition model to obtain a second object category and a corresponding prediction confidence, wherein the second object category is at a second category level, which is a refinement level of the first category level;

[0136] A processing module is used to obtain a first recognition result of the image to be recognized based on the second object category and the corresponding prediction confidence, wherein the first recognition result represents the predicted object category of the main object, and the predicted object category is located between the first category level and the second category level.

[0137] According to one or more embodiments of the present disclosure, the first recognition module is specifically used to: obtain a preset category corresponding to a preset alternative category recognition model; perform target detection on the image to be recognized using the preset category as a detection parameter to obtain at least one corresponding target subject object belonging to the preset category; and obtain a corresponding first object category based on the target subject object.

[0138] According to one or more embodiments of the present disclosure, the processing module is specifically used to: if the prediction confidence is greater than a confidence threshold, determine the second object category as the first recognition result of the image to be recognized; if the prediction confidence is less than the confidence threshold, based on preset label tree data, determine the generalized object category corresponding to the second object category as the first recognition result of the image to be recognized, wherein the label tree data at least records the object categories corresponding to the first category level and the second category level, and the generalized object category is located at the generalization level of the second category level.

[0139] According to one or more embodiments of the present disclosure, when the processing module determines the generalized object category corresponding to the second object category as the first recognition result of the image to be identified based on the preset label tree data, it is specifically used to: determine the target category hierarchy of the label tree data according to the prediction confidence; obtain the target generalized object category corresponding to the target category hierarchy, and determine the target generalized object category as the first recognition result of the image to be identified.

[0140] According to one or more embodiments of the present disclosure, when the processing module determines the target category level of the label tree data based on the prediction confidence, it is specifically used to: obtain the confidence difference and / or confidence ratio value between the prediction confidence and the confidence threshold; and determine the corresponding target category level based on the confidence difference and / or the confidence ratio value.

[0141] According to one or more embodiments of the present disclosure, when the processing module determines the generalized object category corresponding to the second object category as the first recognition result of the image to be identified based on preset label tree data, it is specifically used to: obtain the image access popularity of the image to be identified; determine the target category hierarchy of the label tree data based on the image access popularity; obtain the target generalized object category corresponding to the target category hierarchy, and determine the target generalized object category as the first recognition result of the image to be identified.

[0142] According to one or more embodiments of the present disclosure, after processing the image to be recognized based on the target category recognition model to obtain the second object category and the corresponding prediction confidence, the second recognition module is further used to: process the image to be recognized based on the verification model to obtain a third object category, and the third object category is located at a third category level; if the third category level is not a generalization level of the second category level, then generate a second recognition result, and the second recognition result indicates that the second object category output by the target category recognition model is an incorrect result.

[0143] According to one or more embodiments of the present disclosure, after generating the second recognition result, the second recognition module is further used to: obtain a corresponding modified category recognition model based on the third object category and label tree data; process the image to be recognized based on the modified category recognition model to obtain a second object category and corresponding prediction confidence.

[0144] In a third aspect, according to one or more embodiments of the present disclosure, there is provided an electronic device, comprising: at least one processor and a memory;

[0145] The memory stores computer-executable instructions;

[0146] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the content identification method as described in the first aspect and various possible designs of the first aspect.

[0147] In a fourth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the content recognition method described in the first aspect and various possible designs of the first aspect is implemented.

[0148] In a fifth aspect, according to one or more embodiments of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements the content identification method described in the first aspect and various possible designs of the first aspect.

[0149] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0150] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0151] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A content identification method, characterized in that: include: Obtaining a first object category of the subject object according to the subject object in the image to be identified, where the first object category is located at a first category level; Obtaining a corresponding target category recognition model according to the first object category, and processing the image to be recognized based on the target category recognition model to obtain a second object category and a corresponding prediction confidence, wherein the second object category is at a second category level that is a refinement level of the first category level; Based on the second object category and the corresponding prediction confidence, a first recognition result of the image to be recognized is obtained, wherein the first recognition result represents the predicted object category of the main object, and the predicted object category is located between the first category level and the second category level.

2. The method according to claim 1, characterized in that The step of obtaining a first object category of the subject object according to the subject object in the image to be identified includes: Obtaining a preset category corresponding to a preset candidate category recognition model; Using the preset category as a detection parameter, performing target detection on the image to be identified to obtain at least one corresponding target subject object belonging to the preset category; According to the target subject object, a corresponding first object category is obtained.

3. The method according to claim 1, characterized in that Obtaining a first recognition result of the image to be recognized based on the second object category and the corresponding prediction confidence includes: If the prediction confidence is greater than a confidence threshold, determining the second object category as the first recognition result of the image to be recognized; If the prediction confidence is less than the confidence threshold, the generalized object category corresponding to the second object category is determined as the first recognition result of the image to be recognized based on the preset label tree data, wherein the label tree data records at least the object categories corresponding to the first category level and the second category level, and the generalized object category is located at the generalization level of the second category level.

4. The method according to claim 3, characterized in that The step of determining, based on the preset label tree data, the generalized object category corresponding to the second object category as the first recognition result of the image to be recognized includes: Determining a target category level of the label tree data according to the prediction confidence; A target generalized object category corresponding to the target category level is obtained, and the target generalized object category is determined as a first recognition result of the image to be recognized.

5. The method according to claim 4, characterized in that Determining the target category level of the label tree data according to the prediction confidence includes: Obtaining a confidence difference and / or a confidence ratio between the prediction confidence and the confidence threshold; A corresponding target category level is determined according to the confidence difference and / or the confidence ratio.

6. The method according to claim 3, characterized in that The step of determining, based on the preset label tree data, the generalized object category corresponding to the second object category as the first recognition result of the image to be recognized includes: Obtaining the image access popularity of the image to be identified; Determining a target category level of the tag tree data according to the image access popularity; A target generalized object category corresponding to the target category level is obtained, and the target generalized object category is determined as a first recognition result of the image to be recognized.

7. The method according to claim 1, characterized in that After processing the image to be recognized based on the target category recognition model to obtain a second object category and a corresponding prediction confidence, the method further includes: processing the image to be identified based on the verification model to obtain a third object category, where the third object category is located at a third category level; If the third category level is not a generalization level of the second category level, a second recognition result is generated, where the second recognition result indicates that the second object category output by the target category recognition model is an erroneous result.

8. The method according to claim 7, characterized in that After generating the second recognition result, the method further includes: Obtaining a corresponding modified category recognition model according to the third object category and the label tree data; The image to be identified is processed based on the modified category recognition model to obtain a second object category and a corresponding prediction confidence.

9. A content recognition device, characterized in that: include: A first recognition module is configured to obtain a first object category of a subject object in an image to be recognized, wherein the first object category is located at a first category level; a second recognition module, configured to obtain a corresponding target category recognition model based on the first object category, and process the image to be recognized based on the target category recognition model to obtain a second object category and a corresponding prediction confidence, wherein the second object category is at a second category level, which is a refinement level of the first category level; A processing module is used to obtain a first recognition result of the image to be recognized based on the second object category and the corresponding prediction confidence, wherein the first recognition result represents the predicted object category of the main object, and the predicted object category is located between the first category level and the second category level.

10. An electronic device, characterized in that: include: processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the content identification method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, and when a processor executes the computer-executable instructions, the content recognition method according to any one of claims 1 to 8 is implemented.

12. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the content identification method according to any one of claims 1 to 8 is implemented.

Images