Image recognition method and device, electronic equipment and storage medium

Abstract

The invention provides an image recognition method. The method comprises the following steps: acquiring a to-be-recognized image input by a user side; based on the to-be-recognized image, determining a target subject category name corresponding to the target object; based on the target subject category name, a target subject classification tree corresponding to the target object is determined, and the target subject classification tree comprises target sub-classification items of the target object; based on the target theme classification tree, generating knowledge data of the target object; and returning the knowledge data to the user side. According to the method, the target subject category name corresponding to the target object is determined based on the to-be-recognized image, the target subject classification tree corresponding to the target object is determined based on the target subject category name, the target subject classification tree comprises the target sub-classification items of the target object, and knowledge data of different sub-classification items can be generated by using the target subject classification tree; the system knowledge related to the target object is provided for the user, the user is helped to establish a more complete and systematic knowledge system, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to an image recognition method, device, electronic equipment and storage medium. Background Art

[0002] With the rapid development of science and technology, artificial intelligence technology has been widely used in various fields, especially in the field of education, and it plays a significant role in improving learning efficiency and increasing learning interest. At present, it is particularly important for children to recognize and understand the surrounding things in the learning process, especially in the early stages of cognitive world. However, traditional education methods often rely on paper books or teachers' oral teaching. This method not only updates information slowly, but also lacks sufficient interactivity and fun, and it is difficult to stimulate children's interest in learning and desire to explore. In addition, although the existing image recognition technology can identify objects in the image, it often only stays at the recognition level and cannot provide children with more in-depth and systematic knowledge. Therefore, how to combine artificial intelligence technology to provide children with an interesting and efficient learning platform has become a problem that needs to be solved urgently. Summary of the invention

[0003] The embodiment of the present invention provides an image recognition method, which aims to provide an image recognition solution for children's learning and provide children with more in-depth and systematic knowledge. By obtaining the image to be recognized input by the user end, and based on the image to be recognized, the target subject category name corresponding to the target object is determined, and based on the target subject category name, the target subject classification tree corresponding to the target object is determined. The target subject classification tree includes target sub-classification items of the target object. The target subject classification tree can be used to generate knowledge data of different sub-classification items, which can provide users with system knowledge related to the target object, help users build a more complete and systematic knowledge system, and improve user experience.

[0004] In a first aspect, an embodiment of the present invention provides an image recognition method, the method comprising the following steps:

[0005] Obtain the image to be recognized input by the user;

[0006] Based on the image to be identified, determining a target subject category name corresponding to the target object;

[0007] Based on the target subject category name, determining a target subject classification tree corresponding to the target object, the target subject classification tree including target sub-classification items of the target object;

[0008] Based on the target subject classification tree, generating knowledge data of the target object;

[0009] The knowledge data is returned to the user end.

[0010] Optionally, generating the knowledge data of the target object based on the target subject classification tree includes:

[0011] Based on the target subject classification tree, determining a target sub-classification item of the target object;

[0012] Generate a target prompt word corresponding to the target sub-category item, and generate knowledge data of the target object according to the target prompt word.

[0013] Optionally, the generating a target prompt word corresponding to the target sub-category item and generating knowledge data of the target object according to the target prompt word includes:

[0014] Acquire a prompt word template corresponding to the target sub-classification item, where each target sub-classification item corresponds to one prompt word template;

[0015] Based on the prompt word template, generating a target prompt word corresponding to the target sub-category item;

[0016] Input the target prompt word into the large model to generate knowledge data corresponding to the target sub-classification item;

[0017] The knowledge data corresponding to each of the target sub-classification items in the target subject classification tree are integrated to obtain the knowledge data of the target object.

[0018] Optionally, determining a target subject category name corresponding to the target object based on the image to be identified includes:

[0019] Performing target recognition on the image to be recognized by an image recognition engine to obtain at least one recognized object and a subject category name corresponding to the recognized object;

[0020] Determine a target object in at least one of the identified objects;

[0021] Based on the subject category name of the identified object, a target subject category name corresponding to the target object is determined.

[0022] Optionally, determining a target subject classification tree corresponding to the target object based on the target subject category name includes:

[0023] Based on the target subject category name, determining the target standard name of the target object;

[0024] Based on the correspondence between the standard name and the subject classification tree, a target subject classification tree corresponding to the target object is determined, and each target standard name corresponds to one subject classification tree.

[0025] Optionally, before determining the target subject classification tree corresponding to the target object based on the target subject category name, the method further includes:

[0026] Collect name information of different objects;

[0027] Standardizing the name information to obtain a standard name of the object;

[0028] and, in the name information, determining an alias of the object;

[0029] Based on the standard name and the alias, a name database is constructed, and the name database is used to find out the standard name of the object according to the subject category name of the object.

[0030] Optionally, before determining the target subject classification tree corresponding to the target object based on the target subject category name, the method further includes:

[0031] Based on Nice classification or biological taxonomy, classify the objects to obtain a subject classification tree;

[0032] A mapping relationship between the standard name and the subject classification tree is established so as to find the corresponding subject classification tree through the standard name.

[0033] In a second aspect, an embodiment of the present invention further provides an image recognition device, the image recognition device comprising:

[0034] An acquisition module is used to acquire an image to be recognized input by a user terminal;

[0035] A recognition module, used to determine a target subject category name corresponding to a target object based on the image to be recognized;

[0036] A first processing module is used to determine a target subject classification tree corresponding to the target object based on the target subject category name, wherein the target subject classification tree includes target sub-classification items of the target object;

[0037] A generation module, used for generating knowledge data of the target object based on the target subject classification tree;

[0038] The returning module is used to return the knowledge data to the user end.

[0039] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the image recognition method provided in the embodiment of the present invention when executing the computer program.

[0040] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the image recognition method provided in the embodiment of the invention are implemented.

[0041] In an embodiment of the present invention, an image to be identified input by a user terminal is obtained; based on the image to be identified, a target subject category name corresponding to the target object is determined; based on the target subject category name, a target theme classification tree corresponding to the target object is determined, the target theme classification tree includes target sub-categories of the target object; based on the target theme classification tree, knowledge data of the target object is generated; and the knowledge data is returned to the user terminal. By obtaining an image to be identified input by a user terminal and determining the target subject category name corresponding to the target object based on the image to be identified, based on the target subject category name, a target theme classification tree corresponding to the target object is determined, the target theme classification tree includes target sub-categories of the target object, and using the target theme classification tree, knowledge data of different sub-categories can be generated, which can provide users with system knowledge related to the target object, help users build a more complete and systematic knowledge system, and improve user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 is a flow chart of an image recognition method provided by an embodiment of the present invention;

[0044] Figure 2 is a structural schematic diagram of an image recognition device provided by an embodiment of the present invention;

[0045] Figure 3 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] like Figure 1 As shown, Figure 1: is a flowchart of an image recognition method provided by an embodiment of the present invention, the image recognition method comprising the steps of:

[0048] 101. Obtain an image to be recognized input by a user.

[0049] In an embodiment of the present invention, the above-mentioned image recognition method can be applied to an image recognition platform, which can be constructed based on a server or a distributed server, and the above-mentioned image recognition platform includes an image interface (available for user upload), a knowledge database, and an image recognition program. The above-mentioned image interface can be used to obtain image data, which can be a separate image, or a continuous frame image or video stream data. The image recognition program can be used to implement each step of the image recognition method, and the above-mentioned knowledge database is specifically used to provide additional associated information for the identified image subject object, so as to enhance the depth of understanding of the content by the image recognition system.

[0050] The above-mentioned image to be identified includes multiple identifiable physical objects. The above-mentioned image to be identified can be image data uploaded by the user, or image data downloaded after the user provides a download address. The number of the above-mentioned images to be identified can be one or more. In the embodiment of the present invention, only one image to be identified is processed at a time.

[0051] The user terminal may be a device terminal for taking images, such as a smart phone, tablet, computer, camera, VR device, etc. The user takes a photo of an object of interest through the user terminal, and uploads the image obtained by taking the photo to the image recognition platform as an image to be recognized.

[0052] The above-mentioned objects may be animals, plants, commodities, articles, landscapes, scenes, texts and the like.

[0053] 102. Based on the image to be identified, determine the target subject category name corresponding to the target object.

[0054] In an embodiment of the present invention, after obtaining the image to be identified, the target subject category name corresponding to the target object can be determined through the intelligent object recognition engine. The above-mentioned intelligent recognition engine can be obtained based on a trained target recognition model or a large model. Specifically, the target recognition model can be used to perform target recognition on the image to be identified to obtain the target subject category name corresponding to the target object, or the image to be identified can be input into the large model, and the target object in the image to be identified can be identified through the large model, and the target subject category name corresponding to the target object can be generated and output to obtain the target subject category name corresponding to the target object.

[0055] The above target recognition model can be constructed based on a deep learning algorithm, for example, it can be based on deep learning models such as R-CNN, FasterR-CNN, YoLo, SSD, etc. The trained target recognition model can be obtained by training the target recognition model to be trained through the data set. The data set includes sample images and label data, the sample image includes an entity object, the label data is the category name corresponding to the entity object, the target recognition model to be trained outputs the category name, the sample image is input into the target recognition model to be trained, the target recognition model is used to perform target recognition processing, and the output category name corresponding to the entity object in the sample image is obtained. The error loss between the output category name and the label data is calculated by the loss function, and the minimum error loss is used as the optimization goal. The model parameters of the target recognition model to be trained are adjusted by using the back propagation algorithm, and the adjustment process of the above model parameters is iterated to a certain number of times, or after the error loss converges to a predetermined value, the training is completed to obtain a trained target recognition model.

[0056] The above-mentioned big model can be a CV (computer vision) big model or a multimodal big model. The CV (computer vision) big model or the multimodal big model can be a pre-trained big model, such as the GPT-4, GLM-4V, LLaVa, CLIP and other series of big models, or it can be the interface of a commercial big model, such as the interface of Tongyi Qianwen, Wenxin Yiyan and other big models.

[0057] For the pre-trained large model, the pre-trained large model can be adjusted through a pre-prepared data set so that the pre-trained large model can be more in line with the scenario of outputting the name of the entity object in this embodiment. The above-mentioned data set includes sample images and label data, the sample image includes an entity object, the above-mentioned label data is the name corresponding to the entity object, the pre-trained large model output is set to be the category name of the entity object, the sample image is input into the pre-trained large model, the output category name corresponding to the entity object in the sample image is generated by the pre-trained large model, the error loss between the output category name and the label data is calculated by the loss function, the minimum error loss is taken as the optimization goal, and the model parameters of the pre-trained large model are fine-tuned using the back propagation algorithm, and the fine-tuning process of the above-mentioned model parameters is iterated for a certain number of times, or after the error loss converges to a predetermined value, the adjustment of the pre-trained large model is completed.

[0058] For the interface of the commercial model, a preset prompt word can be used as the input of the commercial model, so that the commercial model can output the name of the entity object that conforms to this embodiment. For example, the prompt word can be designed as:

[0059] “You are a classification expert who can accurately classify the physical objects in the image and know the category names of the physical objects under different classification systems.

[0060] Provide a picture: XXX.

[0061] Please identify the entity objects in the image and output the category names of the entity objects in the image. "

[0062] Through targeted prompt words, based on the interface of the commercial big model, the commercial big model can be guided to output the name of the entity object that conforms to the present embodiment as the target subject category name corresponding to the target object.

[0063] By processing the image to be identified through the smart object recognition engine, the target subject category name corresponding to the target object can be obtained. The smart object recognition engine uses advanced image recognition technology to accurately identify the subject in the picture taken by children and match it with the category in the database. The smart object recognition engine can use a multimodal large model as the recognition model to identify the object in the image to be identified.

[0064] In a possible embodiment, if the intelligent object recognition engine recognizes multiple objects, the multiple objects can be marked in the image to be recognized, returned to the user end for display, and the user is prompted to select an object as the target object. After the user selects an object, the user end sends the selection result to the image recognition platform. The image recognition platform determines the object selected by the user as the target object and obtains the target subject category name of the target object.

[0065] In a possible embodiment, if the intelligent object recognition engine recognizes multiple objects, the multiple objects can be marked in the image to be recognized, returned to the user end for display, and the user is prompted to select multiple objects as target objects. After the user selects multiple objects, the user end sends the selection results to the image recognition platform. The image recognition platform determines the object selected by the user as the target object and obtains the target subject category name of the target object.

[0066] In a possible embodiment, if the intelligent object recognition engine recognizes multiple objects, the most important object can be selected from the multiple objects to be determined as the target object. For example, the object with the highest confidence level can be selected as the target object, or the object closest to the center of the image can be selected as the target object, or the object with the largest area can be selected as the target object.

[0067] It should be noted that the subject category name is the category name output by the smart object recognition engine. The category name is related to the training data. If the training data uses a standard name, the category name output by the smart object recognition engine is the standard name. If the training data uses a non-standard name, the category name output by the smart object recognition engine is the non-standard name. For an object, both the standard name and the non-standard name point to the object.

[0068] 103. Based on the target subject category name, determine the target subject classification tree corresponding to the target object.

[0069] In an embodiment of the present invention, each subject category name will correspond to a subject classification tree, and one subject classification tree can correspond to multiple subject category names. After identifying the target subject category name corresponding to the target object, the corresponding subject classification tree can be found according to the target subject category as the target subject classification tree corresponding to the target object.

[0070] The target subject classification tree includes target sub-classification items of the target object. In the target subject classification tree to which the target object belongs, one or more target sub-classification items of the target object can be found.

[0071] It should be noted that the above subject classification tree adopts, including but not limited to, the Nice Classification based on commodities and biological taxonomy to ensure the scientificity and systematicity of the classification. Binding the subject category to the subject classification tree ensures that each subject category can find a corresponding position in the classification tree and belongs to a certain sub-classification item, thereby providing a clear learning path for children.

[0072] For example, in the commodity classification, there may be a major category of "electronic products", which can be divided into subcategories such as "mobile phones", "televisions", "computers", etc. By querying the database or using other algorithms, the subject category name is mapped to a specific node on the subject classification tree to determine the precise classification of the object.

[0073] 104. Generate knowledge data of the target object based on the target subject classification tree.

[0074] In the embodiment of the present invention, after obtaining the target subject classification tree, the knowledge data corresponding to the target sub-classification item can be queried in the knowledge database based on the target subject classification tree, thereby obtaining the knowledge data of the target object. It is also possible to construct prompt words for the target sub-classification item, input the prompt words of the target sub-classification item into the large model, and obtain the knowledge data corresponding to the target sub-classification item output by the large model.

[0075] If the target object corresponds to only one target sub-category, the knowledge data of the target sub-category can be used as the knowledge data of the target object. If the target object corresponds to multiple target sub-category, the knowledge data of the multiple target sub-category can be integrated to obtain the knowledge data of the target object.

[0076] The above-mentioned knowledge data may be the description, purpose, characteristics, history, etc. of the object. The above-mentioned knowledge data may exist in various forms such as text, image, video, etc., aiming to provide users with rich and comprehensive knowledge content.

[0077] 105. Return the knowledge data to the user end.

[0078] In an embodiment of the present invention, after obtaining the knowledge data of the target object, the knowledge data of the target object is returned to the user end, and the target object and the knowledge data corresponding to the target object are displayed on the user end, so that the user can systematically understand the target object through the knowledge data displayed on the user end.

[0079] Taking the Paixue machine as the user end of a child user as an example, suppose a child uses the Paixue machine to take a photo of a dog and uploads it. The image recognition platform first recognizes that the subject in the photo is a dog and gives the subject category name "dog". Then, by querying the subject classification tree, the system determines that the dog belongs to the "Canidae" subcategory under the "Mammals" category. Next, the knowledge data related to "Canidae" is retrieved from the database, such as the dog's breed, habits, breeding methods, etc., and this knowledge data is integrated into an easy-to-understand form. Finally, the final knowledge data is returned to the user end of the child user, and the child user can view or listen to this information through the device to enhance his understanding of dogs.

[0080] In an embodiment of the present invention, an image to be identified input by a user terminal is obtained; based on the image to be identified, a target subject category name corresponding to the target object is determined; based on the target subject category name, a target theme classification tree corresponding to the target object is determined, the target theme classification tree includes target sub-categories of the target object; based on the target theme classification tree, knowledge data of the target object is generated; and the knowledge data is returned to the user terminal. By obtaining an image to be identified input by a user terminal and determining the target subject category name corresponding to the target object based on the image to be identified, based on the target subject category name, a target theme classification tree corresponding to the target object is determined, the target theme classification tree includes target sub-categories of the target object, and using the target theme classification tree, knowledge data of different sub-categories can be generated, which can provide users with system knowledge related to the target object, help users build a more complete and systematic knowledge system, and improve user experience.

[0081] It is understandable that in the specific implementation of this application, related data such as image data, knowledge data, user data, etc. are involved. When the embodiments in this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data, as well as the training, deployment and calling of algorithm models, must comply with relevant laws, regulations and standards of relevant countries and regions.

[0082] Optionally, in the step of generating knowledge data of the target object based on the target subject classification tree, target sub-classification items of the target object can be determined based on the target subject classification tree; target prompt words corresponding to the target sub-classification items are generated, and the knowledge data of the target object is generated based on the target prompt words.

[0083] In an embodiment of the present invention, a pre-constructed subject classification tree is relied upon to determine the target sub-classification item. The above subject classification tree can comprehensively consider multiple classification systems, such as the Nice Classification of commodities and the biological taxonomy of organisms, thereby ensuring the scientificity and systematicity of the classification.

[0084] After identifying the target object in the image to be identified, the subject category name of the object can be used to search for the corresponding category name in the name database. The category name is the standard name of the object. The standard name can be formulated by experts or by definitions in teaching materials. After obtaining the category name corresponding to the subject category name, the subject tree corresponding to the category name can be searched. Each category name can correspond to a subject classification tree. Find the sub-category item to which the object belongs in the subject classification tree. This allows the identified target objects to be accurately classified into the corresponding sub-categories. A target object can correspond to at least one sub-category item.

[0085] Once the target sub-categories are determined, a large language model can be used to generate prompt words for each sub-categories to obtain target prompt words for the target sub-categories. Each target sub-categories can correspond to a target prompt word. These templates are customized by education experts based on the characteristics of the sub-categories or automatically generated by a large model.

[0086] After the target sub-category item is determined, the target prompt word can also be generated based on the corresponding prompt word template. The target prompt word can be used to guide the multimodal large model to generate corresponding content, such as guiding the multimodal large model to generate content that not only meets the characteristics of the sub-category but also meets the learning needs of children as knowledge data. The above prompt word template can be generated based on the large language model or formulated by educational experts.

[0087] For example, in the pet sub-category, prompt words may include "appearance characteristics" and "breeding precautions", etc., to guide the multimodal large model to generate knowledge data about pet appearance descriptions and breeding methods. In the electrical appliance sub-category, prompt words can focus on "related functions" and "safety precautions" to ensure that the generated content can help children understand the use and safety specifications of electrical appliances.

[0088] When generating knowledge data, the target prompt words can be input into the multimodal large model. After training and generalization, the multimodal large model can generate corresponding multimodal knowledge data such as text, images, and sounds according to the input prompt words. The above multimodal knowledge data is not only rich and diverse, but also can be personalized according to children's learning needs and interests. In the case where a target object corresponds to multiple sub-categories, the knowledge data corresponding to each target sub-categories in the target subject classification tree can be integrated to form the complete knowledge data of the target object.

[0089] Optionally, in the step of generating target prompt words corresponding to target sub-classification items and generating knowledge data of the target object based on the target prompt words, a prompt word template corresponding to the target sub-classification items can be obtained, and each target sub-classification item corresponds to a prompt word template; based on the prompt word template, the target prompt words corresponding to the target sub-classification items are generated; the target prompt words are input into the large model to generate knowledge data corresponding to the target sub-classification items; the knowledge data corresponding to each target sub-classification item in the target subject classification tree are integrated to obtain the knowledge data of the target object.

[0090] In the embodiment of the present invention, the prompt word template is pre-set, and each target sub-classification item has a corresponding template. The prompt word template corresponding to the target sub-classification item can be obtained according to the corresponding relationship between the sub-classification item and the prompt word template. The prompt word template can be generated based on a large language model or formulated by an educational expert. The design of the prompt word template can take into account the characteristics of the sub-classification and the educational goal, aiming to guide the generation of subsequent knowledge data.

[0091] After obtaining the prompt word template, the target prompt word corresponding to the target sub-category item can be generated based on the prompt word template, and the information of the target sub-category item can be filled into the prompt word template to obtain the target prompt word. The above target prompt words are used to guide the generation direction and focus of knowledge data. For example, in the pet sub-category, the target prompt words may include "pet appearance", "breeding method", etc. The above prompt words will guide the system to generate knowledge data about pet appearance description and breeding skills.

[0092] The target prompt word is input into the multimodal large model to generate multimodal knowledge data corresponding to the target sub-classification item. The above multimodal large model can be understood as a deep learning model with strong generation capabilities after being trained with a large amount of multimodal data. It can generate multimodal knowledge data such as text, pictures, and videos related to the input prompt word. Multimodal knowledge data is not only rich in content, but also diverse in form, which can further meet the diverse needs of children's learning and improve the learning experience of children users.

[0093] When a target object corresponds to multiple sub-categories, the knowledge data corresponding to each target sub-categories in the target subject classification tree can be integrated to form the complete knowledge data of the target object. In this way, it can be ensured that the generated knowledge data is both comprehensive and coherent, and can form a complete knowledge system. In turn, it can provide children with a more systematic and comprehensive learning experience.

[0094] Optionally, in the step of determining a target subject category name corresponding to a target object based on an image to be identified, the image to be identified can be subjected to target recognition by an image recognition engine to obtain at least one identified object and a subject category name corresponding to the identified object; a target object is determined based on at least one identified object; and a target subject category name corresponding to the target object is determined based on the subject category name of the identified object.

[0095] In an embodiment of the present invention, the above-mentioned image recognition engine may also be referred to as a smart object recognition engine. The smart object recognition engine may be constructed based on a deep learning algorithm, and may be capable of identifying objects in an image and giving a subject category name for the classified object. The above-mentioned smart object recognition engine may also be a multimodal large model or a visual large model, and by training the large model to output subject category names, it may be capable of accurately identifying subject category names corresponding to subject objects in pictures taken by children.

[0096] After the recognition is completed, at least one recognized object and its corresponding subject category name can be obtained, and each recognized object corresponds to a subject category name. The above-mentioned recognized object may be the main object in the image, or it may be the background or a secondary object. In order to ensure the accuracy of subsequent processing, if the image recognition obtains multiple recognized objects, one recognized object can be determined from the multiple recognized objects as the target object; if the image recognition obtains one recognized object, the recognized object is directly determined as the target object. In the case where the image recognition obtains multiple recognized objects, it can be achieved by analyzing factors such as the confidence, size, and position of the recognized objects. For example, the object with the highest confidence can be selected as the target object, the object closest to the center of the image can be selected as the target object, and the object with the largest area can be selected as the target object, and the target subject category name of the target object can be obtained.

[0097] Of course, when multiple objects are identified through image recognition, multiple objects can also be marked in the image to be identified and returned to the user end for display, and the user can be prompted to select multiple objects as target objects. After the user selects multiple objects, the user end sends the selection results to the image recognition platform. The image recognition platform determines the object selected by the user as the target object and obtains the target subject category name of the target object.

[0098] For example, suppose a child user uses a camera to take a photo of a variety of fruits, in which the apple occupies the center of the image and has the largest area. The smart object recognition engine may give multiple subject category names such as "apple", "fruit", "red object" and so on during the recognition process. The confidence of each of the above objects, the distance from the center of the image, and at least one of the area proportion in the image can be analyzed and compared. If the confidence of the apple is the highest, the apple can be determined as the target object, and the subject category name "apple" can be determined as the target subject category name; or, if the apple is found to be the object closest to the center of the image, the apple can be determined as the target object, and the subject category name "apple" can be determined as the target subject category name; or, if the apple is found to be the object with the largest area proportion, the apple can be determined as the target object, and the subject category name "apple" can be determined as the target subject category name; or, based on the confidence, the distance from the center of the image and the area proportion in the image, a weighted integral calculation is performed to obtain the weighted integral sum of each object, and the object with the highest weighted integral sum is taken as the target object.

[0099] Optionally, in the step of determining the target subject classification tree corresponding to the target object based on the target subject category name, the target standard name of the target object can be determined based on the target subject category name; based on the correspondence between the standard name and the subject classification tree, the target subject classification tree corresponding to the target object is determined, and each target standard name corresponds to a subject classification tree.

[0100] In the embodiments of the present invention, it should be noted that the target subject category name can be a standard name or an alias, which is specifically related to the data set used to train the intelligent object recognition engine. If the standard name is used as the output label in the data set, the intelligent object recognition engine will output the standard name as the subject category name. If the alias is used as the output label in the data set, the intelligent object recognition engine will output the alias as the subject category name. If the data set uses a mixture of standard names and aliases as output labels, the intelligent object recognition engine may output the standard name or the alias.

[0101] After obtaining the target subject category name, the target subject category name can be processed into a target standard name through a large language model, or a target standard name corresponding to the target subject category name can be matched by performing name matching in a name database.

[0102] In the name database, various objects can be stored, and each object stores a standard name and at least one alias. In the process of name matching, the target subject category name can be compared with all standard names first. If a standard name that is the same as the target subject category name is matched, the matched standard name is determined as the target standard name; if a standard name that is the same as the target subject category name is not matched, the target subject category name can be matched with all aliases. If an alias that is the same as the target subject category name is matched, the standard name corresponding to the alias is determined as the target standard name; if an alias that is the same as the target subject category name is not matched, the similarity between the target subject category name and each alias is calculated. The standard name corresponding to the alias with the greatest similarity is determined as the target standard name. In a possible embodiment, if no alias identical to the target subject category name is matched, a first similarity between the target subject category name and each alias is calculated, and then a second similarity between the target subject category name and each standard name is calculated. For a standard name, a weighted average similarity between the first similarity and the second similarity is calculated to obtain a weighted average similarity between the target subject category name and the standard name, and the standard name with the largest weighted average similarity is determined as the target standard name.

[0103] The above-mentioned subject classification tree is pre-constructed, and each standard name corresponds to a subject classification tree. After obtaining the target standard name, the target subject classification tree corresponding to the target object can be determined based on the correspondence between the standard name and the subject classification tree.

[0104] Optionally, before determining the target subject classification tree corresponding to the target object based on the target subject category name, name information of different objects can be collected; the name information can be standardized to obtain the standard name of the object; and, in the name information, the alias of the object can be determined; based on the standard name and the alias, a name database can be constructed, and the name database is used to find the standard name of the object according to the subject category name of the object.

[0105] In the embodiment of the present invention, the name database not only stores the standard names of various objects, but also includes their aliases, so that the breadth and accuracy of the identification process can be improved.

[0106] Specifically, the name information of different objects may be collected through public information, professional literature, network resources and other channels, and the above name information may be various titles of the objects.

[0107] After collecting the name information of different objects, the information can be standardized. The purpose of standardization is to eliminate ambiguity, irregularity and duplication in the name, and ensure that each object has a unique and accurate standard name. For example, for animals, "dog" may be commonly known as "dog". In the standardization process, "dog" can be selected as its standard name, and "dog" can be recorded as its alias. The standardization process can be based on naming rules and expert knowledge to ensure the scientificity and universality of the standard name.

[0108] While determining the standard name, you can also filter out the aliases of the object from the collected name information. Aliases are other names used to describe the same object in addition to the standard name. The existence of aliases increases the flexibility and inclusiveness of the name database and improves the accuracy of the recognition process. For example, the aliases of the object "computer" may include "computer", "laptop", etc.

[0109] After the standard name and alias are determined, a name database can be constructed based on this information. The name database establishes a mapping relationship from the standard name of the object to its alias, so that in the subsequent identification process, no matter whether the subject classification name is a standard name or an alias, the system can quickly and accurately find the standard name corresponding to the subject classification name, and then determine the corresponding subject classification tree.

[0110] It should be noted that the construction and maintenance of the name database is an ongoing process. As new objects appear and the names of old objects change, the name database can be continuously updated to maintain its timeliness and accuracy.

[0111] For example, suppose you want to build an object recognition system for children, which includes a large number of objects such as animals, plants, and daily necessities. When collecting name information, you may find that the animal "cat" has many nicknames in different regions and cultures, such as "cat" and "meow star". Through standardization, we determine "cat" as its standard name and record other names as aliases. When children use a camera to take a picture of a cat and identify it, whether they enter "cat" or its alias, they can accurately identify the target object in the name database.

[0112] Optionally, before determining the target subject classification tree corresponding to the target object based on the target subject category name, the object can also be classified based on the Nice classification or biological taxonomy to obtain a subject classification tree; a mapping relationship between the standard name and the subject classification tree is established to find the corresponding subject classification tree through the standard name.

[0113] In the embodiment of the present invention, after collecting information of different objects, these objects are classified according to the information of the objects, and the objects can be classified based on the Nice classification or biological taxonomy, or based on other classification systems.

[0114] The Nice Classification is a classification system established by the Nice Agreement Concerning the International Classification of Goods and Services for the Purposes of the Registration of Marks. It is mainly used to classify goods and services for the convenience of trademark registration and management. Biological taxonomy is the science of studying the classification, identification and naming of organisms in biology. It divides organisms into different levels, such as species, genus, family, order, class, phylum, etc., based on the similarities and relationships between organisms.

[0115] When constructing a subject classification tree, you can choose an appropriate classification method based on the different characteristics of the identification object. If the identification object is a non-biological entity such as a commodity, the Nice classification system can be used to classify the commodity according to its properties, functions, uses and other attributes. For example, electrical appliances, furniture, toys, etc. will be classified into different Nice classification categories to ensure the scientific and systematic classification of commodities. If the identification object is a biological entity, such as animals, plants, etc., it is classified based on biological taxonomy. Biological taxonomy can consider the external morphological characteristics of organisms, and can also comprehensively consider multiple factors such as their physiological structure and genetic information. For example, cats can be further subdivided into the genera Felis and Panthera, and each genus contains multiple species.

[0116] After determining the classification method, you can start to build a subject classification tree. The subject classification tree can be understood as a hierarchical structure, where each node represents a classification level, starting from the root node and gradually refining until the required classification accuracy is achieved. Each leaf node represents a specific identification object or its subclassification. By traversing this subject classification tree, you can find the position of any identification object in the classification system.

[0117] After obtaining the subject classification tree, a mapping relationship between the standard name and the subject classification tree can be established. The standard name is the name of the identified object after standardization, which can uniquely identify an object. When constructing the name database, a standard name has been assigned to each identified object. Furthermore, these standard names can also be associated with the corresponding nodes in the subject classification tree.

[0118] Specifically, a database table containing fields such as standard name, classification level, and node ID can be created. Whenever a new identification object is added, its standard name and corresponding classification information need to be added to this database table.

[0119] In actual applications, when an image to be identified is received from the user, the subject category name of the identified object can be obtained through the intelligent object recognition engine. The subject category name can be converted into a standard name using the name database, and then the corresponding classification information can be found using the mapping relationship. In this way, the corresponding knowledge data can be generated based on the classification information and returned to the user.

[0120] like Figure 2 As shown, an embodiment of the present invention provides an image recognition device, the image recognition device comprising:

[0121] The acquisition module 201 is used to acquire the image to be recognized input by the user terminal;

[0122] The recognition module 202 is used to determine the target subject category name corresponding to the target object based on the image to be recognized;

[0123] A first processing module 203 is used to determine a target subject classification tree corresponding to the target object based on the target subject category name, wherein the target subject classification tree includes target sub-classification items of the target object;

[0124] A generating module 204, configured to generate knowledge data of the target object based on the target subject classification tree;

[0125] The return module 205 is used to return the knowledge data to the user end.

[0126] Optionally, the generation module 204 is further used to determine the target sub-classification items of the target object based on the target subject classification tree; generate target prompt words corresponding to the target sub-classification items, and generate knowledge data of the target object according to the target prompt words.

[0127] Optionally, the generation module 204 is also used to obtain a prompt word template corresponding to the target sub-classification item, each target sub-classification item corresponds to a prompt word template; based on the prompt word template, generate a target prompt word corresponding to the target sub-classification item; input the target prompt word into the large model to generate knowledge data corresponding to the target sub-classification item; integrate the knowledge data corresponding to each of the target sub-classification items in the target subject classification tree to obtain the knowledge data of the target object.

[0128] Optionally, the recognition module 202 is also used to perform target recognition on the image to be recognized through an image recognition engine to obtain at least one recognized object and a subject category name corresponding to the recognized object; determine a target object based on at least one of the recognized objects; and determine a target subject category name corresponding to the target object based on the subject category name of the recognized object.

[0129] Optionally, the first processing module 203 is also used to determine the target standard name of the target object based on the target subject category name; based on the correspondence between the standard name and the subject classification tree, determine the target subject classification tree corresponding to the target object, each target standard name corresponds to one subject classification tree.

[0130] Optionally, the device further comprises:

[0131] A collection module is used to collect name information of different objects;

[0132] A second processing module is used to standardize the name information to obtain a standard name of the object;

[0133] and, in the name information, determining an alias of the object;

[0134] A construction module is used to construct a name database based on the standard name and the alias, and the name database is used to find out the standard name of the object according to the subject category name of the object.

[0135] Optionally, before determining the target subject classification tree corresponding to the target object based on the target subject category name, the method further includes:

[0136] A classification module, used for classifying the object based on Nice classification or biological taxonomy to obtain a subject classification tree;

[0137] A mapping module is used to establish a mapping relationship between a standard name and the subject classification tree so as to find the corresponding subject classification tree through the standard name.

[0138] like Figure 3 As shown, an embodiment of the present invention further provides an electronic device, including a processor, and the processor can execute any one of the above-mentioned image recognition methods.

[0139] Specifically, it includes a processor 301 and a memory 302, and a computer program for executing the image recognition method stored in the memory 302 and capable of running on the processor 301, wherein:

[0140] The processor 301 runs the computer program of the image recognition method stored in the memory 302 to perform the following steps:

[0141] Obtain the image to be recognized input by the user;

[0142] Based on the image to be identified, determining a target subject category name corresponding to the target object;

[0143] Based on the target subject category name, determining a target subject classification tree corresponding to the target object, the target subject classification tree including target sub-classification items of the target object;

[0144] Based on the target subject classification tree, generating knowledge data of the target object;

[0145] The knowledge data is returned to the user end.

[0146] Optionally, the step of generating the knowledge data of the target object based on the target subject classification tree performed by the processor 301 includes:

[0147] Based on the target subject classification tree, determining a target sub-classification item of the target object;

[0148] Generate a target prompt word corresponding to the target sub-category item, and generate knowledge data of the target object according to the target prompt word.

[0149] Optionally, the processor 301 generates a target prompt word corresponding to the target sub-category item, and generates knowledge data of the target object according to the target prompt word, including:

[0150] Acquire a prompt word template corresponding to the target sub-classification item, where each target sub-classification item corresponds to one prompt word template;

[0151] Based on the prompt word template, generating a target prompt word corresponding to the target sub-category item;

[0152] Input the target prompt word into the large model to generate knowledge data corresponding to the target sub-classification item;

[0153] The knowledge data corresponding to each of the target sub-classification items in the target subject classification tree are integrated to obtain the knowledge data of the target object.

[0154] Optionally, the processor 301 determines, based on the image to be identified, a target subject category name corresponding to the target object, including:

[0155] Performing target recognition on the image to be recognized by an image recognition engine to obtain at least one recognized object and a subject category name corresponding to the recognized object;

[0156] Determining a target object based on at least one of the identified objects;

[0157] Based on the subject category name of the identified object, a target subject category name corresponding to the target object is determined.

[0158] Optionally, the processor 301 determines, based on the target subject category name, a target subject classification tree corresponding to the target object, including:

[0159] Based on the target subject category name, determining the target standard name of the target object;

[0160] Based on the correspondence between the standard name and the subject classification tree, a target subject classification tree corresponding to the target object is determined, and each target standard name corresponds to one subject classification tree.

[0161] Optionally, before determining the target subject classification tree corresponding to the target object based on the target subject category name, the method executed by the processor 301 further includes:

[0162] Collect name information of different objects;

[0163] Standardizing the name information to obtain a standard name of the object;

[0164] and, in the name information, determining an alias of the object;

[0165] Based on the standard name and the alias, a name database is constructed, and the name database is used to find out the standard name of the object according to the subject category name of the object.

[0166] Optionally, before determining the target subject classification tree corresponding to the target object based on the target subject category name, the method executed by the processor 301 further includes:

[0167] Based on Nice classification or biological taxonomy, classify the objects to obtain a subject classification tree;

[0168] If the pair is an organism, the classification information is classified based on biological taxonomy to obtain a subject classification tree;

[0169] A mapping relationship between the standard name and the subject classification tree is established so as to find the corresponding subject classification tree through the standard name.

[0170] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the image recognition method provided by the embodiment of the present invention are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0171] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0172] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. An image recognition method, characterized in that: The method comprises the following steps: Obtain the image to be recognized input by the user; Based on the image to be identified, determining a target subject category name corresponding to the target object; Based on the target subject category name, determining a target subject classification tree corresponding to the target object, the target subject classification tree including target sub-classification items of the target object; Based on the target subject classification tree, generating knowledge data of the target object; The knowledge data is returned to the user end.

2. The image recognition method according to claim 1, characterized in that: The step of generating the knowledge data of the target object based on the target subject classification tree includes: Based on the target subject classification tree, determining a target sub-classification item of the target object; Generate a target prompt word corresponding to the target sub-category item, and generate knowledge data of the target object according to the target prompt word.

3. The image recognition method according to claim 2, characterized in that: The step of generating a target prompt word corresponding to the target sub-category item and generating knowledge data of the target object according to the target prompt word includes: Acquire a prompt word template corresponding to the target sub-classification item, where each target sub-classification item corresponds to one prompt word template; Based on the prompt word template, generating a target prompt word corresponding to the target sub-category item; Input the target prompt word into the large model to generate knowledge data corresponding to the target sub-classification item; The knowledge data corresponding to each of the target sub-classification items in the target subject classification tree are integrated to obtain the knowledge data of the target object.

4. The image recognition method according to claim 1, characterized in that: The step of determining a target subject category name corresponding to the target object based on the image to be identified includes: Performing target recognition on the image to be recognized by an image recognition engine to obtain at least one recognized object and a subject category name corresponding to the recognized object; Determine a target object in at least one of the identified objects; Based on the subject category name of the identified object, a target subject category name corresponding to the target object is determined.

5. The image recognition method according to claim 4, characterized in that: The step of determining a target subject classification tree corresponding to the target object based on the target subject category name includes: Based on the target subject category name, determining the target standard name of the target object; Based on the correspondence between the standard name and the subject classification tree, a target subject classification tree corresponding to the target object is determined, and each target standard name corresponds to one subject classification tree.

6. The image recognition method according to claim 5, characterized in that: Before determining the target subject classification tree corresponding to the target object based on the target subject category name, the method further includes: Collect name information of different objects; Standardizing the name information to obtain a standard name of the object; and, in the name information, determining an alias of the object; Based on the standard name and the alias, a name database is constructed, and the name database is used to find out the standard name of the object according to the subject category name of the object.

7. The image recognition method according to claim 6, characterized in that: Before determining the target subject classification tree corresponding to the target object based on the target subject category name, the method further includes: Based on Nice classification or biological taxonomy, classify the objects to obtain a subject classification tree; A mapping relationship between the standard name and the subject classification tree is established so as to find the corresponding subject classification tree through the standard name.

8. An image recognition device, characterized in that: The image recognition device comprises: An acquisition module is used to acquire an image to be recognized input by a user terminal; A first processing module, used to determine a target subject category name corresponding to a target object based on the image to be identified; A second processing module is used to determine a target subject classification tree corresponding to the target object based on the target subject category name, wherein the target subject classification tree includes target sub-classification items of the target object; A generation module, used for generating knowledge data of the target object based on the target subject classification tree; The returning module is used to return the knowledge data to the user end.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps in the image recognition method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the image recognition method according to any one of claims 1 to 7 are implemented.

Images