Object recognition method and device and electronic equipment

Abstract

The embodiment of the invention provides an object recognition method and device and electronic equipment, and relates to the technical field of image recognition, and the method comprises the steps: obtaining a target image to be subjected to object recognition; inputting the target image into a deep learning model obtained by pre-training to obtain an output result of the deep learning model; obtaining description information corresponding to the target object category; based on an image region indicated by the object region information and description information corresponding to the target object category, analyzing whether the to-be-identified object belongs to an object described by the description information, and obtaining an analysis result; and if the analysis result represents that the to-be-recognized object belongs to the object described by the description information, taking the output result as an object recognition result of the to-be-recognized object in the target image. Through the scheme, the object identification accuracy can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, in particular to an object recognition method and device and electronic equipment. BACKGROUND

[0002] At present, deep learning technology is widely used in the field of image recognition. In the training process of a deep learning model for object recognition, a large number of training samples are usually input into the deep learning model, so that the deep learning model obtained by training can perform object recognition on the received image, and the object recognition can be the recognition of the class to which the object belongs.

[0003] In actual application, training samples of objects in some classes can be difficult to obtain, resulting in a lack of training samples of objects in these classes in the model training process. Thus, when the deep learning model trained is used to perform object recognition, the recognition accuracy of the objects is low.

[0004] Therefore, how to improve the recognition accuracy of objects has become a technical problem to be solved. SUMMARY

[0005] The embodiments of the present application aim to provide an object recognition method, device and electronic equipment to improve the recognition accuracy of objects. The specific technical solutions are as follows:

[0006] In a first aspect, the embodiments of the present application provide an object recognition method, which comprises:

[0007] obtaining a target image to be recognized;

[0008] inputting the target image into a pre-trained deep learning model to obtain an output result of the deep learning model; wherein the output result comprises object region information of a to-be-recognized object in the target image, and a class recognition result of the to-be-recognized object; the class recognition result comprises a target object class to which the to-be-recognized object belongs;

[0009] obtaining description information corresponding to the target object class; wherein the description information corresponding to the target object class is used to describe the object features of the object belonging to the target object class;

[0010] based on the image region indicated by the object region information and the description information corresponding to the target object class, analyzing whether the to-be-recognized object belongs to the object described by the description information to obtain an analysis result;

[0011] If the analysis result indicates that the object to be identified belongs to the object described by the descriptive information, then the output result is used as the object identification result of the object to be identified in the target image.

[0012] Secondly, embodiments of this application provide an object recognition device, the device comprising:

[0013] The first acquisition module is used to acquire the target image to be recognized;

[0014] The model processing module is used to input the target image into a pre-trained deep learning model to obtain the output result of the deep learning model; wherein, the output result includes object region information of the object to be identified in the target image, and category identification result for the object to be identified; the category identification result includes the target object category to which the object to be identified belongs;

[0015] The second acquisition module is used to acquire description information corresponding to the target object category; wherein, the description information corresponding to the target object category is used to describe the object characteristics of objects belonging to the target object category;

[0016] The object analysis module is used to analyze whether the object to be identified belongs to the object described by the description information based on the image region indicated by the object region information and the description information corresponding to the target object category, and to obtain the analysis result.

[0017] The first determining module is used to take the output result as the object recognition result of the object to be identified in the target image if the analysis result indicates that the object to be identified belongs to the object described by the description information.

[0018] Thirdly, embodiments of this application provide an electronic device, including: a memory for storing computer programs; and a processor for implementing any of the object recognition methods described above when executing the program stored in the memory.

[0019] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computing program, which, when executed by a processor, implements any of the object recognition methods described above.

[0020] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute any of the object recognition methods described above.

[0021] Beneficial effects of the embodiments in this application:

[0022] In an object recognition method provided in this application embodiment, a pre-trained deep learning model is used to recognize objects in a target image. This allows for the acquisition of object region information and category recognition results for the object to be recognized. Based on the target object category to which the object belongs, as contained in the category recognition results, descriptive information corresponding to that category is obtained. This descriptive information describes the object features of objects belonging to that category. Thus, based on the image region indicated by the object region information and the acquired descriptive information, it is possible to further analyze whether the object to be recognized belongs to the object described by the descriptive information, obtaining an analysis result. When the analysis result indicates that the object to be recognized belongs to the object described by the descriptive information, the output of the deep learning model is used as the object recognition result of the object to be recognized in the target image.

[0023] In this way, during the object recognition process, by combining the acquired descriptive information and the image region indicated by the object region information, auxiliary reasoning is performed to determine the object category to which the object to be identified in the target image belongs. This further determines the accuracy of the target object category in the output of the deep learning model. Thus, when the target object category is a scarce object category, the supplementary reasoning based on the above descriptive information provides sufficient reasoning information for the object recognition process, thereby improving the accuracy of object recognition. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0025] Figure 1 A flowchart illustrating the first object recognition method provided in this application embodiment;

[0026] Figure 2 A flowchart illustrating the second object recognition method provided in this application embodiment;

[0027] Figure 3 A schematic diagram of an object recognition architecture provided in an embodiment of this application;

[0028] Figure 4 This is a schematic diagram of the structure of an object recognition system provided in an embodiment of this application;

[0029] Figure 5 This is a schematic diagram of the structure of an object recognition device provided in an embodiment of this application;

[0030] Figure 6This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0031] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0032] To address the aforementioned problem of how to improve the accuracy of object recognition, embodiments of this application provide an object recognition method, apparatus, and electronic device.

[0033] This method is applicable to various application scenarios for object recognition. For example, in a maritime scenario, it identifies the type of ship in an image; in this case, the object is a ship, and the object category is ship. Similarly, in a road traffic scenario, it identifies the type of vehicle in a video frame; in this case, the object is a vehicle, and the object category is vehicle. Furthermore, this method can be applied to various electronic devices such as laptops and desktop computers (hereinafter referred to as electronic devices). Therefore, this application does not limit the application scenarios or the executing entity of this method.

[0034] An object recognition method provided in this application embodiment may include the following steps:

[0035] Acquire the target image to be recognized;

[0036] The target image is input into a pre-trained deep learning model to obtain the output of the deep learning model; wherein, the output includes object region information of the object to be identified in the target image, and category identification result for the object to be identified; the category identification result includes the target object category to which the object to be identified belongs;

[0037] Obtain the description information corresponding to the target object category; wherein, the description information corresponding to the target object category is used to describe the object characteristics of objects belonging to the target object category;

[0038] Based on the image region indicated by the object region information and the description information corresponding to the target object category, analyze whether the object to be identified belongs to the object described by the description information, and obtain the analysis result;

[0039] If the analysis result indicates that the object to be identified belongs to the object described by the descriptive information, then the output result is used as the object identification result of the object to be identified in the target image.

[0040] As can be seen from the above, in the object recognition method provided in this application embodiment, object recognition of a target image based on a pre-trained deep learning model can obtain object region information of the object to be recognized in the target image, as well as the category recognition result for the object to be recognized. Then, based on the target object category to which the object to be recognized belongs, as contained in the category recognition result, descriptive information corresponding to that target object category is obtained. This descriptive information is used to describe the object features of objects belonging to that target object category. Thus, based on the image region indicated by the aforementioned object region information and the obtained descriptive information, it is possible to further analyze whether the object to be recognized belongs to the object described by the aforementioned descriptive information, obtain an analysis result, and when the obtained analysis result indicates that the object to be recognized belongs to the object described by the descriptive information, the output result of the deep learning model is used as the object recognition result of the object to be recognized in the target image.

[0041] In this way, during the object recognition process, by combining the acquired descriptive information and the image region indicated by the object region information, auxiliary reasoning is performed to determine the object category to which the object to be identified in the target image belongs. This further determines the accuracy of the target object category in the output of the deep learning model. Thus, when the target object category is a scarce object category, the supplementary reasoning based on the above descriptive information provides sufficient reasoning information for the object recognition process, thereby improving the accuracy of object recognition.

[0042] The following description, in conjunction with the accompanying drawings, details an object recognition method provided in an embodiment of this application.

[0043] Figure 1 This is a flowchart illustrating an object recognition method provided in an embodiment of this application, as shown below. Figure 1 As shown, the method includes the following steps:

[0044] S101: Obtain the target image to be recognized.

[0045] In this application, the first step is to acquire an image for object recognition as the target image. This target image can be understood as an image for which a deep learning model needs to perform object recognition processing. The deep learning model then determines the object category to which the object to be recognized in the image belongs. Of course, the target image may not contain any object to be recognized; in this case, the deep learning model does not perform subsequent object recognition processing on the target image. For example, the object categories that the deep learning model can recognize can be multiple subcategories under any given category. For instance, if the given category is "ships," the object categories can be various ship categories, each representing a model or type of ship. Similarly, if the given category is "vehicles," the object categories can be various vehicle categories, each representing a model of vehicle. Or, if the given category is "transportation tools," the object categories can be ships, cars, airplanes, etc.; if the given category is "agricultural implements," the object categories can be shovels, rakes, etc. Of course, the object categories that the deep learning model can recognize may not be subcategories of any given category; for example, the object categories may be categories of different scenes, which is also reasonable.

[0046] Typically, deep learning models have input limitations on the input image, such as the image size being smaller than a predetermined size or the image resolution being greater than a predetermined resolution. Therefore, optionally, in one implementation, model input conditions can be pre-set for these input limitations. Then, when the target image to be identified does not meet these model input conditions, specified image processing is applied to the target image to make it meet the model input conditions, and the processed target image is then input into the deep learning model. The specified image processing can be image cropping or image resolution adjustment, etc., and this embodiment does not specifically limit its application.

[0047] S102: Input the target image into the pre-trained deep learning model to obtain the output of the deep learning model;

[0048] The output includes object region information of the object to be identified in the target image, as well as the category identification result of the object to be identified; the category identification result includes the target object category to which the object to be identified belongs.

[0049] In this application, the acquired target image is input into a pre-trained deep learning model so that the deep learning model performs object recognition processing on the target image, determines whether there is an object to be recognized in the target image, and if so, determines the object region information of the object to be recognized in the target image, and the category recognition result of the object to be recognized is used as the output result of the deep learning model.

[0050] It should be noted that the purpose of deep learning models in performing object recognition processing on target images is to determine the object category of the object to be recognized in the target image. Therefore, the above category recognition results include the target object category to which the object to be recognized belongs. For example, when the object to be recognized is an airplane, the target object category to which the object to be recognized belongs in the output result of the deep learning model is: Model P airplane. In other words, the deep learning model can identify the specific category to which the object to be recognized belongs, such as the specific model of the object.

[0051] Optionally, the object region of the object to be identified is marked in the target image using a location bounding box. Accordingly, the determined object region information of the object to be identified can be understood as the region information of the location bounding box corresponding to the object to be identified.

[0052] It should be noted that each object to be identified corresponds to a bounding box. That is, it is determined that the target image includes at least one bounding box containing an object to be identified. By determining the confidence level of each bounding box, the bounding box with the highest confidence level is determined as the bounding box corresponding to the object to be identified.

[0053] Optionally, when there are multiple objects to be identified in the target image, the deep learning model outputs the corresponding output result for each object to be identified, and then performs subsequent steps S103-S105 for each output result.

[0054] The training process of the above deep learning model will be explained below:

[0055] First, a sample library is pre-built, in which each sample image is assigned a sample label, which is used to characterize the object category to which the sample object in the sample image belongs.

[0056] In the process of training the aforementioned deep learning model, the electronic device used for model training can pre-build an initial model, and then input sample images obtained from the sample library into the initial model for training, thereby obtaining the aforementioned deep learning model for object recognition. The electronic device used for model training and the electronic device used for model inference can be the same device or different devices; this application does not specifically limit this.

[0057] During training, the initial model can learn the object features of sample objects in sample images. After learning from a large number of sample objects, the initial model gradually establishes a correspondence between object categories and object features, thus obtaining the deep learning model described above.

[0058] Training is stopped when the initial model meets the predetermined training stopping condition, and a deep learning model is obtained.

[0059] Optionally, the above training stopping conditions include the initial model's model loss reaching a predetermined loss.

[0060] Optionally, the above training stopping conditions include the initial model reaching a predetermined number of iterations.

[0061] In this way, after the deep learning model is trained, a correspondence between object categories and object features can be established in the deep learning model. Thus, after obtaining the target image to be identified, the target image is input into the deep learning model so that the deep learning model can perform object recognition processing on the target image.

[0062] The aforementioned deep learning models are models used for object recognition in images; therefore, they can be referred to as image recognition models. Examples include CNN (Convolutional Neural Network), Moonshot-v1-vision-preview (a multimodal vision model), and AlexNet. This application does not impose any limitations on the specific structure or training process of the deep learning models.

[0063] It's important to note that the accuracy of a deep learning model depends on the number of sample images. The more sample images a model has for a given category of objects, the higher its accuracy in recognizing those objects. However, in practical applications, sample images for some categories may be difficult to obtain, leading to a scarcity of such images during model training. Consequently, the accuracy of object recognition using a trained deep learning model may be lower.

[0064] Based on this, optionally, an initial sample image is acquired, and the initial sample image is subjected to predetermined image processing to increase the number of sample images containing the same sample object. The predetermined image processing may include rotation, flipping, affine transformation, elastic deformation, etc., for geometric transformations and deformations of the image; it may also include histogram equalization of the color space, brightness dithering, etc., for color space; and it may include Gaussian mixture noise processing, local binary mode processing, etc., for texture feature reconstruction. This application does not specifically limit the scope of the embodiments.

[0065] In this implementation, by processing the initial sample images, the number of sample images containing the same sample object can be increased when training samples are scarce, thereby expanding the number of sample images in the sample library and improving the model's accuracy in recognizing objects.

[0066] S103: Obtain the description information corresponding to the target object category.

[0067] The descriptive information corresponding to the target object category is used to describe the object characteristics of objects belonging to the target object category.

[0068] In this application, descriptive information about each object category can be obtained in advance. The descriptive information corresponding to each object category is information describing the object characteristics of objects within that object category. These object characteristics may include geometric features; for example, information describing geometric features could be: the proportions of a ship's main dimensions, hull form factors, or the shape characteristics of a vehicle. The object characteristics may also include behavioral features; for example, information describing behavioral features could be: a ship's sailing position or a vehicle's driving scenario. It should be noted that the data format of the descriptive information can be text or an image. This application does not specifically limit the specific type of object characteristics or the data format of the descriptive information. Any type of object characteristic can be used as an object characteristic of an object belonging to its object category, and any data format used to describe the object characteristics can be used as the descriptive information corresponding to the object's category.

[0069] After determining the target object category, descriptive information about the object characteristics of objects belonging to that target object category can be obtained. This descriptive information can then be combined with other descriptive information to assist in reasoning and analysis during the object recognition process, thereby improving the accuracy of object recognition.

[0070] For example, if the target object category is aircraft type P, then the obtained descriptive information is information used to describe the object characteristics of aircraft type P.

[0071] Optionally, the aforementioned descriptive information can be stored in a predetermined target database. This target database can record descriptive information corresponding to each object category, with each descriptive information describing the object characteristics of objects belonging to that category. Furthermore, for the same object category, one or more descriptive information entries can be set. These multiple descriptive information entries can be data in different storage formats representing the same object characteristic of objects within that object category, or they can represent different object characteristics; both are reasonable.

[0072] S104: Based on the image region indicated by the object region information and the description information corresponding to the target object category, analyze whether the object to be identified belongs to the object described by the description information, and obtain the analysis result.

[0073] In this application, the output of the deep learning model includes: object region information of the object to be identified in the target image, and the target object category to which the object to be identified belongs. The object region information indicates the image region where the object to be identified is located in the target image. Therefore, after obtaining the descriptive information corresponding to the target object category, the analysis can be performed based on the aforementioned image region and the obtained descriptive information to determine whether the object to be identified belongs to the object described in the descriptive information, and the analysis result can be obtained. The aforementioned descriptive information can also be referred to as domain knowledge or prior knowledge about the target object category.

[0074] In this way, by using the acquired descriptive information, the object category to which the object to be identified belongs is verified to be the target object category output by the deep learning model, thereby enabling auxiliary reasoning analysis of the object recognition process and improving the accuracy of object recognition.

[0075] For example, the target object category is a P-type aircraft, and the obtained descriptive information is all information used to describe the object characteristics of the P-type aircraft. When verifying whether the object to be identified is a P-type aircraft, the object characteristics of the object to be identified are determined by using the object characteristics of the P-type aircraft described in the descriptive information to determine whether the object to be identified is a P-type aircraft.

[0076] S105: If the analysis result indicates that the object to be identified belongs to the object described by the descriptive information, then the output result will be used as the object identification result of the object to be identified in the target image.

[0077] In this application, when the obtained analysis results indicate that the object to be identified belongs to the object described by the descriptive information, it indicates that the deep learning model's identification result for the object category to which the object to be identified belongs is accurate. At this time, in response to obtaining the analysis results used to indicate the belonging, the output result of the deep learning model is used as the object identification result of the object to be identified in the target image.

[0078] Conversely, when the analysis results indicate that the object to be identified does not belong to the object described in the descriptive information, it indicates that the deep learning model's identification of the object's category is inaccurate. In this case, the model outputs pre-defined information to prompt the user. This pre-defined information can be a reminder indicating recognition failure or information indicating a null value, i.e., an empty output. Upon receiving this pre-defined information, the user can confirm that the deep learning model has not identified the object's category, and that object recognition for the target image has failed.

[0079] Optionally, when there are multiple descriptive information, the analysis result corresponding to any descriptive information indicates that the object to be identified belongs to the object described by the descriptive information, and the above step S105 can be executed to use the output result as the object recognition result of the object to be identified in the target image.

[0080] As can be seen from the above, in the object recognition method provided in this application embodiment, object recognition of a target image based on a pre-trained deep learning model can obtain object region information of the object to be recognized in the target image, as well as the category recognition result for the object to be recognized. Then, based on the target object category to which the object to be recognized belongs, as contained in the category recognition result, descriptive information corresponding to that target object category is obtained. This descriptive information is used to describe the object features of objects belonging to that target object category. Thus, based on the image region indicated by the aforementioned object region information and the obtained descriptive information, it is possible to further analyze whether the object to be recognized belongs to the object described by the aforementioned descriptive information, obtain an analysis result, and when the obtained analysis result indicates that the object to be recognized belongs to the object described by the descriptive information, the output result of the deep learning model is used as the object recognition result of the object to be recognized in the target image.

[0081] In this way, during the object recognition process, by combining the acquired descriptive information and the image region indicated by the object region information, auxiliary reasoning is performed to determine the object category to which the object to be identified in the target image belongs. This further determines the accuracy of the target object category in the output of the deep learning model. Thus, when the target object category is a scarce object category, the supplementary reasoning based on the above descriptive information provides sufficient reasoning information for the object recognition process, thereby improving the accuracy of object recognition.

[0082] Optionally, in one embodiment, the target object category is one of multiple object categories, and all multiple object categories belong to subcategories within a specified category; the category identification result also includes the specified category; the object identification method provided in this application embodiment, such as Figure 2 As shown, it also includes the following steps:

[0083] S106: If the analysis result indicates that the object to be identified does not belong to the object described by the description information, then the result containing the object region information and the specified category will be used as the object identification result of the object to be identified in the target image.

[0084] In this embodiment, the object categories that the deep learning model can identify can be multiple subcategories under any specified category. For example, if the specified category is "ships," the object categories can be various ship categories, and each ship category can represent a type or model of ship. Another example is "vehicles," where the object categories can be various vehicle categories, and each vehicle category can represent a type of vehicle. Yet another example is "transportation," where the object categories can be ships, cars, airplanes, etc. In other words, in addition to identifying the object category, the deep learning model can also identify the specified category to which the object category belongs. The category identification result output by the deep learning model can also include the specified category to which the target object category belongs.

[0085] Thus, when the analysis results indicate that the object to be identified does not belong to the object described by the descriptive information, it indicates that the deep learning model's identification result for the object category to which the object belongs is inaccurate. In this case, the result containing object region information and the specified category can be used as the object identification result of the object to be identified in the target image.

[0086] For example, the object to be identified is actually a Q-type aircraft, but due to the inaccuracy of the deep learning model's recognition result, the output target object category is a P-type aircraft, resulting in the obtained description information describing the object as a P-type aircraft, which does not match the actual aircraft model. Therefore, the obtained analysis result indicates that the object to be identified does not belong to the object described in the description information. In this case, the specified category to which the object to be identified belongs: aircraft, and the object region information output by the deep learning model can be determined as the object recognition result of the object to be identified in the target image.

[0087] In this embodiment, when no analysis results are obtained to characterize the object's affiliation, the results containing object region information and a specified category can be used as the object recognition results of the object to be identified in the target image, thereby improving the model's recognition accuracy.

[0088] Optionally, in one embodiment, the category identification result further includes a target confidence level regarding whether the object to be identified belongs to the target object category; in an object identification method provided in this application embodiment, before obtaining the description information corresponding to the target object category in the above step S103, the following steps are further included:

[0089] Step A: Determine whether the target confidence level is higher than the predetermined threshold; if so, determine that the output result is the object recognition result of the object to be identified in the target image; otherwise, obtain the description information corresponding to the target object category.

[0090] In this embodiment, considering that there is time cost and computational resource consumption in the process of verifying the output results of the deep learning model by querying description information and combining the description information, a predetermined threshold for the confidence level of the object to be identified belonging to the target object category is set in advance, and the predetermined threshold is used as the condition for whether to trigger auxiliary reasoning analysis.

[0091] Thus, after obtaining the target confidence score output by the deep learning model, by determining whether the target confidence score is higher than the aforementioned predetermined threshold, it is determined whether to verify the output result in conjunction with descriptive information, thereby triggering the auxiliary inference analysis process. Specifically:

[0092] When the target confidence level is higher than the predetermined threshold, it indicates that the object to be identified by the deep learning model is more likely to belong to the target object category. At this time, there is no need to trigger the auxiliary reasoning analysis process; the output result can be directly determined as the object recognition result of the object to be identified in the target image.

[0093] When the target confidence level is not higher than a predetermined threshold, it indicates that the probability of the object to be identified output by the deep learning model belonging to the target object category is low. It is necessary to verify whether the object to be identified belongs to the target object category. At this time, the auxiliary reasoning analysis process is triggered to obtain the descriptive information corresponding to the target object category. Then, the output result of the deep learning model is verified in combination with the descriptive information, that is, to verify whether the object to be identified belongs to the target object category output by the deep learning model.

[0094] In this embodiment, a predetermined threshold is set as the condition for whether to trigger the auxiliary reasoning analysis process. The output results of the deep learning model are verified in combination with the descriptive information. That is, when the confidence of the target included in the category recognition result is not higher than the predetermined threshold, the auxiliary reasoning analysis process is triggered to obtain the descriptive information corresponding to the target object category, so as to reduce the time cost and computing resource consumption in the object recognition process, thereby improving the object recognition efficiency.

[0095] Optionally, in one implementation, step S104 above, based on the image region indicated by the object region information and the description information corresponding to the target object category, analyzes whether the object to be identified belongs to the object described by the description information to obtain the analysis result, and may include the following steps:

[0096] Step B1: Extract object features from the image region indicated by the object region information to obtain the first feature;

[0097] Step B2: Based on the descriptive information corresponding to the target object category, determine the object characteristics of the target object category to obtain the second characteristic;

[0098] Step B3: Identify whether the first feature and the second feature match. If they do, obtain the analysis result that characterizes the object to be identified as belonging to the object described by the descriptive information; otherwise, obtain the analysis result that characterizes the object to be identified as not belonging to the object described by the descriptive information.

[0099] In this implementation, when triggering the auxiliary reasoning analysis process and verifying the output of the deep learning model in conjunction with descriptive information, firstly, object features can be extracted from the image region indicated by the object region information to obtain the first feature. Then, based on the descriptive information corresponding to the target object category, the object features of the target object category are determined to obtain the second feature. Thus, by identifying whether the first and second features match, it is determined whether the object to be identified belongs to the object described by the descriptive information, i.e., whether the output of the deep learning model is accurate. Specifically:

[0100] When the first feature and the second feature do not match, it indicates that the object to be identified does not belong to the object described by the descriptive information. In this case, the analysis result is used to indicate that the object to be identified does not belong to the object described by the descriptive information. Based on this analysis result, it can be determined that the output result of the deep learning model is inaccurate. In the above case, in order to ensure the accuracy of recognition, the result containing object region information and the specified category can be used as the object recognition result of the object to be identified in the target image.

[0101] When the first feature and the second feature match, it indicates that the object to be identified belongs to the object described by the descriptive information. At this time, the analysis result is used to indicate that the object to be identified belongs to the object described by the descriptive information. Based on the analysis result, it can be determined that the output result of the deep learning model is accurate, and the output result is used as the object recognition result of the object to be identified in the target image.

[0102] In this implementation, feature extraction and feature matching are used to determine the accuracy of the output results of the deep learning model by combining descriptive information. This achieves the purpose of assisting reasoning analysis in the object recognition process by introducing descriptive information, thereby improving the accuracy of object recognition.

[0103] Optionally, in one implementation, object features include object geometric features and / or object behavioral features.

[0104] In this implementation, object features may include object geometric features. For example, object geometric features may include the object's principal scale parameter ratio, the object's shape features, etc.

[0105] Taking a container ship as the target object category, when validating the output of the deep learning model by combining descriptive information, the following steps are taken: First, the shape features of the image region indicated by the object region information are extracted to obtain the beam arch parameters of the container ship (i.e., the height difference between the centerline and the edgeline of the deck), which serves as the first feature. Then, based on the descriptive information corresponding to the target object category, the beam arch parameters of the container ship are determined, resulting in the second feature. Finally, by identifying whether the beam arch parameters represented by the first feature and the second feature are the same, the analysis result regarding whether the target object belongs to the container ship described in the descriptive information is determined.

[0106] Object characteristics may also include object behavioral characteristics. For example, object behavioral characteristics may include the object's location, the scene in which the object is located, etc.

[0107] Taking the target object category of the object to be identified as a container ship as an example, when verifying the output of the deep learning model in conjunction with the descriptive information, firstly, the latitude and longitude location information of the image region indicated by the object region information is extracted to obtain the first feature; then, based on the descriptive information corresponding to the target object category, the latitude and longitude location that the container ship can be located is determined to obtain the second feature; then, by determining whether the latitude and longitude location represented by the first feature is the same as that represented by the second feature, the analysis result regarding whether the object to be identified belongs to the container ship described by the descriptive information is determined.

[0108] Object features may also include object geometric features and object behavioral features. In this case, the extracted first feature includes a first sub-feature for the object geometric features and a second sub-feature for the object behavioral features. Correspondingly, the extracted second feature includes a third sub-feature for the object geometric features and a fourth sub-feature for the object behavioral features. Then, when determining feature matching, the first feature and the second feature are determined to match by identifying whether the first sub-feature and the third sub-feature match, and / or identifying whether the second sub-feature and the fourth sub-feature match, so as to obtain the analysis result on whether the object to be identified belongs to the object described by the descriptive information.

[0109] Optionally, in one embodiment, the descriptive information corresponding to the target object category is information represented in a multimodal data form, which includes text modality and image modality.

[0110] Step B2 above, based on the descriptive information corresponding to the target object category, determines the object characteristics of the object in the target object category to obtain the second feature, and may include the following steps:

[0111] Step C1: Based on the description information corresponding to the target object category, determine the object features of the target object category and each modality, and obtain the second feature corresponding to each modality;

[0112] Accordingly, step B3 above, identifying whether the first feature and the second feature match, may include the following steps:

[0113] Step C2: For each modality, convert the second feature and the first feature of that modality to the same modality, and identify whether the first feature and the second feature that have been converted to the same modality match.

[0114] In this embodiment, when storing the descriptive information corresponding to the target object category, it can be stored in a multimodal data format, including text modality and image modality. For example, the descriptive information corresponding to a cargo ship may include an image of the cargo ship, text information describing the external features of the cargo ship, etc.

[0115] Therefore, when obtaining the second feature, the object features of each modality of the target object category can be determined based on the description information corresponding to the target object category, so as to obtain the second feature corresponding to each modality. Then, for each modality, after converting the second feature of the modality and the first feature to the same modality, it is identified whether the first feature and the second feature converted to the same modality match.

[0116] Deep learning models are essentially image recognition models. In other words, the input data received by the model is presented as an image, and the object region information output by the model can also be presented in the form of an image. Therefore, it can be understood that the modality to which the extracted first feature belongs is the image modality.

[0117] If the data format of the description information corresponding to the target object category is text modality, that is, the presentation mode of the description information is text, then the text features in the description information are extracted as the second feature. Then, the modality to which the second feature belongs is converted to the same image modality as the first feature, so as to facilitate subsequent identification of whether the first feature and the second feature match.

[0118] If the data format of the descriptive information corresponding to the target object category is an image modality, that is, the presentation mode of the descriptive information is an image, then after extracting the image features from the descriptive information as the second feature, the first feature and the second feature are directly identified to determine whether they match.

[0119] Of course, the modality to which the first feature belongs can also be converted to the same modality as the second feature, or both the first and second features can be converted. These are all reasonable. In this embodiment, the purpose of converting the features is to convert the first and second features to the same modality so that the first and second features can be identified as matching in the future. Therefore, this application does not limit the modality conversion method, modality conversion process, or modality conversion method (converting the first feature or the second feature).

[0120] If the first feature matches the second feature, it indicates that the object to be identified belongs to the object described by the descriptive information. At this time, the analysis result is used to indicate that the object to be identified belongs to the object described by the descriptive information. Based on the analysis result, it can be determined that the output result of the deep learning model is accurate, and the output result is used as the object recognition result of the object to be identified in the target image.

[0121] If the first feature and the second feature do not match, it indicates that the object to be identified does not belong to the object described by the descriptive information. In this case, the analysis result is used to indicate that the object to be identified does not belong to the object described by the descriptive information. Based on this analysis result, it can be determined that the output result of the deep learning model is inaccurate. In the above case, in order to ensure the accuracy of recognition, the result containing object region information and the specified category is used as the object recognition result of the object to be identified in the target image.

[0122] In this embodiment, by performing modality transformation on the features extracted from multimodal descriptive information, multiple data sources (text and images) are combined to assist in reasoning in the object recognition process, thereby improving the accuracy and stability of object recognition.

[0123] Optionally, in one implementation, step S103 above, obtaining the description information corresponding to the target object category, may include the following steps:

[0124] Step D: Retrieve the description information corresponding to the target object category from the target database;

[0125] The target database is used to record descriptive information corresponding to each object category;

[0126] An object recognition method provided in this application embodiment further includes the following steps:

[0127] Step E1: If no description information corresponding to the target object category is obtained from the target database, the output result is used as the object recognition result of the object to be identified in the target image and output.

[0128] Step E2: In response to the user's evaluation of the object recognition result, if the evaluation result indicates that the recognition is correct, the object feature is extracted from the image region indicated by the object region information to obtain the first feature;

[0129] Step E3: Add the first feature as descriptive information of the target object category in the object recognition result to the target database.

[0130] In this implementation, a target database is pre-set to record the description information corresponding to each object category. In this way, after the target object category is determined, the description information corresponding to the target object category can be directly obtained from the target database.

[0131] Considering that the descriptive information recorded in the target database cannot completely cover all object categories, there may be cases where the target object category's descriptive information is not found in the target database. In such cases, the output result can be used as the object recognition result of the object to be identified in the target image and output. That is to say, even if the target confidence is lower than a predetermined threshold, the output result of the deep learning model will still be output.

[0132] Then, through user confirmation, the accuracy of the output object recognition results is evaluated to determine whether the object to be identified belongs to the target object category, and the user's evaluation result on the object recognition results is obtained.

[0133] If the obtained evaluation result indicates that the recognition is correct, then the object feature is extracted from the image region indicated by the object region information in the output result to obtain the first feature. Then, the extracted first feature is added to the target database as the descriptive information of the target object category in the object recognition result.

[0134] If the evaluation result indicates an error in the identification, the object identification result is discarded; or, based on the user's prior experience, descriptive information describing the object characteristics of the target object category to which the object to be identified belongs is added to the target database.

[0135] In this implementation, if the required descriptive information is not obtained from the target database, the descriptive information recorded in the target database is added based on the user's evaluation of the object recognition results, so as to achieve dynamic updating of the target database, improve the accuracy of object recognition, and thus further improve the applicability and generalization of the object recognition method provided in this application embodiment.

[0136] For ease of understanding, the following detailed description of an object recognition method provided in this application will be provided with reference to a specific embodiment.

[0137] In the fields of modern image processing and target recognition, artificial intelligence technologies (such as deep learning) have become mainstream solutions. Especially in areas such as public place monitoring, key area protection, and traffic control, deep learning models and other AI technologies are widely used for target detection and recognition. However, existing target recognition technologies still face some bottlenecks and challenges. The following provides an example of the shortcomings of existing technologies:

[0138] (1) Deep learning models rely on a large amount of high-quality labeled data, but samples are scarce in certain specific fields (such as rare objects like ships and motorboats), resulting in insufficient training sample data, which in turn affects the recognition accuracy of the trained deep learning model.

[0139] (2) Most current object recognition systems rely on pure data-driven deep learning methods and lack domain-specific prior knowledge support, which makes it difficult for deep learning models to effectively infer the object category and state of an object in environments with few samples or in complex scenarios.

[0140] (3) Existing object recognition systems have weak generalization capabilities. When faced with unknown targets or environmental changes, their recognition performance drops significantly, and they are prone to misidentification or missed identification.

[0141] Therefore, how to improve the accuracy and adaptability of target recognition (object recognition) in environments with scarce samples has become a key issue that urgently needs to be addressed.

[0142] To address the aforementioned issues, this specific embodiment provides an object recognition framework to support the execution of an object recognition method provided in this application embodiment. Figure 3 This is a schematic diagram of an object recognition framework provided in an embodiment of this application.

[0143] like Figure 3 As shown, the object recognition framework provided in this application embodiment includes a system layer, an architecture layer, a framework layer, and a module layer.

[0144] The system layer includes a target recognition system (i.e., an object recognition system), which includes a feature extraction module, a recognition module, a domain knowledge reasoning module, a knowledge base (i.e., the target database in this application), a sample database, and a few-shot learning module; wherein, the aforementioned feature extraction module, recognition module, domain knowledge reasoning module, knowledge base, sample database, and few-shot learning module together constitute the module layer in the object recognition framework.

[0145] The architecture layer includes a deep learning model, a knowledge reasoning module (i.e., a module for performing the relevant steps in this application to verify the output of the deep learning model in combination with descriptive information), and a multimodal data fusion module (i.e., a module for performing the step in this application to identify whether the first feature and the second feature match).

[0146] The framework layer includes a feature extraction module (i.e., a module for performing the object feature extraction step in this application), a recognition module (i.e., a module for performing the object recognition step in this application), and a domain knowledge reasoning module (i.e., a module for performing the steps in this application of obtaining descriptive information corresponding to the target object category from the target database, and subsequently analyzing whether the object to be identified belongs to the object described by the descriptive information based on the image region indicated by the object region information and the descriptive information corresponding to the target object category, and obtaining the analysis result). The domain knowledge reasoning module is connected to a knowledge base and a sample library. The training samples in the sample library are obtained based on a few-shot learning module. This few-shot learning module is used to perform the steps in this application of obtaining initial sample images and performing predetermined image processing on the initial sample images to increase the number of sample images containing the same sample object.

[0147] It should be noted that the processing flow of a deep learning model involves a feature extraction module and a recognition module, the processing flow of the knowledge reasoning module involves a recognition module and a domain knowledge reasoning module, and the processing flow of the multimodal data fusion module involves a feature extraction module.

[0148] The purpose of this specific embodiment is to overcome the limitations of current object recognition systems by combining artificial intelligence technology and knowledge reasoning technology, and to improve the recognition ability and generalization performance of target samples (i.e., the objects to be identified in this application).

[0149] By introducing a knowledge reasoning mechanism, that is, by adding the processing flow related to the aforementioned knowledge reasoning module, the relevant knowledge about object characteristics of object categories in the fields of geometric features, physical attributes, and behavioral patterns of the target (object) is integrated (i.e., the descriptive information in this application, which can also be called domain knowledge or prior knowledge), thereby enhancing the reasoning ability of the deep learning model, especially improving the accuracy of object recognition when samples are scarce.

[0150] Based on the above Figure 3 The object recognition framework shown in this application provides an object recognition method that can supplement the shortcomings of deep learning models through knowledge reasoning in environments where the amount of training sample data is insufficient or the object categories are complex, thereby enabling the object recognition system to have stronger adaptability.

[0151] Furthermore, this specific embodiment also supports few-shot learning and incremental training, enabling the knowledge base in the object recognition system to dynamically absorb new knowledge and still perform effective reasoning and recognition when faced with unseen objects, thereby improving the generalization ability and robustness of the object recognition system.

[0152] This specific embodiment also provides an object recognition system to implement the above-described object recognition method. Figure 4 This is a schematic diagram of the structure of an object recognition system provided in an embodiment of this application, as shown below. Figure 4 As shown, the object recognition system may include an input layer, a core processing layer, and an output layer. These three layers can be understood as the division of the execution entity provided in this application from the perspective of functional module division.

[0153] The core processing layer includes a feature extraction module for deep learning models, a recognition module for deep learning models, a domain knowledge reasoning module, a knowledge base, a sample database, and a few-shot learning module.

[0154] The following is combined Figure 4 The object recognition system shown here is a detailed description of an object recognition method provided in the embodiments of this application.

[0155] The object recognition method provided in this application improves the recognition ability of the object recognition system in complex or scarce sample environments by adding a domain knowledge reasoning module to the deep learning model and combining artificial intelligence and knowledge reasoning.

[0156] After receiving the target to be identified (the object to be identified in this application), the input layer of the object recognition system inputs the target to be identified into the feature extraction module of the deep learning model in the core processing layer. The feature extraction module extracts image features (i.e., object features in this application) of the target to be identified and inputs the extracted image features into the recognition module of the deep learning model, so that the recognition module of the deep learning model can determine the object region information of the target to be identified and the target object category to which the target to be identified belongs, as the output result of the entire deep learning model.

[0157] Subsequently, when the confidence level of the target to be identified as belonging to the target object category is not higher than a predetermined threshold, it indicates that samples identical to the target to be identified are scarce samples during the training of the deep learning model. Therefore, after the trained deep learning model obtains the identification result of the target to be identified for that object category, the identification result needs to be verified to improve the identification accuracy. At this point, by adding an inference mechanism to the traditional deep learning target (object) detection, domain knowledge related to the target object category (such as the geometric features and physical properties of the object) is introduced to assist in inference analysis, enhancing the entire object recognition system's understanding of the target to be identified. For example, when identifying ships, information such as the target's geometric dimensions and navigation patterns is used to further confirm whether the target belongs to the target object category. Specifically:

[0158] The output results are input into the domain knowledge reasoning module, which then obtains the domain knowledge of the target object category (i.e., the descriptive information in this application) from the knowledge base. The obtained domain knowledge is used to supplement the object identification process, thereby assisting in reasoning analysis and identification. This helps to identify targets lacking samples, especially rare objects and objects in special environments, to analyze whether the target to be identified belongs to the target object category in the above output results and obtain the analysis results.

[0159] If the analysis results indicate that the target to be identified belongs to the target object category in the above output results, then the output results are determined as the identification results and output to the output layer; if the analysis results indicate that the target to be identified does not belong to the target object category in the above output results, then the specified category corresponding to the target object category is determined as the identification result and output to the output layer.

[0160] It should be noted that in the above recognition process, in addition to using deep learning models for data-driven feature extraction, the object recognition system also uses domain knowledge (prior knowledge) in the knowledge base to supplement reasoning, realize auxiliary reasoning analysis, and help identify targets lacking samples, especially for rare targets and targets in special environments, in order to further confirm whether the target to be identified belongs to the target object category. That is, in the object recognition process performed by the object recognition system, not only is visual feature learning carried out, but also reasoning judgment based on the knowledge base is performed.

[0161] The knowledge base contains domain knowledge in multimodal data formats, such as historical data describing the geometric features and behavioral patterns (object behavioral features in this application) of objects in various formats including image data, text descriptions, and sensor data. Therefore, the domain knowledge reasoning module also involves multimodal data fusion technology to identify the target by integrating multiple pieces of information during the assisted reasoning process based on domain knowledge.

[0162] Optionally, the knowledge base can be dynamically updated through information interaction with the deep learning model and manual operation by the user. In this way, the adaptability and accuracy of the object recognition system can be improved by continuously optimizing and improving the domain knowledge contained in the knowledge base.

[0163] Furthermore, to address the potential scarcity of sample images, the object recognition system provided in this specific embodiment also includes a few-shot learning module. This module increases the number of scarce sample images by performing image processing on them, such as image flipping, and combines this with existing sample images in the sample library and domain knowledge inferred from the knowledge base to incrementally train the deep learning model. In this way, the object recognition system can effectively improve target recognition capabilities even when sample data is scarce, avoiding the accuracy degradation caused by insufficient data in traditional deep learning methods.

[0164] This specific embodiment combines artificial intelligence and knowledge reasoning technologies. By introducing a domain knowledge reasoning module, it improves object recognition accuracy, especially when samples are scarce. The object recognition system can utilize prior knowledge in the knowledge base to enhance its reasoning ability, significantly improving recognition accuracy, particularly for scarce objects and objects in complex scenarios. Furthermore, the domain knowledge reasoning mechanism enables the object recognition system to effectively reason and adjust when facing new targets or changing environments, enhancing its adaptability and generalization capabilities, especially significantly improving its ability to recognize unknown targets.

[0165] Furthermore, in the process of using deep learning models to perform feature learning and object recognition from image data, supplementing the data with data from other sources (such as text descriptions or sensor data) can improve the overall accuracy and precision of the recognition. Moreover, through multimodal data fusion, the object recognition system can comprehensively consider data from different sources, effectively reducing errors from a single data source and improving the robustness of the object recognition system in various environments.

[0166] Furthermore, by introducing a few-shot learning module, the object recognition system can effectively train and optimize the recognition process using domain knowledge from the knowledge base, even when samples are scarce, thus avoiding a decline in object recognition capability due to insufficient samples. Moreover, with the dynamic updating of the knowledge base, new samples and domain knowledge are continuously added, constantly optimizing the deep learning model and inference mechanism, further improving the long-term performance and adaptability of the object recognition system.

[0167] Corresponding to the above method embodiments, this application also provides an object recognition device, such as... Figure 5 As shown, the device includes:

[0168] The first acquisition module 510 is used to acquire the target image to be recognized by object;

[0169] The model processing module 520 is used to input the target image into a pre-trained deep learning model to obtain the output result of the deep learning model; wherein, the output result includes object region information of the object to be identified in the target image, and category identification result for the object to be identified; the category identification result includes the target object category to which the object to be identified belongs;

[0170] The second acquisition module 530 is used to acquire description information corresponding to the target object category; wherein, the description information corresponding to the target object category is used to describe the object characteristics of objects belonging to the target object category;

[0171] The object analysis module 540 is used to analyze whether the object to be identified belongs to the object described by the description information based on the image region indicated by the object region information and the description information corresponding to the target object category, and to obtain the analysis result.

[0172] The first determining module 550 is used to take the output result as the object recognition result of the object to be identified in the target image if the analysis result indicates that the object to be identified belongs to the object described by the description information.

[0173] Optionally, in one implementation, the target object category is one of multiple object categories, and all of the multiple object categories belong to subcategories within a specified category; the category identification result also includes the specified category;

[0174] The device further includes:

[0175] The second determining module is used to, if the analysis result indicates that the object to be identified does not belong to the object described by the description information, take the result containing the object region information and the specified category as the object identification result of the object to be identified in the target image.

[0176] Optionally, in one implementation, the category identification result further includes a target confidence level regarding whether the object to be identified belongs to the target object category;

[0177] The device further includes:

[0178] The third determining module is used to determine whether the target confidence level is higher than a predetermined threshold before obtaining the description information corresponding to the target object category;

[0179] If so, the output result is determined to be the object recognition result of the object to be identified in the target image; otherwise, the second acquisition module 530 is triggered.

[0180] Optionally, in one implementation, the object analysis module 540 includes:

[0181] The extraction submodule is used to extract object features from the image region indicated by the object region information to obtain the first feature;

[0182] The determination submodule is used to determine the object features of the object in the target object category based on the description information corresponding to the target object category, and obtain the second feature;

[0183] The matching submodule is used to identify whether the first feature and the second feature match. If they do, an analysis result is obtained to characterize that the object to be identified belongs to the object described by the description information; otherwise, an analysis result is obtained to characterize that the object to be identified does not belong to the object described by the description information.

[0184] Optionally, in one implementation, the descriptive information corresponding to the target object category is information represented in a multimodal data form, which includes text modality and image modality.

[0185] The determining submodule is specifically used for:

[0186] Based on the description information corresponding to the target object category, determine the object features of the target object category and each modality, and obtain the second feature corresponding to each modality;

[0187] The matching submodule is specifically used for:

[0188] For each modality, the second feature of that modality is converted to the same modality as the first feature, and it is then identified whether the first feature and the second feature converted to the same modality match.

[0189] Optionally, in one implementation, the object features include object geometric features and / or object behavioral features.

[0190] Optionally, in one implementation, the second acquisition module 530 is specifically used for:

[0191] Obtain the description information corresponding to the target object category from the target database; wherein, the target database is used to record the description information corresponding to each object category;

[0192] The device further includes:

[0193] The data processing module is configured to: if no description information corresponding to the target object category is obtained from the target database, output the result as the object recognition result of the object to be identified in the target image and output it; respond to the user's evaluation result of the object recognition result, if the evaluation result indicates that the recognition is correct, extract object features from the image region indicated by the object region information to obtain a first feature; and add the first feature as the description information of the target object category in the object recognition result to the target database.

[0194] This application also provides an electronic device, such as... Figure 6 As shown, it includes a processor 601, a communication interface 602, a memory 603, and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604.

[0195] Memory 603 is used to store computer programs;

[0196] When the processor 601 executes the program stored in the memory 603, it implements the object recognition method described in any of the above embodiments.

[0197] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0198] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0199] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0200] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0201] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described object recognition methods.

[0202] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the object recognition methods described above.

[0203] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0204] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0205] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments, electronic device embodiments, computer-readable storage medium embodiments, and computer program product embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0206] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. An object recognition method, characterized in that, The method includes: Acquire the target image to be recognized; The target image is input into a pre-trained deep learning model to obtain the output of the deep learning model; wherein, the output includes object region information of the object to be identified in the target image, and category identification result for the object to be identified; the category identification result includes the target object category to which the object to be identified belongs; Obtain the description information corresponding to the target object category; wherein, the description information corresponding to the target object category is used to describe the object characteristics of objects belonging to the target object category; Based on the image region indicated by the object region information and the description information corresponding to the target object category, analyze whether the object to be identified belongs to the object described by the description information, and obtain the analysis result; If the analysis result indicates that the object to be identified belongs to the object described by the descriptive information, then the output result is used as the object identification result of the object to be identified in the target image.

2. The method according to claim 1, characterized in that, The target object category is one of multiple object categories, and all of the multiple object categories belong to subcategories within the specified category; The category identification result also includes a specified category; The method further includes: If the analysis result indicates that the object to be identified does not belong to the object described by the description information, then the result containing the object region information and the specified category will be used as the object identification result of the object to be identified in the target image.

3. The method according to claim 1 or 2, characterized in that, The category identification result also includes a target confidence level regarding whether the object to be identified belongs to the target object category; Before obtaining the description information corresponding to the target object category, the method further includes: Determine whether the target confidence level is higher than a predetermined threshold; If so, the output result is determined to be the object recognition result of the object to be identified in the target image; otherwise, the description information corresponding to the target object category is obtained.

4. The method according to claim 1 or 2, characterized in that, The method involves analyzing whether the object to be identified belongs to the object described by the description information based on the image region indicated by the object region information and the description information corresponding to the target object category, and obtaining the analysis result, including: Object features are extracted from the image region indicated by the object region information to obtain the first feature; Based on the description information corresponding to the target object category, the object characteristics of the object in the target object category are determined to obtain the second characteristic; If the first feature and the second feature match, an analysis result is obtained to characterize that the object to be identified belongs to the object described by the description information; otherwise, an analysis result is obtained to characterize that the object to be identified does not belong to the object described by the description information.

5. The method according to claim 4, characterized in that, The descriptive information corresponding to the target object category is information represented in a multimodal data form, which includes text modality and image modality. The step of determining the object characteristics of the object in the target object category based on the description information corresponding to the target object category, and obtaining the second characteristic, includes: Based on the description information corresponding to the target object category, determine the object features of the target object category and each modality, and obtain the second feature corresponding to each modality; The step of identifying whether the first feature and the second feature match includes: For each modality, the second feature of that modality is converted to the same modality as the first feature, and it is then identified whether the first feature and the second feature converted to the same modality match.

6. The method according to claim 1 or 2, characterized in that, The object features include object geometric features and / or object behavioral features.

7. The method according to claim 1 or 2, characterized in that, The step of obtaining the description information corresponding to the target object category includes: Obtain the description information corresponding to the target object category from the target database; wherein, the target database is used to record the description information corresponding to each object category; The method further includes: If no description information corresponding to the target object category is obtained from the target database, the output result is used as the object recognition result of the object to be identified in the target image and output. In response to the user's evaluation of the object recognition result, if the evaluation result indicates that the recognition is correct, object features are extracted from the image region indicated by the object region information to obtain the first feature; The first feature is added to the target database as descriptive information of the target object category in the object recognition result.

8. An object recognition device, characterized in that, The device includes: The first acquisition module is used to acquire the target image to be recognized; The model processing module is used to input the target image into a pre-trained deep learning model to obtain the output result of the deep learning model; wherein, the output result includes object region information of the object to be identified in the target image, and category identification result for the object to be identified; the category identification result includes the target object category to which the object to be identified belongs; The second acquisition module is used to acquire description information corresponding to the target object category; wherein, the description information corresponding to the target object category is used to describe the object characteristics of objects belonging to the target object category; The object analysis module is used to analyze whether the object to be identified belongs to the object described by the description information based on the image region indicated by the object region information and the description information corresponding to the target object category, and to obtain the analysis result. The first determining module is used to take the output result as the object recognition result of the object to be identified in the target image if the analysis result indicates that the object to be identified belongs to the object described by the description information.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-7.

Images