Target identification method and system, processing equipment and storage medium

Abstract

The embodiment of the invention provides a target recognition method and system, processing equipment and a storage medium, and relates to the technical field of computer vision. The method comprises: acquiring a to-be-recognized image; inputting the to-be-recognized image into a pre-trained target feature extraction model, and obtaining a target feature of a target object output by the target feature extraction model; based on a current object feature library and the target features, determining an object matching result of the target object, the current object feature library including registration features of registered objects; and based on the target features and the object matching result, updating the current object feature library, and returning to the step of obtaining the image to be recognized so as to perform target recognition by using a new current object feature library. In this way, the current object feature library is updated by using the object features obtained in the target recognition process and the object matching result, and the accuracy of the target recognition result can be improved.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a target recognition method, system, processing device, and storage medium. Background Technology

[0002] ReID (Person re-identification) technology is a technique that uses computer vision to determine whether a specific target exists in an image or video sequence. It has wide applications in fields such as intelligent video surveillance and intelligent security.

[0003] For example, in the field of intelligent video surveillance, ReID technology is used to extract ReID features from images of people captured by image acquisition devices, and the extracted ReID features are matched with the ReID features of registered personnel to obtain the identification results of the personnel.

[0004] However, factors such as a person's posture, clothing, and the environment of the collection area can all affect the feature extraction results of a person's image. This can lead to a significant difference between the extracted ReID features of the person and the ReID features at the time of registration, resulting in low accuracy of the recognition results. Summary of the Invention

[0005] The purpose of this application is to provide a target recognition method, system, processing device, and storage medium to improve the accuracy of recognition results. The specific technical solution is as follows:

[0006] In a first aspect, embodiments of this application provide a target recognition method, the method comprising:

[0007] Acquire the image to be recognized;

[0008] The image to be identified is input into a pre-trained target feature extraction model to obtain the target features of the target object output by the target feature extraction model.

[0009] Based on the current object feature library and the target features, the object matching result of the target object is determined, wherein the current object feature library includes the registration features of registered objects;

[0010] Based on the target features and the object matching results, the current object feature library is updated, and the step of obtaining the image to be identified is returned, so as to perform target recognition using the new current object feature library.

[0011] Optionally, before the step of determining the object matching result of the target object based on the current object feature library and the target features, the method further includes:

[0012] The image to be identified is input into a pre-trained human attribute classifier, and the human attribute classification result of the target object output by the human attribute classifier is obtained.

[0013] The step of updating the current object feature library based on the target features and the object matching results includes:

[0014] Based on the quality index of the recognition result corresponding to the image to be recognized, the reliability of the object matching result is determined;

[0015] The quality indicators include at least one of the following: the degree of object matching represented by the object matching result, the accuracy of the object matching result, and the reliability of attribute recognition represented by the human attribute classification result.

[0016] Based on the level of confidence, the current object feature library is updated using the object matching results and the target features.

[0017] Optionally, the step of updating the current object feature library based on the object matching result and the target features according to the degree of confidence includes:

[0018] If the level of confidence is greater than a first preset threshold, the registration features of the registered objects indicated by the object matching results in the current object feature library are updated using the target features;

[0019] If the confidence level is less than the first preset threshold and greater than the second preset threshold, an annotation notification about the image to be identified is output, the annotation result of the user annotating the image to be identified based on the annotation notification is obtained, and the registration feature of the registered object indicated by the annotation result in the current object feature library is updated using the annotation result and the target feature.

[0020] If the credibility level is less than the second preset threshold, output the first registration prompt information, obtain the registration name and registration characteristics of the target object entered by the user based on the first registration prompt information, and store the registration name and registration characteristics in the current object feature library.

[0021] Optionally, the method further includes:

[0022] If the difference between the classification results of multiple human attributes corresponding to the registered object within a first preset time period and the human attributes represented by the registration features of the registered object is greater than a preset difference threshold, an update notification of the registration features of the registered object will be output.

[0023] Optionally, before the step of determining the object matching result of the target object based on the current object feature library and the target features, the method further includes:

[0024] Based on the relationship between the average matching degree of the valid identification samples within the second preset time period before the current time and the preset maximum matching threshold and the preset minimum matching threshold, the current matching threshold is determined. The valid identification samples are images that meet the preset image quality among the images to be identified acquired within the second preset time period. The current matching threshold is used to determine the matching degree between the target feature and the registered features of registered objects in the current object feature library.

[0025] Optionally, the step of determining the current matching threshold based on the relationship between the average matching degree of valid identification samples within a second preset time period prior to the current time and the preset maximum matching threshold and the preset minimum matching threshold includes:

[0026] If the average matching degree of the validly identified samples in the second preset time period before the current time is greater than the preset maximum matching threshold, the preset maximum matching threshold is determined as the current matching threshold.

[0027] If the average matching degree is less than a preset minimum matching threshold, the preset minimum matching threshold is determined as the current matching threshold;

[0028] If the average matching degree is not greater than the preset maximum matching threshold and not less than the preset minimum matching threshold, the average matching degree is determined as the current matching threshold.

[0029] Optionally, the method further includes:

[0030] If the average matching degree is less than the preset minimum matching threshold for a duration that reaches a preset duration, at least one of the following three methods shall be executed: output an abnormal warning of the credibility of the object matching result; output a second registration prompt message; or update the target feature extraction model using valid unidentified samples within the second preset time period.

[0031] Optionally, the method further includes:

[0032] The image to be identified, the target features of the target object, and the object matching result of the target object are used as sample data;

[0033] According to a preset period, target sample data that meets the preset sample quality is extracted from the sample data, and the target sample data is used to train the target feature extraction model.

[0034] Optionally, the method further includes:

[0035] The object matching result, the device identifier of the image acquisition device that acquired the image to be identified, and the target features are input into a preset large language model so that the large language model can determine the target name represented by the object matching result, determine the location name corresponding to the device identifier, determine the action category corresponding to the target feature based on the correspondence between the target features, preset action features, and action categories, and obtain a natural language description of the recognition result of the target object based on the target name, the location name, and the action category.

[0036] Secondly, embodiments of this application provide a target recognition system, the system including a processing device and a distributed image acquisition device;

[0037] Each image acquisition device is used to detect moving targets and, when a moving target is detected, to acquire an image of the moving target;

[0038] A processing device is configured to acquire the image to be identified acquired by the image acquisition device and execute the method described in any one of the first aspects.

[0039] Optionally, the processing device is further configured to construct a spatiotemporal association map of the registered object based on the registration name of the registered object, the acquisition time corresponding to each image to be identified, and the obtained identification result of the registered object when the identification results of the images to be identified acquired by multiple image acquisition devices correspond to the same registered object.

[0040] Thirdly, embodiments of this application provide a processing apparatus, including:

[0041] Memory, used to store computer programs;

[0042] When a processor executes a program stored in memory, it implements any of the steps described above.

[0043] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the steps described above.

[0044] Fifthly, embodiments of this application also provide a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the steps described above.

[0045] Beneficial effects of the embodiments in this application:

[0046] In the technical solution provided in this application embodiment, the electronic device can acquire an image to be recognized, input the image to be recognized into a pre-trained target feature extraction model, and acquire the target features of the target object output by the target feature extraction model; determine the object matching result of the target object based on the current object feature library and the target features, wherein the current object feature library includes the registration features of registered objects; update the current object feature library based on the target features and the object matching result, and return to the step of acquiring the image to be recognized, so as to use the new current object feature library for target recognition.

[0047] As can be seen, electronic devices can update the current object feature library by utilizing the object features extracted by the target feature extraction model and the obtained object matching results during the target recognition process. Through adaptation and self-evolution, the current object feature library can be dynamically updated and continuously optimized without human intervention, enabling it to constantly adapt to feature changes and thus maintain high accuracy in target recognition. Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above simultaneously. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0049] Figure 1 A flowchart illustrating a target recognition method provided in an embodiment of this application;

[0050] Figure 2 This is a schematic diagram illustrating the generation of natural language descriptions provided in an embodiment of this application;

[0051] Figure 3 This is a schematic diagram of a specific implementation of step S104;

[0052] Figure 4 This is a schematic diagram of the intelligent agent structure provided in the embodiments of this application;

[0053] Figure 5 A flowchart illustrating an example of target recognition provided in an embodiment of this application;

[0054] Figure 6 A schematic diagram illustrating the intelligent agent closed-loop feedback optimization provided in the embodiments of this application;

[0055] Figure 7 This is a schematic diagram of the structure of a target recognition system provided in an embodiment of this application;

[0056] Figure 8 This is a schematic diagram of the structure of a target recognition device provided in an embodiment of this application;

[0057] Figure 9 This is a schematic diagram of the structure of a processing device provided in an embodiment of this application. Detailed Implementation

[0058] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0059] In the technical solution of this application, the acquisition, storage, use, processing, transmission, provision and disclosure of user personal information are all carried out with the user's authorization.

[0060] It should be noted that the human body model in this embodiment is not a human body model for a specific user and cannot reflect the personal information of a specific user.

[0061] It should be noted that the images of people in this embodiment are from a publicly available dataset.

[0062] To improve the accuracy of target recognition results, embodiments of this application provide a target recognition method, system, processing device, computer-readable storage medium, and computer program product. The target recognition method provided by the embodiments of this application will be introduced first.

[0063] The target recognition method provided in this application can be applied to any electronic device capable of target recognition, such as a backend server of an image acquisition device, a processing device, or an intelligent agent. The electronic device can be a single device or a cluster of multiple electronic devices, etc., without specific limitations. For clarity, it will be referred to as an electronic device below. Hereinafter, an intelligent agent is a core concept in artificial intelligence and computer science, typically referring to an entity capable of perceiving its environment, making autonomous decisions, and executing actions.

[0064] like Figure 1 As shown in the embodiment of this application, a target recognition method is provided, the method comprising:

[0065] S101: Acquire the image to be recognized;

[0066] S102: Input the image to be identified into a pre-trained target feature extraction model to obtain the target features of the target object output by the target feature extraction model;

[0067] S103: Based on the current object feature library and the target feature, determine the object matching result of the target object.

[0068] The current object feature library includes the registration features of registered objects.

[0069] S104: Based on the target features and the object matching results, update the current object feature library and return to the step of obtaining the image to be identified, so as to use the new current object feature library for target recognition.

[0070] In the technical solution provided in this application embodiment, the electronic device can acquire an image to be recognized, input the image to be recognized into a pre-trained target feature extraction model, and acquire the target features of the target object output by the target feature extraction model; determine the object matching result of the target object based on the current object feature library and the target features, wherein the current object feature library includes the registration features of registered objects; update the current object feature library based on the target features and the object matching result, and return to the step of acquiring the image to be recognized, so as to use the new current object feature library for target recognition.

[0071] It is evident that electronic devices can update the current object feature library by utilizing the object features extracted by the target feature extraction model and the obtained object matching results during the target recognition process. Through adaptation and self-evolution, the current object feature library can be dynamically updated and continuously optimized without human intervention, enabling the current object feature library to continuously adapt to feature changes, thereby maintaining a high accuracy of target recognition results.

[0072] When conducting intelligent video surveillance and intelligent security operations in scenarios such as residential homes, office parks, and commercial complexes, objects in the monitored scenarios can be pre-registered to build an object feature database for easier object identification. For example, in smart home scenarios, to meet application needs such as home monitoring and elderly care, family members can be pre-registered to build a family object feature database, which can then be used to identify family members.

[0073] Furthermore, image acquisition devices can be set up in scenarios requiring intelligent video surveillance and intelligent security. These devices can capture images or videos of objects in the scenario and send them to electronic devices.

[0074] Then the electronic device can acquire the image to be identified, that is, perform the above step S101.

[0075] In one implementation, the image acquisition device can acquire an image of a moving target when a moving target is detected and send it to an electronic device, which can receive the image and use it as an image to be identified.

[0076] In one implementation, the image acquisition device can acquire scene monitoring video and send it to an electronic device. The electronic device can obtain the video uploaded by the image acquisition device and extract frames from the video to obtain the image to be recognized.

[0077] In one implementation, the image acquisition device can acquire scene images at preset intervals and send them to an electronic device. The electronic device can perform object detection on the image sequence uploaded by the image acquisition device and take the image frames containing the object to be identified as the images to be identified.

[0078] In order to identify the target object in the image to be identified, the electronic device can perform the above step S102, that is, input the image to be identified into the pre-trained target feature extraction model, and obtain the target features of the target object output by the target feature extraction model.

[0079] The aforementioned target feature extraction model can be any type of feature extraction model, such as the ReID feature extraction model or other models that can perform object feature recognition. The category of the feature extraction model is not limited here.

[0080] For example, the pre-trained target feature extraction model mentioned above is a ReID feature extraction model, which can extract a 128-dimensional ReID feature vector of the target object.

[0081] The target object in the image to be recognized can be set according to the business needs of the actual recognition scenario. For example, in a parking management scenario, the target object can be a vehicle; in a personnel monitoring scenario, the target object can be a person; and in an animal management scenario, the target object can be an animal. There is no limitation on the type of target object in the image to be recognized here. For ease of explanation, the following explanation uses a person as the target object.

[0082] Furthermore, the extracted target features include the target object's shape and posture features, enabling identity and posture recognition. For example, if the target feature is a ReID feature, the target object's identity and posture can be determined by recognizing its ReID features.

[0083] In one implementation, the electronic device can input the image to be recognized into a pre-trained human detection model for human detection, obtain the human detection result output by the human detection model, and then input the human detection result into a pre-trained target feature extraction model to obtain the target features of the target object output by the target feature extraction model. Here, the human detection model is a lightweight model.

[0084] In one implementation, the pre-trained target feature extraction model includes a human detection sub-model and a target feature extraction sub-model. The electronic device can input the image to be recognized into the human detection sub-model of the target feature extraction model. The human detection sub-model can perform human detection on the image to be recognized and input the human detection result into the target feature extraction sub-module. The target feature extraction sub-module performs target feature extraction on the human detection result and outputs the target feature of the target object in the image to be recognized.

[0085] It should be noted that the features stored in the current object feature library are also the features of the target feature extraction model.

[0086] The current object feature database stores the registration features of multiple registered objects, and this database is a long-term memory. During object registration, users can complete the registration process through an interactive interface. The registration steps include: basic registration information entry (entering the registration name and other object identity information); feature sample collection (extracting 128-dimensional features from the user's image to construct the object's initial feature prototype); and camera location marking (selecting the device ID that captured the user's image and extracting location information). After completing these steps, the object's registration name and features are stored in the current object feature database, completing the registration process.

[0087] For example, in a home care scenario, basic registration information for family members can be pre-entered. This basic registration information may include name, gender (in binary representation), and age. Feature samples are collected from family members; that is, at least one image of a family member is acquired by an image acquisition device set up in the home setting, and the 128-dimensional ReID feature vector of the family member in the image is automatically extracted as the registration feature. The basic information and registration features are then stored in the current object feature library to complete the registration of the family member. Furthermore, the current object feature library can also store the device ID of the image acquisition device that collected the feature samples from the family member, and record the association between the device ID and location information in the home care scenario.

[0088] After obtaining the target features of the target object, the electronic device can perform the above step S103, that is, determine the object matching result of the target object based on the current object feature library and the target features.

[0089] Electronic devices can calculate the degree of matching between the target feature and the registration feature of each registered object in the current object feature library, and determine the object matching result of the target object based on the relationship between each degree of matching and the current matching threshold.

[0090] The electronic device can calculate the cosine distance, Euclidean distance, etc. between the target feature and the registered features of each registered object in the current object feature library to determine the degree of matching between the target feature and the registered features. This degree of matching can be represented by a similarity score or by other parameters. Here, the calculation method and representation method of the degree of matching are not limited.

[0091] Electronic devices can identify registered objects corresponding to registered features whose matching degree is greater than the current matching threshold as the matching objects of the target object, and obtain the object matching result of the target object; or they can identify registered objects corresponding to registered features with the highest matching degree of the target feature as the matching objects of the target object, and obtain the object matching result of the target object.

[0092] The object matching result includes at least the object name of the registered object that matches the target object, and may also include the degree of matching between the target feature of the target object and the registration feature of the registered object.

[0093] For example, in a smart home scenario, the current object feature library includes the registered names and registered ReID features of each family member. After extracting the target ReID features of the target object through the target feature extraction model, the cosine distance between the target ReID feature and each registered ReID feature is calculated. Based on the registered object "Mom" corresponding to the registered ReID feature with the highest score, the object matching result of the target object is determined, that is, the target object is identified as "Mom".

[0094] In order to improve the dynamic updating and long-term retention of the current object feature library, and to enhance the adaptability of the current object feature library to changes in object features, thereby maintaining stable recognition performance, the electronic device can perform the above step S104, that is, update the current object feature library based on the target features and object matching results, and return to the above step S101 to obtain the image to be recognized, so as to use the new current object feature library for target recognition.

[0095] The registered features of registered objects stored in the current object feature library are limited. When the posture, clothing, and environment of a registered object are different, the target features of the registered object obtained by extracting target features from the image to be recognized may differ significantly from the registered features, leading to inaccurate recognition results. In order to maintain high target recognition accuracy, the current object feature library can be updated using the target features and object matching results obtained during the target recognition process. By continuously updating and optimizing the current object feature library, the registered features of the registered members in the current object feature library can be continuously enriched, thereby ensuring the accuracy of recognition of registered members.

[0096] After updating the current object feature library using the current target features and object matching results, the electronic device can continue to acquire the image to be identified, use the new current object feature library to perform target identification on the image to be identified, and use the target features and object matching results of the image to be identified to update the current object feature library again. Thus, the current object feature library is continuously updated during the target identification process, realizing the self-adaptation and self-evolution of the current object feature library.

[0097] As can be seen, in the technical solution provided by the embodiments of this application, the electronic device can use the object features extracted by the target feature extraction model and the obtained object matching results to update the current object feature library during the target recognition process. Through adaptation and self-evolution, the dynamic update and continuous optimization of the current object feature library without human intervention is realized, so that the current object feature library can continuously adapt to feature changes, thereby maintaining a high accuracy of target recognition results and improving the robustness of the recognition method.

[0098] As one embodiment of this application, before step S103 above, i.e., the step of determining the object matching result of the target object based on the current object feature library and the target features, the target recognition method provided in this application embodiment may further include:

[0099] The image to be identified is input into a pre-trained human attribute classifier, and the human attribute classification result of the target object output by the human attribute classifier is obtained.

[0100] When the target object in the image to be identified is a person, in order to avoid misidentification of the target due to the lack of obvious single visual features, after acquiring the image to be identified, the image to be identified can be input into a pre-trained human attribute classifier. In this way, the human attribute classifier can identify the target object in the image to be identified and classify the human attributes of the target object, thereby obtaining the human attribute classification result of the target object.

[0101] Among them, the human attribute classifier can classify information such as clothing attributes, posture estimation, personal belongings, and facial related attributes of the target object.

[0102] Facial attributes include whether a mask or glasses are worn; clothing characteristics include outerwear type (with or without an outerwear), color and style of top (long-sleeved / short-sleeved), type of bottoms (long pants / shorts), and footwear characteristics; posture includes standing, sitting, standing sideways, and sitting sideways; personal items include whether a backpack or handbag is carried.

[0103] The human attribute classification results of the aforementioned target objects include fine-grained human attribute classification results for a preset number of categories, as well as the reliability of attribute recognition.

[0104] The preset number can be set according to actual needs, such as 14 dimensions, 23 dimensions, etc., without specific limitations.

[0105] For example, the image to be recognized can be input into a human attribute recognition model (human attribute classifier) ​​that supports 23 fine-grained attributes, wherein the MAP (Mean Average Precision) of the human attribute recognition model is not less than 92%. The human attribute recognition model can determine whether the target object is wearing a mask based on the IOU (Intersection over Union) between the mask and the target object's face, and determine six types of outerwear features (outerwear type, top color and style, bottom type) and four types of footwear features based on the user's clothing.

[0106] Furthermore, based on an angle error of <5°, the pose estimation of the target object is achieved by determining the side angle of the target object according to the standing / sitting posture and by using discretization grading.

[0107] In one implementation, such as Figure 2 As shown, after identifying the human attributes classification results of the target object, the human attributes classification results and the object matching results can be fused together, and the fusion result can be encoded and input into the preset Large Language Model (LLM). Combined with the device identifier of the image acquisition device, the location label corresponding to the device identifier can be determined, such as living room, kitchen, office, etc. Combined with the environmental context and spatiotemporal information, a natural language description of the target object can be realized.

[0108] The electronic device can perform human detection on the original image, obtain the human detection result, and input the human detection result into a ReID feature extraction model for ReID feature extraction and feature library comparison. Then, it inputs the human detection result into a human attribute recognition model for attribute analysis. The human attribute analysis result, feature library comparison result, and environmental context are encoded into JSON-LD format and input into an LLM (Local Level Manager). This LLM then integrates the spatiotemporal context to generate a natural language description and outputs a report. The output includes, but is not limited to, human attribute information, environmental context, and object matching results from dimensions such as facial attributes, clothing features, personal belongings, etc., achieving multi-dimensional information feedback.

[0109] Furthermore, after identifying each image to be identified and obtaining the object matching result of the target object in the object to be identified, and combining the human attribute analysis result, feature library comparison result and environmental context to obtain the natural language description of the target object, the device identifier of the image acquisition device that acquired the image to be identified and the acquisition time of the image to be identified can be stored together and a report can be output to facilitate the generation of spatiotemporal trajectory management reports for each registered object.

[0110] Correspondingly, such as Figure 3 As shown, step S104 above, namely the step of updating the current object feature library based on the target features and the object matching results, may include:

[0111] S301: Based on the quality index of the recognition result corresponding to the image to be recognized, determine the credibility of the object matching result.

[0112] The quality indicators include at least one of the following: the degree of object matching represented by the object matching result, the accuracy of the object matching result, and the reliability of attribute recognition represented by the human attribute classification result.

[0113] S302: Based on the level of confidence, update the current object feature library using the object matching results and the target features.

[0114] After obtaining the object matching results and the human attribute classification results, the electronic device can determine the credibility of the object matching results based on the quality index of the recognition results corresponding to the image to be recognized.

[0115] The quality indicators include at least one of the following: the degree of object matching represented by the object matching results, the accuracy of the object matching results, and the reliability of attribute recognition represented by the human attribute classification results.

[0116] The object matching results described above represent the degree of object matching, which is the degree of matching between the target features of the target object and the registration features of the matched registered object. This degree of matching can be represented by the similarity score between the target features and the registration features, or by the number of matches between the target features and the registration features. Of course, other parameters that can represent the degree of matching can also be used, which are not specifically limited here.

[0117] The accuracy of the above object matching results is the accuracy of the target detection box. The accuracy of the object matching results can be represented by the stability of the target detection box, or by the recognition accuracy of the target detection box. Of course, it can also be represented by other parameters that can represent the accuracy of the object matching results, which are not specifically limited here.

[0118] The confidence level of attribute recognition represented by the above human attribute classification results can be expressed by the attribute recognition confidence level output by the human attribute classifier, or by other parameters that can represent the confidence level of attribute recognition, without specific limitations here.

[0119] The methods for determining the reliability of object matching results vary depending on the parameters included in the quality indicators.

[0120] In one implementation, if the above quality index only includes one of the following: the degree of object matching represented by the object matching result, the accuracy of the object matching result, and the reliability of attribute recognition represented by the human attribute classification result, the quality index can be directly used as the reliability of the object matching result.

[0121] In one implementation, when the aforementioned quality indicators include multiple items such as the degree of object matching represented by the object matching result, the accuracy of the object matching result, and the reliability of attribute recognition represented by the human attribute classification result, the reliability of the object matching result can be obtained by weighted summation of each indicator.

[0122] For example, when the above quality indicators include the degree of object matching represented by the object matching result and the accuracy of the object matching result, the degree of object matching represented by the object matching result and the accuracy of the object matching result can be weighted and summed to obtain the credibility of the object matching result.

[0123] After determining the credibility of the object matching results, the electronic device can update the current object feature library based on the object matching results and target features.

[0124] As can be seen, in this embodiment, the electronic device can input the image to be recognized into a pre-trained human attribute classifier and obtain the human attribute classification result of the target object output by the human attribute classifier; based on the quality index of the recognition result corresponding to the image to be recognized, the credibility of the object matching result is determined; according to the credibility, the current object feature library is updated using the object matching result and target features. In this way, by evaluating the credibility of the object matching result through at least one of the object matching degree represented by the object matching result, the accuracy of the object matching result, and the attribute recognition credibility represented by the human attribute classification result, the credibility of the object matching result can be determined quickly and accurately by combining indicators of different dimensions. Then, the credibility of the object matching result can be used to determine an update strategy adapted to the credibility, and the current object feature library can be updated using the determined update strategy, thereby improving the adaptive capability.

[0125] As one embodiment of this application, step S302 above, namely the step of updating the current object feature library based on the confidence level using the object matching result and the target features, may include:

[0126] If the level of confidence is greater than a first preset threshold, the registration features of the registered objects indicated by the object matching results in the current object feature library are updated using the target features;

[0127] If the confidence level is less than the first preset threshold and greater than the second preset threshold, an annotation notification about the image to be identified is output, the annotation result of the user annotating the image to be identified based on the annotation notification is obtained, and the registration feature of the registered object indicated by the annotation result in the current object feature library is updated using the annotation result and the target feature.

[0128] If the credibility level is less than the second preset threshold, output the first registration prompt information, obtain the registration name and registration characteristics of the target object entered by the user based on the first registration prompt information, and store the registration name and registration characteristics in the current object feature library.

[0129] If the credibility of the object matching result is greater than the first preset threshold, the target feature of the target object matches the registration feature of the registered object indicated by the object matching result to a high degree, and the target object can be determined to be a registered object in the current object feature library.

[0130] Electronic devices can use target features to update the registration features of registered objects indicated by object matching results in the current object feature library. The aforementioned first preset threshold can be set according to actual needs, such as 0.8, 0.75, etc., and is not specifically limited here.

[0131] Specifically, when the credibility of the object matching result is greater than the first preset threshold, the registration feature of the registered object indicated by the object matching result in the current object feature library is updated using the target feature. This can be done by replacing the registration feature with the target feature, by superimposing the target feature with the registration feature to strengthen the target feature, or by storing the target feature and the registration feature in the current object feature library in correspondence, thereby directly using the target feature for incremental updates of the feature library to strengthen the feature representation of the registered object.

[0132] If the reliability of the object matching result is less than the first preset threshold but greater than the second preset threshold, the target feature of the target object matches the registration feature of the registered object indicated by the object matching result with a low degree of match. Therefore, it cannot be accurately determined that the target object is a registered object in the current object feature library, and the identity of the target object needs to be confirmed. The second preset threshold can be set according to actual needs, such as 0.6, 0.55, etc., without specific limitations here. Furthermore, the second preset threshold is less than the first preset threshold.

[0133] The electronic device can output a labeling notification about the image to be recognized, so that the user can view the labeling notification, label the target objects in the image to be recognized based on the labeling notification, and feed the labeling results back to the electronic device.

[0134] Electronic devices can obtain the annotation results of target objects from user feedback. Using the annotation results and the extracted target features of the target objects, since the matching degree between the target features and the registered features is low, the registered features of the registered objects indicated by the annotation results in the current object feature library can be updated using the target features.

[0135] In cases where the credibility of the object matching result is less than the first preset threshold but greater than the second preset threshold, the registration feature of the registered object indicated by the annotation result in the current object feature library can be updated using the target feature. This can be achieved by storing the target feature and the registration feature in the current object feature library. In this way, by pushing the target object to be determined with a low matching degree to the user, the user is guided to participate in secondary annotation, and the object feature of the registered object indicated by the user's annotation result is updated using the target feature.

[0136] If the credibility of the object matching result is less than the second preset threshold, the target characteristics of the target object have a very low degree of matching with the registration characteristics of the registered object indicated by the object matching result, and the target object is suspected to be a new individual outside of the registered object.

[0137] The electronic device can output a first registration prompt message to invite the user to enter the basic registration information of the target object and collect the registration characteristics of the target object. The user can view the first registration prompt message and enter the registration name and registration characteristics of the target object based on the first registration prompt message.

[0138] Electronic devices can obtain the user's registered name and registration characteristics, and store the registered name and registration characteristics in the current object feature library. When the credibility of the object matching result is less than a second preset threshold, a registration prompt is triggered to invite the user to enter basic information and object characteristics, which can expand the current object feature library.

[0139] As can be seen, in this embodiment, when the confidence level is greater than a first preset threshold, the registration features of the registered objects indicated by the object matching results in the current object feature library are updated using the target features; when the confidence level is less than the first preset threshold but greater than a second preset threshold, a labeling notification for the image to be identified is output, the labeling results of the user's object labeling of the image to be identified based on the labeling notification are obtained, and the registration features of the registered objects indicated by the labeling results in the current object feature library are updated using the labeling results and target features; when the confidence level is less than the second preset threshold, a first registration prompt message is output, and the registration name and registration features of the target object entered by the user based on the first registration prompt message are obtained, and the registration name and registration features are stored in the current object feature library accordingly. By adopting a hierarchical feedback strategy based on the confidence level of the object matching results, the purpose of optimizing and updating the current object feature library using the object matching results is achieved, adapting to the confidence level of the object matching results.

[0140] As one embodiment of this application, the target recognition method provided in this application embodiment may further include:

[0141] If the difference between the classification results of multiple human attributes corresponding to the registered object within a first preset time period and the human attributes represented by the registration features of the registered object is greater than a preset difference threshold, an update notification of the registration features of the registered object will be output.

[0142] Because the pose, clothing, and environmental factors of the object during image acquisition will affect the human characteristics of the object in the image, it may lead to a large difference between the human attribute classification result of the object and the human attributes represented by the registered characteristics of the registered object.

[0143] For example, if the clothing in the registered characteristics of a registered object is a short-sleeved shirt, shorts, and no glasses on the face, but the registered object is wearing a long-sleeved jacket, long pants, and glasses, the difference between the classification result of the registered object's human attributes obtained from image acquisition and the human attributes represented by the registered characteristics will affect the identification of the registered object.

[0144] The electronic device can record the human attribute classification results of each registered object. If the difference between the multiple human attribute classification results corresponding to the registered object and the human attributes represented by the registered features of the registered object within a first preset time period is greater than a preset difference threshold, in order to avoid the registered object being significantly affected by changes in its appearance, the electronic device can output a registration feature update notification for the registered object, and proactively remind the user to update the feature profile of the registered object in the current object feature database.

[0145] As can be seen, in this embodiment, if the difference between the classification results of multiple human attributes corresponding to a registered object within a first preset time period and the human attributes represented by the registered features of the registered object exceeds a preset difference threshold, a registration feature update notification for the registered object is output. In this way, by proactively requesting the user to update the feature profile in the current object feature library, the problem of misidentification caused by changes in appearance can be reduced, and the accuracy of target recognition can be improved.

[0146] As one embodiment of this application, before step S103 above, i.e., the step of determining the object matching result of the target object based on the current object feature library and the target features, the target recognition method provided in this application embodiment may further include:

[0147] Based on the relationship between the average matching degree of the valid identification samples within the second preset time period before the current time and the preset maximum matching threshold and the preset minimum matching threshold, the current matching threshold is determined. The valid identification samples are images that meet the preset image quality among the images to be identified acquired within the second preset time period. The current matching threshold is used to determine the matching degree between the target feature and the registered features of registered objects in the current object feature library.

[0148] Considering that factors such as image quality, occlusion ratio, and pose angle may affect the quality of target features obtained by object feature recognition of target objects in the image to be recognized, and thus affect the object matching results of matching target objects in the image to be recognized with registered users.

[0149] Furthermore, the matching result between the target features of the target object in the image to be identified and the registered user is determined based on the current matching threshold. This current matching threshold is used to determine the degree of matching between the target features and the registration features of registered objects in the current object feature library. If the matching value between the target features and the registration features of registered objects in the current object feature library is greater than the current matching threshold, the degree of matching between the target features and the registered features of the registered objects is high, meaning the target features match the registered features of the registered objects. If the matching value between the target features and the registration features of registered objects in the current object feature library is not greater than the current matching threshold, the degree of matching between the target features and the registered features of the registered objects is low, meaning the target features do not match the registered features of the registered objects.

[0150] In order to effectively adapt to factors such as changes in lighting, occlusion, and differences in viewing angle, the current matching threshold is adjusted in real time according to factors such as lighting intensity and occlusion ratio. This allows for the determination of the average matching degree of valid identification samples within a second preset time period prior to the current moment. The average matching degree can reflect the specific conditions of factors such as image quality (e.g., lighting intensity, blur level), occlusion ratio, and pose angle within the most recent time period.

[0151] Among them, the effective recognition sample is the image that meets the preset image quality among the images to be recognized acquired within the second preset time period. The preset image quality can be set according to actual needs, such as: front face detection score ≥ 0.8, image quality index ≥ good level, for example, QI (Quality Index) ≥ 75, illumination level ∈ [200, 1000] LUX, occlusion area ratio ≤ 15%, and pose angle deviation ≤ 30°, etc.

[0152] Then, based on the relationship between the average matching degree of the valid identification samples in the second preset time period before the current time and the preset maximum matching threshold and the preset minimum matching threshold, the current matching threshold is determined.

[0153] The preset maximum matching threshold and preset minimum matching threshold can be set according to actual needs. For example, the preset maximum matching threshold can be 0.75, and the preset minimum matching threshold can be 0.55, etc.

[0154] As can be seen, in this embodiment, the current matching threshold is determined based on the relationship between the average matching degree of the valid identification samples within the second preset time period prior to the current time and the preset maximum and minimum matching thresholds. Valid identification samples are images that meet the preset image quality among the images to be identified acquired within the second preset time period. The current matching threshold is used to determine the matching degree between the target features and the registered features of registered objects in the current object feature library. Thus, by dynamically adjusting the current matching threshold using the relationship between the average matching degree of the valid identification samples within the second preset time period prior to the current time and the preset maximum and minimum matching thresholds, stable recognition performance can be maintained in different scenarios.

[0155] As one embodiment of this application, the step of determining the current matching threshold based on the relationship between the average matching degree of valid identification samples within a second preset time period before the current time and the preset maximum matching threshold and the preset minimum matching threshold may include:

[0156] If the average matching degree of the validly identified samples in the second preset time period before the current time is greater than the preset maximum matching threshold, the preset maximum matching threshold is determined as the current matching threshold.

[0157] If the average matching degree is less than a preset minimum matching threshold, the preset minimum matching threshold is determined as the current matching threshold;

[0158] If the average matching degree is not greater than the preset maximum matching threshold and not less than the preset minimum matching threshold, the average matching degree is determined as the current matching threshold.

[0159] The electronic device can determine the average matching degree, the preset minimum matching threshold, and the preset maximum matching threshold of the valid identification samples within a second preset time period before the current time.

[0160] Furthermore, if the average matching degree of the validly identified samples in the second preset time period before the current time is greater than the preset maximum matching threshold, the preset maximum matching threshold is determined as the current matching threshold; if the average matching degree is less than the preset minimum matching threshold, the preset minimum matching threshold is determined as the current matching threshold; and if the average matching degree is neither greater than the preset maximum matching threshold nor less than the preset minimum matching threshold, the average matching degree is determined as the current matching threshold.

[0161] In one implementation, a dynamic threshold calculation model can be used to optimize the threshold calculation formula as follows:

[0162] P = clamp(μ_iday, P _min, P _max);

[0163] Where: P is the current matching threshold; μ_iday is the average matching degree of valid identification samples in the past i days; P_min is the preset minimum matching threshold; and P_max is the preset maximum matching threshold.

[0164] For example, the average matching degree μ_7day corresponding to the valid identification samples in the past 7 days can be obtained, and the relationship between μ_7day and P_min and P_max can be compared. When μ_7day is greater than P_max, P_max is used as the current matching threshold; when μ_7day is less than P_min, P_min is used as the current matching threshold; when μ_7day is less than P_max and greater than P_min, μ_7day is used as the current matching threshold.

[0165] As can be seen, in this embodiment, if the average matching degree of the validly identified samples within the second preset time period before the current time is greater than the preset maximum matching threshold, the preset maximum matching threshold is determined as the current matching threshold; if the average matching degree is less than the preset minimum matching threshold, the preset minimum matching threshold is determined as the current matching threshold; and if the average matching degree is neither greater than the preset maximum matching threshold nor less than the preset minimum matching threshold, the average matching degree is determined as the current matching threshold. This allows for quick and accurate adjustment of the matching threshold based on image quality and environmental factors, determining the current matching threshold and improving the stability of cross-time period recognition.

[0166] As one embodiment of this application, the target recognition method provided in this application embodiment may further include:

[0167] If the average matching degree is less than the preset minimum matching threshold for a duration that reaches a preset duration, at least one of the following three methods shall be executed: output an abnormal warning of the credibility of the object matching result; output a second registration prompt message; or update the target feature extraction model using valid unidentified samples within the second preset time period.

[0168] The electronic device can monitor the average matching degree. If it detects that the average matching degree of valid identification samples within a second preset time period prior to the current moment is less than a preset minimum matching threshold for a duration that reaches a preset duration, it can be determined that the average matching degree is abnormal. The preset duration can be set according to actual needs, such as 5 days, 10 days, 1 month, etc., which are all reasonable and are not specifically limited here.

[0169] To eliminate anomalies, electronic devices can use an adaptive feedback mechanism to perform at least one of the following three methods:

[0170] Output an alert indicating an anomaly in the credibility of the object matching results to remind the user that the credibility of the object matching results is abnormal, so that the user can investigate the anomaly based on the alert.

[0171] Output a second registration prompt message to proactively guide the user to perform annotation, and update the current object feature library by utilizing the annotation information provided by the user and the target features corresponding to the samples effectively identified within the second preset time period before the current time.

[0172] The target feature extraction model is updated using valid samples to be identified within the second preset time period. The model is then fine-tuned using these valid samples to improve the accuracy of the object features extracted by the target feature extraction model.

[0173] As can be seen, in this embodiment, when the average matching degree is less than the preset minimum matching threshold for a certain duration, at least one of the following three methods is executed: outputting an abnormality warning of the credibility of the object matching result; outputting a second registration prompt message; and updating the target feature extraction model using valid samples to be identified within a second preset time period. In this way, the electronic device can adaptively identify abnormalities in the average matching degree and, through an adaptive feedback mechanism, achieve at least one of abnormality warning, user annotation guidance, and model updating, thereby improving target recognition capabilities and achieving self-evolution and adaptation.

[0174] As one embodiment of this application, the target recognition method provided in this application embodiment may further include:

[0175] The image to be identified, the target features of the target object, and the object matching result of the target object are used as sample data;

[0176] According to a preset period, target sample data that meets the preset sample quality is extracted from the sample data, and the target sample data is used to train the target feature extraction model.

[0177] Electronic devices can use the image to be identified acquired in each target recognition process, the target features of the target object in the image to be identified extracted by the target feature extraction model, and the object matching results of the target object as sample data, and store the sample data in a short-term memory bank for caching.

[0178] In order to continuously enhance the feature extraction performance of the target feature extraction model and optimize the target feature extraction model, the electronic device can extract target sample data that meets the preset sample quality from the sample data according to a preset period, and use the target sample data to train the target feature extraction model. Specifically, the electronic device can input the target sample data into the target feature extraction model so that the target feature extraction model can learn the correspondence between the image features in the target sample data and the object features of the target object in the image, and update the model parameters.

[0179] The preset cycle can be set according to actual needs. For example, it can be a week, a 10-day cycle, a month, etc. There are no specific limitations here.

[0180] Furthermore, the preset sample quality can be set according to actual needs. For example, the frontal face detection score is ≥0.8, the image quality index is ≥good (QI≥75), the illumination level is ∈[200,1000]LUX, the occlusion area ratio is ≤15%, and the pose angle deviation is ≤30°. No specific limitations are made here.

[0181] In one implementation, the electronic device can also generate pseudo-label data for the image to be identified using a big data processing model, and use the image to be identified and its pseudo-label data to assist in training the human body model and human attribute classification model, thereby achieving cross-modal knowledge transfer and improving generalization ability.

[0182] As can be seen, in this embodiment, the image to be identified, the target features of the target object, and the object matching results of the target object are used as sample data; target sample data that meets the preset sample quality is extracted from the sample data according to a preset period, and the target sample data is used to train the target feature extraction model. High-quality feedback samples are periodically extracted from the short-term memory, thereby realizing the optimization of the target feature extraction model without human intervention.

[0183] As one embodiment of this application, the target recognition method provided in this application embodiment may further include:

[0184] The object matching result, the device identifier of the image acquisition device that acquired the image to be identified, and the target features are input into a preset large language model so that the large language model can determine the target name represented by the object matching result, determine the location name corresponding to the device identifier, determine the action category corresponding to the target feature based on the correspondence between the target features, preset action features, and action categories, and obtain a natural language description of the recognition result of the target object based on the target name, the location name, and the action category.

[0185] The electronic device can acquire object matching results, the device identifier of the image acquisition device, and target features of the image to be recognized, and input the object matching results, device identifier, and target features into a preset large language model. The device identifier can be a location label for the image acquisition device.

[0186] In this way, the large language model can determine the target name represented by the object matching result; determine the location name corresponding to the device identifier; and determine the action category corresponding to the target feature based on the correspondence between target features, preset action features, and action categories. Then, based on the target name, location name, and action category, a natural language description of the target object recognition result is obtained.

[0187] For example, the object matching result includes the target name as "Mom", the device identifier as "1", and the corresponding kitchen; based on the correspondence between the target features, preset action features, and action categories, the action category corresponding to the target features is determined to be "cooking". The large language model can output a natural language description of the target object recognition result: Mom is cooking in the kitchen.

[0188] As can be seen, in this embodiment, the object matching result, the device identifier of the image acquisition device that acquires the image to be identified, and the target features are input into a preset large language model. This allows the large language model to determine the target name represented by the object matching result, the location name corresponding to the device identifier, and, based on the correspondence between the target features, preset action features, and action categories, the action category corresponding to the target features is determined. Based on the target name, location name, and action category, a natural language description of the target object's recognition result is obtained. By inputting structured data such as location tags, identity information, and action states as context into the large language model, the accuracy of semantic reasoning is improved. Furthermore, the multimodal large language model is used to semantically model the objects, locations, and behaviors in the image to be identified, generating natural language descriptions and outputting behavioral descriptions in natural language form, which facilitates user understanding and interaction, thus improving the user experience.

[0189] To facilitate understanding of the target recognition method provided in the embodiments of this application, the following is combined with... Figures 4-5 The following explanation is provided. The intelligent agent in the figure is an electronic device in the embodiments of this application.

[0190] The intelligent agent comprises a perception layer, a cognition layer, a decision layer, a user feedback interface, a confidence assessment module, and a labeling platform. The perception layer includes a camera array for moving target detection and acquiring raw image frames (images to be identified). The cognition layer includes a feature memory, which consists of a short-term memory cache (MySQL cluster 1), a long-term memory cache (MySQL cluster 2), a learning engine, and a person feature database. It employs a two-layer storage architecture combining short-term and long-term memory to achieve dynamic updates and long-term retention of person features. The decision layer includes a cognitive reasoning center, which performs LLM semantic association and descriptive output.

[0191] After acquiring the original image frame, the agent performs agent analysis on the original image frame. Specifically, the agent uses a ReID feature extraction model to extract the ReID features of the target person in the original image frame, and calculates the ReID matching confidence score by comparing the extracted ReID features with the registration features of registered persons stored in the feature library (current object feature library), thus obtaining the matching confidence score result (object matching result).

[0192] The intelligent agent utilizes a human attribute classification model to perform facial attribute detection, clothing feature recognition, posture analysis, and personal item detection on target individuals in the original image frames, obtaining human attribute classification results. It also obtains environmental context labels by acquiring the location tags of the image acquisition device. The matching confidence results, human attribute classification results, and environmental context labels are then encapsulated into structured data and fed back as a data packet to a short-term memory cache.

[0193] The agent evaluates the confidence level of the recognition results based on the feedback data packet. If the confidence level is greater than 0.8, it directly uses the feedback data packet to incrementally update the feature library, strengthening the feature representation of existing individuals. If the confidence level is greater than 0.5 but not greater than 0.8, the feedback data packet is marked as an item to be confirmed, added to the annotation task queue, and pushed to the user for secondary annotation. The user can enter the annotation workbench to annotate the target person in the original image frame. The agent can obtain the manual annotation results and use them to fine-tune the model and incrementally update the feature library. If the confidence level is not greater than 0.5, a new registration process is triggered, inviting the user to enter the registration interface to enter the basic information of the target person and new feature samples to expand the ReID person database.

[0194] Furthermore, the intelligent agent can also detect changes in the appearance of registered individuals. If it detects three consecutive changes in the appearance of the same registered individual (such as changing clothes or accessories), it can remind the user to confirm the update of their feature profile. If the user confirms the update of the registered individual's feature profile, it will perform an incremental update of the feature database.

[0195] In addition, the agent can periodically extract high-quality feedback samples from short-term memory and use these high-quality feedback samples to update the inference model.

[0196] This intelligent agent can achieve self-evolutionary optimization through closed-loop feedback. The closed-loop optimization stage is as follows: Figure 6 As shown, the agent can generate multi-dimensional feedback and perform automated evaluation based on this feedback. When the recognition result is of high confidence, the multi-dimensional feedback information can be used for incremental updates of the feature library. When the recognition result is of low confidence, user annotation guidance can be provided, and the user annotation guidance results can be used for subsequent online model optimization using the image to be recognized and the user annotation results. When the recognition result is a new individual, registration prompts can be provided, and the user registration results can be used for subsequent incremental updates of the feature library and online model optimization.

[0197] The agent can dynamically adjust the threshold based on the average matching degree of historical valid samples, and adjust the matching threshold according to image quality and environmental factors to improve the stability of recognition across time periods. The ReID matching threshold is adjusted in real time based on factors such as image quality (e.g., illumination intensity, blur level), occlusion ratio, and pose angle (default threshold 0.65, dynamic range 0.55~0.75) to maintain stable recognition performance in different scenarios. Furthermore, the agent can extract target sample data that meets the preset sample quality from the sample data stored in the short-term memory cache at preset intervals, use the target sample data to train the ReID feature extraction model, update the model parameters of the ReID feature extraction model, and use the updated ReID feature extraction model to continue performing the target recognition task in the cognitive reasoning stage.

[0198] In intelligent monitoring scenarios with complex and dynamic environments such as residential homes, office parks, and commercial complexes, the target recognition method is applied. Through closed-loop optimization and database model self-evolution, the recognition accuracy is improved, the stability of cross-time period recognition (i.e., the accuracy fluctuation of continuous 24-hour operation) decreases, and the recognition accuracy of unregistered personnel is improved. This achieves continuous optimization of recognition capabilities and significantly enhances environmental adaptability.

[0199] Corresponding to the target recognition method, this application also provides a target recognition system.

[0200] like Figure 7 As shown, a target recognition system includes a processing device 701 and a distributed image acquisition device 702.

[0201] Each image acquisition device 702 is used to detect moving targets and, when a moving target is detected, to acquire an image of the moving target;

[0202] The processing device 701 is used to acquire the image to be identified acquired by the image acquisition device and execute a target recognition method provided in the embodiments of this application.

[0203] The target recognition system provided in this application may include a processing device 701 and a distributed image acquisition device 702, wherein each image acquisition device is distributed and deployed at different locations in the application scenario. Each image acquisition device detects moving targets in the image acquisition area, and when a moving target is detected, acquires an image of the moving target and sends the image to the processing device.

[0204] The processing device can acquire the image to be identified acquired by the image acquisition device, execute a target recognition method provided in the embodiments of this application, and obtain the target recognition result.

[0205] The steps of the processing device executing the target recognition method have been described in detail above and will not be repeated here.

[0206] In the technical solution provided in this application embodiment, the image acquisition device in the target recognition system can detect moving targets and acquire images of the moving targets when they are detected. The processing device can acquire the image to be recognized, input the image to be recognized into a pre-trained target feature extraction model, and obtain the target features of the target object output by the target feature extraction model; determine the object matching result of the target object based on the current object feature library and the target features, wherein the current object feature library includes the registration features of registered objects; update the current object feature library based on the target features and the object matching result, and return to the step of acquiring the image to be recognized, so as to perform target recognition using the new current object feature library.

[0207] It is evident that the processing device can update the current object feature library by utilizing the object features extracted by the target feature extraction model and the obtained object matching results during the target recognition process. Through adaptation and self-evolution, it achieves dynamic updating and continuous optimization of the current object feature library without human intervention, enabling the current object feature library to continuously adapt to feature changes, thereby maintaining a high accuracy of target recognition results.

[0208] As one embodiment of this application, the processing device 701 is further configured to construct a spatiotemporal association map of the registered object based on the registration name of the registered object, the acquisition time corresponding to each image to be identified, and the obtained identification result of the registered object when the identification results of the images to be identified acquired by multiple image acquisition devices correspond to the same registered object.

[0209] The processing device can perform target recognition on images to be recognized acquired by multiple image acquisition devices, obtaining recognition results for the images acquired by multiple image acquisition devices. Furthermore, for the recognition results of the same registered person among the recognition results of the images acquired by multiple image acquisition devices, the processing device can determine when and where the registered person performed what action based on the registered person's registration name, the acquisition time corresponding to each image to be recognized, and the obtained recognition results for the registered person. It then associates the recognition results of the registered person according to time, constructing a three-dimensional cross-temporal and spatial correlation map of the registered person's person-location-behavior, realizing trajectory correlation of the same user across image acquisition devices.

[0210] For example, the processing device recognizes the image to be recognized captured by the image acquisition device in the living room in the morning, and obtains the recognition result that the mother is exercising in the living room in the morning; the processing device recognizes the image to be recognized captured by the image acquisition device in the kitchen at noon, and obtains the recognition result that the mother is cooking in the kitchen at noon; the processing device recognizes the image to be recognized captured by the image acquisition device in the balcony in the afternoon, and obtains the recognition result that the mother is watering flowers on the balcony in the afternoon, thereby outputting the trajectory association message of the mother throughout the day.

[0211] As can be seen, in this embodiment, the processing device is also used to construct a cross-temporal association map of the registered object based on the registration name of the registered object, the acquisition time corresponding to each image to be identified, and the obtained identification result of the registered object when the identification results of the images to be identified acquired by multiple image acquisition devices correspond to the same registered object, thereby realizing the trajectory association of the same user across image acquisition devices.

[0212] Corresponding to the target recognition method, this application also provides a target recognition device.

[0213] like Figure 8 As shown, a target identification device includes:

[0214] Image acquisition module 801 is used to acquire the image to be recognized;

[0215] The feature acquisition module 802 is used to input the image to be identified into a pre-trained target feature extraction model and acquire the target features of the target object output by the target feature extraction model.

[0216] The matching result acquisition module 803 is used to determine the object matching result of the target object based on the current object feature library and the target feature, wherein the current object feature library includes the registration features of registered objects;

[0217] The update module 804 is used to update the current object feature library based on the target features and the object matching results, and return to the step of obtaining the image to be identified, so as to perform target recognition using the new current object feature library.

[0218] In the technical solution provided in this application embodiment, the electronic device can acquire an image to be recognized, input the image to be recognized into a pre-trained target feature extraction model, and acquire the target features of the target object output by the target feature extraction model; determine the object matching result of the target object based on the current object feature library and the target features, wherein the current object feature library includes the registration features of registered objects; update the current object feature library based on the target features and the object matching result, and return to the step of acquiring the image to be recognized, so as to use the new current object feature library for target recognition.

[0219] It is evident that electronic devices can update the current object feature library by utilizing the object features extracted by the target feature extraction model and the obtained object matching results during the target recognition process. Through self-adaptation and self-evolution, the current object feature library can be dynamically updated and continuously optimized without human intervention, enabling the current object feature library to continuously adapt to changes in factors such as personnel posture, clothing, and the environment of the collection area, thereby maintaining a high accuracy of target recognition results.

[0220] As one embodiment of this application, the apparatus further includes:

[0221] The classification result acquisition module is used to input the image to be identified into a pre-trained human attribute classifier before the step of determining the object matching result of the target object based on the current object feature library and the target features, and to obtain the human attribute classification result of the target object output by the human attribute classifier.

[0222] The update module includes:

[0223] The credibility determination submodule is used to determine the credibility of the object matching result based on the quality index of the recognition result corresponding to the image to be identified; wherein, the quality index includes at least one of the object matching degree represented by the object matching result, the accuracy of the object matching result, and the attribute recognition credibility represented by the human attribute classification result.

[0224] The update submodule is used to update the current object feature library based on the credibility level, using the object matching results and the target features.

[0225] As one embodiment of this application, the update submodule includes:

[0226] The first update unit is used to update the registration features of the registered objects indicated by the object matching results in the current object feature library using the target features when the credibility level is greater than a first preset threshold.

[0227] The second update unit is configured to output a labeling notification about the image to be identified when the confidence level is less than the first preset threshold and greater than the second preset threshold, obtain the labeling result of the user labeling the image to be identified based on the labeling notification, and update the registration feature of the registered object indicated by the labeling result in the current object feature library using the labeling result and the target feature.

[0228] The third update unit is used to output first registration prompt information when the credibility level is less than the second preset threshold, and to obtain the registration name and registration characteristics of the target object entered by the user based on the first registration prompt information, and to store the registration name and registration characteristics in the current object feature library.

[0229] As one embodiment of this application, the apparatus further includes:

[0230] The first output module is used to output a registration feature update notification for the registered object when the difference between the classification results of multiple human attributes corresponding to the registered object within a first preset time period and the human attributes represented by the registration features of the registered object is greater than a preset difference threshold.

[0231] As one embodiment of this application, the apparatus further includes:

[0232] The threshold determination module is used to determine the current matching threshold based on the relationship between the average matching degree of valid identification samples within a second preset time period before the current time and the preset maximum matching threshold and the preset minimum matching threshold, before the step of determining the object matching result of the target object based on the current object feature library and the target features. The valid identification samples are images that meet a preset image quality among the images to be identified acquired within the second preset time period. The current matching threshold is used to determine the matching degree between the target features and the registration features of registered objects in the current object feature library.

[0233] As one embodiment of this application, the threshold determination module includes:

[0234] The first determining submodule is used to determine the preset maximum matching threshold as the current matching threshold when the average matching degree of the validly identified samples in the second preset time period before the current time is greater than the preset maximum matching threshold.

[0235] The second determining submodule is used to determine the preset minimum matching threshold as the current matching threshold when the average matching degree is less than the preset minimum matching threshold;

[0236] The third determining submodule is used to determine the average matching degree as the current matching threshold when the average matching degree is not greater than the preset maximum matching threshold and not less than the preset minimum matching threshold.

[0237] As one embodiment of this application, the apparatus further includes:

[0238] The execution module is configured to, when the duration for which the average matching degree is less than the preset minimum matching threshold reaches a preset duration, perform at least one of the following three methods: output an abnormal warning of the credibility of the object matching result; output a second registration prompt message; and update the target feature extraction model using valid unidentified samples within the second preset time period.

[0239] As one embodiment of this application, the apparatus further includes:

[0240] The sample determination module is used to take the image to be identified, the target features of the target object, and the object matching result of the target object as sample data;

[0241] The training module is used to extract target sample data that meets the preset sample quality from the sample data according to a preset period, and to train the target feature extraction model using the target sample data.

[0242] As one embodiment of this application, the apparatus further includes:

[0243] The language description acquisition module is used to input the object matching result, the device identifier of the image acquisition device that acquires the image to be identified, and the target features into a preset large language model, so that the large language model determines the target name represented by the object matching result, determines the location name corresponding to the device identifier, determines the action category corresponding to the target feature based on the correspondence between the target features, preset action features, and action categories, and obtains a natural language description of the recognition result of the target object based on the target name, the location name, and the action category.

[0244] This application also provides a processing device, such as... Figure 9 As shown, it includes:

[0245] Memory 901 is used to store computer programs;

[0246] The processor 902 is used to execute the program stored in the memory 901 to implement the target recognition method provided in the embodiments of this application.

[0247] Furthermore, the aforementioned electronic device may also include a communication bus and / or a communication interface, with the processor 902, communication interface, and memory 901 communicating with each other via the communication bus.

[0248] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0249] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0250] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0251] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0252] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the target recognition methods described above.

[0253] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the target recognition methods described above.

[0254] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state drive (SSD), etc.

[0255] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0256] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system, apparatus, processing device, computer-readable storage medium, and computer program product embodiments are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0257] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A target recognition method characterized by, The method comprises: acquiring a to-be-recognized image; inputting the to-be-recognized image into a pre-trained target feature extraction model to acquire a target feature of a target object output by the target feature extraction model; determining an object matching result of the target object based on a current object feature library and the target feature, wherein the current object feature library comprises registered features of registered objects; updating the current object feature library based on the target feature and the object matching result, and returning to the step of acquiring the to-be-recognized image to perform target recognition by using a new current object feature library.

2. The method of claim 1, wherein, Before the step of determining the object matching result of the target object based on the current object feature library and the target feature, the method further comprises: inputting the to-be-recognized image into a pre-trained human body attribute classifier, and acquiring a human body attribute classification result of the target object output by the human body attribute classifier; the step of updating the current object feature library based on the target feature and the object matching result comprises: determining a credibility of the object matching result based on a quality index of a recognition result corresponding to the to-be-recognized image; wherein the quality index comprises at least one of an object matching degree represented by the object matching result, an accuracy degree of the object matching result, and an attribute recognition credibility represented by the human body attribute classification result; updating the current object feature library by using the object matching result and the target feature according to the credibility.

3. The method of claim 2, wherein, The step of updating the current object feature library by using the object matching result and the target feature according to the credibility comprises: in a case where the credibility is greater than a first preset threshold, updating a registered feature of a registered object indicated by the object matching result in the current object feature library by using the target feature; in a case where the credibility is less than the first preset threshold and greater than a second preset threshold, outputting a labeling notification about the to-be-recognized image, acquiring a labeling result of object labeling of the to-be-recognized image by a user based on the labeling notification, and updating a registered feature of a registered object indicated by the labeling result in the current object feature library by using the labeling result and the target feature; in a case where the credibility is less than the second preset threshold, outputting first registration prompt information, and acquiring a registered name and a registered feature of the target object input by the user based on the first registration prompt information, and storing the registered name and the registered feature to the current object feature library correspondingly.

4. The method of claim 2, wherein, The method further comprises: in a case where a difference value between a plurality of human body attribute classification results corresponding to the registered object within a first preset time period and human body attributes represented by the registered feature of the registered object is greater than a preset difference threshold, outputting a registered feature update notification for the registered object.

5. The method according to any one of claims 1 to 4, characterized in that, Before the step of determining the object matching result of the target object based on the current object feature library and the target feature, the method further comprises: Determine the current matching threshold based on a size relationship between an average matching degree corresponding to effective recognition samples in a second preset time period before the current moment and a preset maximum matching threshold and a preset minimum matching threshold, wherein the effective recognition samples are images meeting a preset image quality in the to-be-recognized images obtained in the second preset time period; and the current matching threshold is used to determine a matching degree between the target feature and a registered feature of a registered object in the current object feature library.

6. The method of claim 5, wherein, The step of determining the current matching threshold based on the size relationship between the average matching degree corresponding to the effective recognition samples in the second preset time period before the current moment and the preset maximum matching threshold and the preset minimum matching threshold comprises: In a case where the average matching degree corresponding to the effective recognition samples in the second preset time period before the current moment is greater than the preset maximum matching threshold, determining the preset maximum matching threshold as the current matching threshold; In a case where the average matching degree is less than the preset minimum matching threshold, determining the preset minimum matching threshold as the current matching threshold; In a case where the average matching degree is not greater than the preset maximum matching threshold and not less than the preset minimum matching threshold, determining the average matching degree as the current matching threshold.

7. The method of claim 5, wherein, The method further comprises: In a case where a duration that the average matching degree is less than the preset minimum matching threshold reaches a preset duration, performing at least one of the following three modes: outputting an object matching result credibility abnormality warning; outputting a second registration prompt information; and updating the target feature extraction model by using the effective to-be-recognized samples in the second preset time period.

8. The method of claim 5, wherein, The method further comprises: Taking the to-be-recognized images, the target feature of the target object, and the object matching result of the target object as sample data; Extracting target sample data meeting a preset sample quality from the sample data according to a preset period, and training the target feature extraction model by using the target sample data.

9. The method of claim 5, wherein, The method further comprises: Inputting the object matching result, a device identifier of an image acquisition device acquiring the to-be-recognized images, and the target feature into a preset large language model, so that the large language model determines a target name represented by the object matching result, determines a place name corresponding to the device identifier, determines an action category corresponding to the target feature based on a correspondence relationship between the target feature, a preset action feature, and an action category, and obtains a natural language description of an identification result of the target object based on the target name, the place name, and the action category.

10. A target identification system characterized by, The system comprises a processing device and distributed image acquisition devices; Each image acquisition device is configured to detect a moving target and acquire an image of the moving target when the moving target is detected. The processing device is configured to acquire to-be-recognized images acquired by the image acquisition devices and execute the method according to any one of claims 1-9.

11. The system of claim 10, wherein, The processing device is further configured to, in a case where the identification results of the to-be-identified images collected by the plurality of image collection devices correspond to a same registered object, construct a cross-time-and-space correlation graph of the registered object based on a registration name of the registered object, collection times corresponding to each to-be-identified image, and the obtained identification result of the registered object.

12. A processing device, characterized by The computer program product comprises: a memory for storing a computer program; a processor for executing the program stored in the memory to implement the method of any one of claims 1-9.

13. A computer-readable storage medium, characterized in that, The computer program is stored in the computer readable storage medium, and the computer program is executed by the processor to implement the method of any one of claims 1-9.

Images