Target perception method and device, robot, storage medium and program product

Abstract

The present disclosure provides a target perception method and device, a robot, a storage medium, and a program product, relating to the field of computer technology. The target perception method includes: assigning tracking identifiers to targets included in multiple images to be processed; identifying the target using each of N target recognition algorithms in ascending order of algorithm numbers until the target's identity is identified, where N is a natural number greater than 1, the recognition confidence of the target recognition algorithm decreases as the algorithm number increases, and the target feature vectors used by multiple target recognition algorithms are different. The present disclosure can accurately obtain the identity information of human targets when the collected data is single and the computing power is limited.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a target perception method and device, a robot, a storage medium, and a program product. Background Art

[0002] The perception, positioning, and tracking of human targets are classic problems in computer vision. Human target perception systems are used to achieve vision-based perception, positioning, and tracking of human targets. They have been applied in many scenarios, including intelligent security, autonomous driving, smart campuses, and robotics. Existing human target perception systems process multiple video streams to generate perception results. Summary of the Invention

[0003] The inventors note that identity recognition is far more important in robotics scenarios than in other scenarios. If a robot fails to recognize the identity of a human target, or incorrectly identifies the identity of a human target, the subsequent human-machine interaction experience will be greatly reduced.

[0004] In related technologies, the following characteristics exist in robot scenarios.

[0005] 1) Single Data: Due to power consumption and cost constraints, robots collect relatively single data. Typically, robots use a single image acquisition device to achieve complete information perception of human targets, making the multi-sensor, multi-view fusion verification strategy commonly used in autonomous driving scenarios unusable in robotics scenarios.

[0006] 2) Limited computing power: Under relatively low power consumption conditions, robots cannot provide computing power close to that required for autonomous driving scenarios. Therefore, human target perception systems with high computing power requirements cannot be effectively deployed in robot scenarios.

[0007] Based on this, the present disclosure provides a target perception method that can accurately obtain the identity information of a human target when the collected data is single and the computing power is limited.

[0008] In a first aspect of the present disclosure, a target perception method is provided, comprising: assigning tracking identifications to targets included in a plurality of images to be processed; identifying the targets using each of N target recognition algorithms in ascending order of algorithm numbers until the identity of the target is identified, where N is a natural number greater than 1, the recognition confidence of the target recognition algorithm decreases as the algorithm number increases, and the target feature vectors used by the plurality of target recognition algorithms are different from each other.

[0009] In some embodiments, identifying the target includes: in the process of identifying the target using the i-th target recognition algorithm, identifying the i-th target feature vector of the target, ; Determine whether the i-th information library includes identity identification information that matches the i-th target feature vector; if the i-th information library includes the identity identification information, use the identity identification information as the identity identification of the target.

[0010] In some embodiments, the determining whether the i-th information library includes identity identification information that matches the i-th target feature vector includes: calculating the similarity between the i-th target feature vector and each candidate feature vector in the i-th information library to obtain multiple similarities; determining whether the maximum similarity among the multiple similarities is greater than a similarity threshold; if the maximum similarity is greater than the similarity threshold, using the candidate feature vector corresponding to the maximum similarity as a matching feature vector; and using the identity identification information corresponding to the matching feature vector as the identity identification information that matches the i-th target feature vector.

[0011] In some embodiments, the association between the tracking identifier and the target identity identifier is written into a cache.

[0012] In some embodiments, when i is 1, the first target feature vector is written into the first field corresponding to the identity identification information in the first information library.

[0013] In some embodiments, the total number of current feature vectors in the first field is detected; if the total number of current feature vectors is greater than a total threshold, each feature vector in the first field is deleted in sequence according to the order of writing time until the total number of feature vectors in the first field is no greater than the total threshold.

[0014] In some embodiments, obtaining the j-th target feature vector of the target, ; Write the jth target feature vector into the jth field corresponding to the identity information in the jth information library.

[0015] In some embodiments, the total number of current feature vectors in the j-th field is detected; if the total number of current feature vectors is greater than a total threshold, each feature vector in the j-th field is deleted in sequence according to the order of writing time until the total number of feature vectors in the j-th field is no greater than the total threshold.

[0016] In some embodiments, identifying the target includes: identifying the Nth target feature vector of the target in the process of identifying the target using the Nth target recognition algorithm; calculating the similarity between the Nth target feature vector and the feature vector of each candidate identity identification information in the Nth information library to obtain multiple similarities; and using the candidate identity identification information corresponding to the maximum similarity among the multiple similarities as the identity of the target.

[0017] In some embodiments, when the target includes M sub-targets, whether there is an identical identity identifier among the identity identifiers of the M sub-targets is detected, where M is a natural number greater than 1; if K sub-targets among the M sub-targets have the same identity identifier, then the algorithm sequence number of the target recognition algorithm used to identify the identity identifier of each sub-target among the K sub-targets is determined to obtain K algorithm sequence numbers. ; Among the K algorithm numbers, delete the identity identifiers corresponding to the K-1 algorithm numbers except the smallest algorithm number; if the association between the identity identifier corresponding to the t-th algorithm number among the K-1 algorithm numbers and the tracking identifier has been written into the cache, delete the association from the cache, .

[0018] In some embodiments, after assigning tracking identifiers to the targets included in the multiple images to be processed, the method further includes: detecting whether the cache includes identity information corresponding to the tracking identifier; if the cache includes identity information corresponding to the tracking identifier, using the identity information as the identity of the target; if the cache does not include identity information corresponding to the tracking identifier, executing each target recognition algorithm in N target recognition algorithms in ascending order of algorithm sequence number to identify the target.

[0019] In some embodiments, each of the multiple images to be processed includes a planar image and a depth image collected simultaneously, and the planar image and the depth image both include the target; assigning a tracking identifier to the target included in the multiple images to be processed includes: detecting the area information of the target in each planar image; detecting the planar position information of each key point of the target based on the area information; obtaining the depth information of each key point based on the depth image corresponding to each planar image; determining the three-dimensional spatial position information of each key point based on the planar position information of each key point and the depth information of each key point; tracking the target based on the three-dimensional spatial position information, and assigning the tracking identifier to the target based on the tracking result.

[0020] In some embodiments, the multiple images to be processed are filtered according to preset rules to obtain a target image; and the target image is sent to a server so that the server performs multimodal processing on the target image using a multimodal model.

[0021] In some embodiments, when N is 3, the first target feature vector used by the first target recognition algorithm is the facial feature vector of the target; the second target feature vector used by the second target recognition algorithm is the human body feature vector of the target; and the third target feature vector used by the third target recognition algorithm is the human body attribute feature vector of the target.

[0022] In some embodiments, the identity information of the target is recorded, wherein the recording of the identity information of the target includes: obtaining an image to be detected; if the target is included in the image to be detected, detecting whether a designated area of the target satisfies a predetermined condition; if the designated area of the target satisfies the predetermined condition, obtaining a target feature vector of the designated area of the target; writing the identity identifier assigned to the target and the first target feature vector of the designated area into a first information library; and writing the received Nth target feature vector of the target into an Nth information library.

[0023] In a second aspect of the present disclosure, a target perception device is provided, comprising: a memory; a processor coupled to the memory, the processor being configured to execute a target perception method as described in any of the above embodiments based on instructions stored in the memory.

[0024] In a third aspect of the present disclosure, a robot is provided, comprising: a target perception device as described in any of the above embodiments; and an image acquisition device configured to acquire images and send the images to the target perception device.

[0025] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the target perception method as described in any of the above embodiments is implemented.

[0026] In a fifth aspect of the present disclosure, a computer program product is provided, comprising computer instructions, wherein when the computer instructions are executed by a processor, the target perception method as described in any of the above embodiments is implemented.

[0027] Other features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0029] Figure 1 A flowchart of a target perception method according to an embodiment of the present disclosure.

[0030] Figure 2 A flowchart of a target perception method according to another embodiment of the present disclosure is shown.

[0031] Figure 3 A flowchart of a target perception method according to another embodiment of the present disclosure.

[0032] Figure 4 A flowchart of a target perception method according to another embodiment of the present disclosure.

[0033] Figure 5 This is a schematic structural diagram of a target sensing device according to an embodiment of the present disclosure.

[0034] Figure 6 The figure is a schematic structural diagram of a robot according to an embodiment of the present disclosure.

[0035] Figure 7 A flowchart of a target perception method according to another embodiment of the present disclosure.

[0036] Figure 8 The figure is a flowchart of an identity entry method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is in no way intended to limit the present disclosure and its application or use. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.

[0038] Unless specifically stated otherwise, the relative arrangement of components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present disclosure.

[0039] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0040] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0041] In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0042] It should be noted that like reference numerals and letters refer to like items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0043] Figure 1 1 is a flow chart of a target sensing method according to an embodiment of the present disclosure. In some embodiments, the following target sensing method is performed by a target sensing device in a robot, including steps 11-12.

[0044] In step 11, tracking identifiers are assigned to targets included in a plurality of images to be processed.

[0045] In some embodiments, each of the multiple images to be processed includes a planar image and a depth image that are acquired simultaneously, and both the planar image and the depth image include an object.

[0046] For example, a depth camera may be used to capture a planar image and a depth image. The planar image may be an RGB image.

[0047] In some embodiments, the step of assigning tracking identifiers to targets included in a plurality of images to be processed includes the following steps S1-S5.

[0048] S1) Detect the target area information in each plane image.

[0049] For example, a human body detection model such as Yolov8 is used to process a planar image to detect the human body area in the planar image. The detection result is shown in the following formula (1).

[0050] (1).

[0051] In expression (1), is the pixel coordinate of the upper left corner of the human target detection box in the image, is the height of the detection box area, is the width of the detection box area, is the confidence score of the target.

[0052] S2) Detect the plane position information of each key point of the target based on the region information.

[0053] For example, a human key point detection model such as Yolov8 is used to detect the human body region to detect the position information of the human key points. If 17 human key points are detected, the detection result is shown in the following formula (2).

[0054] (2).

[0055] In expression (2), is the coordinate information of the first key point of the human body, and so on.

[0056] S3) obtaining depth information of each key point according to the depth image corresponding to each planar image.

[0057] For example, the coordinates of each key point are used to obtain the depth information of the key point in the depth image, as shown in the following formula (3).

[0058] (3).

[0059] In expression (3), is the depth information of the first key point of the human body, and so on.

[0060] S4) determining the three-dimensional spatial position information of each key point based on the planar position information of each key point and the depth information of each key point.

[0061] For example, the depth information is mapped to the three-dimensional space under the camera coordinate system to obtain the position of each key point in the three-dimensional space, as shown in the following formula (4).

[0062] (4).

[0063] In expression (4), is the 3D coordinate value of the first key point of the human body, and so on.

[0064] S5) Tracking the target according to the three-dimensional spatial position information, and assigning a tracking identifier to the target according to the tracking result.

[0065] For example, the same human target detected in multiple consecutive frames of images is given a unique identifier. express.

[0066] It should be noted here that in home scenarios, since there are fewer human targets, the human movement speed is low, and the trajectory complexity is low, in order to reduce the computing resource overhead as much as possible, the Kalman filter + Hungarian matching method can be used to track the target.

[0067] Here we select the pixel center of the target on the plane image and the aspect ratio of the target ,area and average depth As the predicted value of Kalman filtering, the calculation formula is shown in formula (5) to formula (9).

[0068] (5).

[0069] (6).

[0070] (7).

[0071] (8).

[0072] (9).

[0073] Applying the Kalman filter to a sequence of detection results of a continuous target can obtain the predicted value of the detection result of the t+1 frame based on the results of frames tn to t. The predicted value of the t+1 frame obtained based on the Kalman filter is matched with the true value obtained by the detection model of the t+1 frame using the Hungarian matching method. If the match is successful, the target represented by the true value of the t+1 frame is assigned a corresponding tracking identifier. If the match cannot be made, a new Kalman filter is created to start predicting and tracking the new detection result sequence. This solution can ensure the closure effect of a small number of moving targets in limited scenarios of robot applications. Depth information The introduction of CNN can effectively solve the problem of tracking loss or error in traditional detection box-based Kalman filters after target overlap and occlusion, achieving a balance between performance and computational resource overhead.

[0074] In some embodiments, multiple images to be processed are filtered according to preset rules to obtain a target image, and the target image is sent to a server so that the server performs multimodal processing on the target image using a multimodal model.

[0075] It should be noted here that high-level semantic processing of images requires a large amount of computing resources. Since the computing power of the robot is limited, the problem of insufficient computing power of the robot itself is effectively solved by handing complex tasks to a server deployed in the cloud, which then calls a multimodal model for processing.

[0076] For example, the multimodal model can be a VLM (Visual Language Model), which uses the VLM model to simultaneously process vision (images) and language (text) to perform high-order visual language tasks, such as recognizing emotions and behaviors of human targets.

[0077] In step 12, each of the N target recognition algorithms is used to identify the target in ascending order of the algorithm number until the identity of the target is identified. N is a natural number greater than 1. The recognition confidence of the target recognition algorithm decreases as the algorithm number increases. The target feature vectors used by multiple target recognition algorithms are different.

[0078] For example, when N is 3, the first target feature vector used by the first target recognition algorithm is the target's facial feature vector, the second target feature vector used by the second target recognition algorithm is the target's human body feature vector, and the third target feature vector used by the third target recognition algorithm is the target's human body attribute feature vector.

[0079] It should be noted here that among the above three target recognition algorithms, since the confidence of recognition using facial feature vectors is the highest and the confidence of recognition using human attribute feature vectors is the lowest, the recognition confidence of the first target recognition algorithm is the highest and the recognition confidence of the third target recognition algorithm is the lowest.

[0080] It should be noted that when each of the N target recognition algorithms is used to identify a target, the first target recognition algorithm is first used to identify the target. If the first target recognition algorithm is able to identify the target's identity, no further target recognition algorithms are used to identify the target. Conversely, if the first target recognition algorithm is unable to identify the target's identity, the second target recognition algorithm is used to identify the target, and so on.

[0081] In some embodiments, as Figure 2 As shown, in the process of identifying the target using the i-th target recognition algorithm among N target recognition algorithms, the following steps 21-23 are performed, wherein .

[0082] In step 21, the i-th target feature vector of the target is identified.

[0083] In step 22, it is determined whether the i-th information database includes identity information that matches the i-th target feature vector.

[0084] For example, the above step of determining whether the i-th information database includes identity identification information that matches the i-th target feature vector includes the following steps 221-224.

[0085] 221) Calculate the similarity between the i-th target feature vector and each candidate feature vector in the i-th information library to obtain multiple similarities.

[0086] 222) Determine whether the maximum similarity among multiple similarities is greater than a similarity threshold.

[0087] 223) When the maximum similarity is greater than the similarity threshold, the candidate feature vector corresponding to the maximum similarity is used as the matching feature vector.

[0088] 224) The identity information corresponding to the matching feature vector is used as the identity information matched with the i-th target feature vector.

[0089] In step 23, if the i-th information database includes identity information, the identity information is used as the identity of the target.

[0090] For example, the i-th target feature vector is a face feature vector, and the i-th information library is a face feature library, which includes multiple identity identification information and a face feature vector corresponding to each identity identification information.

[0091] For example, identity information The storage format of the corresponding multiple facial feature vectors is shown in the following formula (10).

[0092] (10).

[0093] If the i-th target feature vector of the target is , the eigenvector and the feature vector in the i-th information base The similarity is the largest, and the similarity is greater than the similarity threshold, then the feature vector As the matching feature vector, since the feature vector and identity information Correspondingly, the identity information As the target identity, that is, the target identity is .

[0094] It should be noted here that if the eigenvector and the feature vector in the i-th information base The similarity is the largest, but the similarity is less than the similarity threshold, which means that the currently used i-th target recognition algorithm cannot recognize the identity of the target.

[0095] In some embodiments, the association between the tracking identifier and the target's identity is written into a cache, so that the identity associated with the tracking identifier can be directly queried through the cache.

[0096] For example, if the target's tracking ID is , the identified identity is , then the content stored in the cache is as shown in the following formula (11).

[0097] (11).

[0098] In expression (11), The source of the identity, To identify the identity using the current i-th target recognition algorithm When , the similarity between the i-th target feature vector of the target and the matching feature vector in the i-th information library.

[0099] In some embodiments, when i is 1, the first target feature vector is written into the first field corresponding to the identity information in the first information library.

[0100] It should be noted here that since the recognition confidence of the first target recognition algorithm is the highest, the identity identified by the first target recognition algorithm can be considered correct. At the same time, the first target feature vector extracted by the first target recognition algorithm can accurately describe the latest status of the target. Therefore, by writing the first target feature vector of the target into the first field corresponding to the identity information in the first information database, the subtle changes in the feature vector of the target can be recorded, so as to further improve the success rate of recognition.

[0101] For example, in the first information base, the identity information The corresponding fields are shown in the above formula (10). If the identity identified by the first target recognition algorithm is , then the target's current facial feature vector ( ) is written into the first information base, and the identity information The corresponding fields are shown in formula (12).

[0102] (12).

[0103] In some embodiments, after the first target feature vector of the target is written into the first field corresponding to the identity information in the first information database, the total number of feature vectors currently in the first field is further detected. If the total number of feature vectors currently in the first field is greater than a total threshold, each feature vector in the first field is deleted sequentially in the order of writing time until the total number of feature vectors in the first field is no greater than the total threshold.

[0104] It should be noted that, under normal circumstances, a target's feature vector will change to a certain extent over time. Compared to earlier written target feature vectors, the currently written target feature vector is more conducive to target recognition. Therefore, if the field corresponding to the identity information contains too many target feature vectors, deleting the earlier written target feature vectors can effectively save storage space without affecting recognition performance.

[0105] For example, in the first information base, the identity information The corresponding fields are shown in formula (12). By deleting the target feature vector written earlier , then with the identity information The corresponding fields are shown in formula (13).

[0106] (13).

[0107] In some embodiments, when the first target feature vector is written into the first field corresponding to the identity information in the first information library, the j-th target feature vector of the target is further obtained. , write the jth target feature vector into the jth field corresponding to the identity information in the jth information library.

[0108] It should be noted that since the first target recognition algorithm has the highest recognition confidence, the identity identified by this first target recognition algorithm can be considered correct. In this case, other feature vectors of the target are provided and stored in the fields corresponding to the identity information in the corresponding information database. This allows for recording subtle changes in the other feature vectors of the target, further improving the recognition success rate.

[0109] For example, after the target's facial feature vector is written into a first information library for storing facial feature vectors, the target's body feature vector is further written into a second information library for storing body feature vectors.

[0110] In some embodiments, after the jth target feature vector is written into the jth field corresponding to the identity information in the jth information library, the total number of current feature vectors in the jth field is further detected. If the total number of current feature vectors is greater than the total number threshold, each feature vector in the jth field is deleted in sequence according to the order of writing time until the total number of feature vectors in the jth field is no greater than the total number threshold.

[0111] Because the target's feature vector changes over time, the currently written target feature vector is more conducive to target recognition than the earlier written target feature vector. Therefore, if the field corresponding to the identity information contains too many target feature vectors, deleting the earlier written target feature vectors can effectively save storage space without affecting the recognition effect.

[0112] In some embodiments, as Figure 3 As shown, in the process of identifying a target using the Nth target recognition algorithm among N target recognition algorithms, the following steps 31-33 are performed.

[0113] In step 31, the Nth target feature vector of the target is identified.

[0114] In step 32, the similarity between the Nth target feature vector and the feature vector of each candidate identity identification information in the Nth information database is calculated to obtain a plurality of similarities.

[0115] In step 33, the candidate identity information corresponding to the maximum similarity among the multiple similarities is used as the target identity.

[0116] It should be noted that since the first N-1 of the N target recognition algorithms failed to identify the target's identity, the Nth target recognition algorithm is required to identify the target's identity. Among the multiple similarities calculated above, the candidate identity corresponding to the maximum similarity is used as the target's identity, without determining whether the maximum similarity exceeds a predetermined threshold. This results in a low confidence level in the recognition result. In this case, the association between the tracking identifier and the identified identity is not written to the cache.

[0117] In some embodiments, when a target includes M sub-targets, M is a natural number greater than 1, and the following steps S21-S24 are performed.

[0118] S21) Detect whether there is an identical identity among the identities of the M sub-targets.

[0119] S22) If K sub-goals among the M sub-goals have the same identity, then determine the algorithm sequence number of the target recognition algorithm used to identify the identity of each sub-goal among the K sub-goals to obtain K algorithm sequence numbers. .

[0120] S23) Among the K algorithm numbers, delete the identities corresponding to the K-1 algorithm numbers except the smallest algorithm number.

[0121] S24) If the association between the identity identifier and the tracking identifier corresponding to the t-th algorithm serial number among the K-1 algorithm serial numbers has been written into the cache, then the association is deleted from the cache. .

[0122] It should be noted here that if the image contains two targets, and the tracking mark of target 1 is , the tracking identifier of target 2 is , but the identities of target 1 and target 2 are both , which indicates that an error occurred during the target identification process. To resolve this error, it is necessary to detect the identification methods of target 1 and target 2. For example, the identification of target 1 It is identified by the first target recognition algorithm, and the identity of target 2 It is identified by the second target recognition algorithm. Since the recognition confidence of the first target recognition algorithm is greater than the recognition confidence of the second target recognition algorithm, the identity of target 2 can be determined. Identification error. In this case, delete the target 2's identity This ensures that in one frame of image, multiple targets do not share the same identity.

[0123] In addition, if the tracking marker of target 2 and identity The association relationship has been written to the cache. To avoid the incorrect association relationship from causing erroneous effects on target identification, delete the tracking identifier from the cache. and identity The relationship between them.

[0124] In the target perception method provided by the above-mentioned embodiments of the present disclosure, multiple target recognition algorithms with different recognition confidence levels are set up. Each of the multiple target recognition algorithms is used to identify the target in descending order of recognition confidence until the target's identity is determined. This allows the target's identity to be accurately identified using less data, even with limited computing power.

[0125] In some embodiments, the target registration is achieved by recording the target's identity information. For example, the step of recording the target's identity information includes steps S31-S35.

[0126] S31) Acquire an image to be detected.

[0127] S32) When the image to be detected includes a target, detecting whether a designated area of the target satisfies a predetermined condition.

[0128] For example, the predetermined conditions include: whether the confidence level of face detection is greater than a threshold, whether the current image includes only one face, and the like.

[0129] S33) When the designated area of the target satisfies a predetermined condition, obtaining a target feature vector of the designated area of the target.

[0130] S34) Writing the identity assigned to the target and the first target feature vector of the designated area into the first information database.

[0131] S35) Writing the identity assigned to the target and the Nth target feature vector of the target into the Nth information database.

[0132] It should be noted here that among the N target recognition algorithms, the recognition confidence of the first target recognition algorithm is the highest. In order to ensure the recognition effect of the first target recognition algorithm, when the target is registered, the first target feature vector of the target used by the first target recognition algorithm is written into the first information database corresponding to the first target recognition algorithm.

[0133] In addition, among the N target recognition algorithms, the recognition confidence of the Nth target recognition algorithm is the lowest. In order to ensure the recognition effect of the Nth target recognition algorithm, when registering the target, the Nth target feature vector of the target used by the Nth target recognition algorithm is written into the Nth information library corresponding to the Nth target recognition algorithm.

[0134] Figure 4 4 is a flow chart of a target sensing method according to another embodiment of the present disclosure. In some embodiments, the following target sensing method is performed by a target sensing device in a robot, including steps 41-44.

[0135] In step 41 , tracking identifiers are assigned to targets included in a plurality of images to be processed.

[0136] In step 42, it is detected whether the cache includes identity information corresponding to the tracking identifier.

[0137] If the cache includes the identity information corresponding to the tracking identifier, then step 43 is executed. Otherwise, step 44 is executed.

[0138] In step 43, the identity information corresponding to the tracking identifier in the cache is used as the identity of the target.

[0139] In step 44, each of the N target recognition algorithms is used to recognize the target in ascending order of algorithm number.

[0140] It should be noted here that if the cache stores the association between the tracking identifier and the identity identifier, the identity identifier corresponding to the tracking identifier can be directly obtained through the cache. In this case, there is no need to use the target recognition algorithm for target recognition, which can effectively save computing resources.

[0141] Figure 5 This is a schematic structural diagram of a target sensing device according to an embodiment of the present disclosure.

[0142] like Figure 5 As shown, the target sensing device 50 can be in the form of a general-purpose computing device and includes a memory 51, a processor 52, and a bus 53 for connecting different system components.

[0143] The memory 51 may include, for example, system memory, non-volatile storage media, and the like. The system memory may store, for example, an operating system, application programs, a boot loader, and other programs. The system memory may include volatile storage media, such as random access memory (RAM) and / or cache memory. The non-volatile storage media may store, for example, instructions corresponding to at least one embodiment of the target sensing method being executed. Non-volatile storage media include, but are not limited to, disk storage, optical storage, and flash memory.

[0144] The processor 52 can be implemented using a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, or discrete hardware components such as discrete gates or transistors. Accordingly, each module, such as the acquisition module, the calculation module, and the adjustment module, can be implemented by a central processing unit (CPU) executing instructions in memory that execute corresponding steps, or by dedicated circuits that execute corresponding steps.

[0145] For example, the processor 52 is configured to execute instructions stored in the memory to implement the following Figures 1 to 4 The method according to any one of the embodiments.

[0146] The bus 53 may use any of a variety of bus architectures, including, but not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, and a Peripheral Component Interconnect (PCI) bus.

[0147] These interfaces 54, 55, and 56 of the target sensing device 50, as well as the memory 51 and the processor 52, can be connected via a bus 53. The input / output interface 54 provides a connection interface for input / output devices such as a display, mouse, and keyboard. The network interface 55 provides a connection interface for various networked devices. The storage interface 56 provides a connection interface for external storage devices such as floppy disks, USB flash drives, and SD cards.

[0148] Here, various aspects of the present disclosure are described with reference to flowcharts and / or block diagrams of methods, devices, and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks, can be implemented by computer-readable program instructions.

[0149] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable device to produce a machine, so that the processor executes the instructions to produce means for implementing the functions specified in one or more blocks in the flowcharts and / or block diagrams.

[0150] These computer-readable program instructions may also be stored in a computer-readable memory, which cause the computer to operate in a specific manner to produce an article of manufacture, including instructions for implementing the functions specified in one or more blocks in the flowcharts and / or block diagrams.

[0151] The present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects.

[0152] The present disclosure also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, which, when executed by a processor, implement the following Figures 1 to 4 The method according to any one of the embodiments.

[0153] The present disclosure also provides a computer program product, including computer instructions, wherein when the computer instructions are executed by a processor, the following is achieved: Figures 1 to 4 The method according to any one of the embodiments.

[0154] Figure 6 The figure is a schematic structural diagram of a robot according to an embodiment of the present disclosure.

[0155] like Figure 6 As shown, the robot 60 includes a target sensing device 50 and an image acquisition device 61. The target sensing device 50 is Figure 5 The target sensing device involved in any embodiment.

[0156] The image acquisition device 61 is configured to acquire images and send the images to the target perception device 50 .

[0157] In some embodiments, the image acquisition device 61 is a depth camera that can simultaneously acquire planar images and depth images.

[0158] The present disclosure is described below through a specific example.

[0159] For example, the target perception method uses three target recognition algorithms, wherein the first target recognition algorithm uses a face feature vector, the second target recognition algorithm uses a body feature vector, and the third target recognition algorithm uses a body attribute feature vector.

[0160] It should be noted here that among the above three target recognition algorithms, since the confidence of recognition using facial feature vectors is the highest and the confidence of recognition using human attribute feature vectors is the lowest, the recognition confidence of the first target recognition algorithm is the highest and the recognition confidence of the third target recognition algorithm is the lowest.

[0161] like Figure 7 As shown, the target's facial features are first entered into the facial feature library through the identity entry process, and the target's body attributes are also entered into the body attribute library. Next, the target is identified using the cache, the first target recognition algorithm, the second target recognition algorithm, and the third target recognition algorithm in sequence until the target's identity is determined.

[0162] like Figure 8 As shown, the identity entry process includes steps 81-86.

[0163] In step 81, a human body detection model is used to perform human body detection on the collected image.

[0164] In step 82, based on the human body detection result, it is determined whether there is a human body in the image.

[0165] If a human body exists in the image, step 83 is executed. Otherwise, step 86 is executed.

[0166] In step 83, the face detection model is used to detect the face.

[0167] In step 84, it is determined whether the face detection result is legal.

[0168] For example, the legality of the face detection result can be determined based on the following conditions.

[0169] 1) Whether the confidence level of face detection is greater than a threshold. This value can be used to ensure that the current user is facing the camera.

[0170] 2) Whether there is only one face in the current image, and whether the pixel area of the face area is greater than the set value.

[0171] 3) Whether there is a depth difference when the current facial key points are mapped to the depth map is used to determine whether it is a real face and prevent the use of photos or face images displayed on the screen for registration.

[0172] If the face detection result is legal, execute step 85; otherwise, execute step 86.

[0173] In step 85, the face detection result is processed using the face feature model to obtain a face feature vector , and the facial feature vector and the identity assigned to the human body The corresponding relationship is stored in the face feature database.

[0174] In addition, the human body attribute vector corresponding to the human body attribute information of the human body is obtained. , the human body attribute vector and the identity of the person The corresponding relationship is stored in the human body attribute library.

[0175] It should be noted that in different application scenarios, the attributes of the human body can be set differently. For example, in a home application scenario, the human body attribute information may include the following attributes: (1) whether the person is a woman, (2) whether the person is a child, (3) whether the person is wearing glasses, (4) whether the person is wearing a hat, and (5) whether the person is obese. It includes 5 information bits, each of which is represented by 0 or 1. When the information bit is 1, it means the corresponding attribute exists. For example, the human body attribute vector If the first information bit is 1, it indicates that the target person is a female, and if the second information bit is 0, it indicates that the target person is not a child.

[0176] In step 86, an error message is returned.

[0177] return Figure 7 ,When executing the target perception process, the following steps are performed.

[0178] 1) Use a lightweight human body detection model to process the collected planar image to detect the human body area in the planar image. The detection result is shown in the above formula (1).

[0179] For example, human detection models include Yolov8.

[0180] 2) Use a lightweight human key point detection model to process the human body area and detect the planar position information of each key point of the target human body.

[0181] For example, a human key point detection model such as Yolov8 is used to detect the human body area and 17 human key points are detected. The detection result is shown in the above formula (2).

[0182] 3) Based on the depth image corresponding to each plane image, the depth information of each key point is obtained. Based on the plane position information and the depth information of each key point, the 3D spatial position information of each key point is determined.

[0183] For example, if 17 key points of the target human body are detected, the depth information of these 17 key points is shown in the above formula (3). The depth information is mapped to the three-dimensional space under the camera coordinate system, and the position of each key point in the three-dimensional space is obtained, as shown in the above formula (4).

[0184] Next, step 4.1 and step 4.2 can be performed in parallel.

[0185] 4.1) Filter multiple images to be processed according to preset rules to obtain a target image, and send the target image to the server so that the server can use the VLM to perform multimodal processing on the target image to identify the target person's emotions, behaviors, etc.

[0186] It should be noted here that high-level semantic processing of images requires a large amount of computing resources. Since the computing power of the robot is limited, the problem of insufficient computing power of the robot itself is effectively solved by handing complex tasks to a server deployed in the cloud, which then calls a multimodal model for processing.

[0187] 4.2) Track the target based on the three-dimensional spatial position information and assign a tracking identifier to the target based on the tracking results.

[0188] For example, the same human target detected in multiple consecutive frames of images is given a unique identifier. express.

[0189] It should be noted here that in home scenarios, since there are fewer human targets, the human movement speed is low, and the trajectory complexity is low, in order to reduce the computing resource overhead as much as possible, the Kalman filter + Hungarian matching method can be used to track the target.

[0190] 5) Check if the cache contains the tracking identifier Corresponding identity information. If the cache contains tracking identification The corresponding identity information is used as the identity of the target person. .

[0191] It should be noted here that if the cache stores the association between the tracking identifier and the identity identifier, the identity identifier corresponding to the tracking identifier can be directly obtained through the cache. In this case, there is no need to use the target recognition algorithm for target recognition, which can effectively save computing resources.

[0192] If the cache does not include the tracking identifier The corresponding identity information may have the following three situations: (1) This is a new target person; (2) The target person has been identified before, but due to occlusion or large movement amplitude or other reasons, the tracking identification Lost; (3) With the tracking mark The target person has not been identified effectively before. In this case, the identity of the target person needs to be re-identified.

[0193] 6) Obtain the face area of the target person by using the face detection model.

[0194] For example, by using the insightface library, face area detection is achieved.

[0195] 7) Obtain the facial feature vector of the target person by using the face recognition model Next, calculate the facial feature vector The feature similarity with each facial feature vector in the facial feature library.

[0196] It should be noted here that in the face feature database, identity information The storage format of the corresponding multiple facial feature vectors is shown in the above formula (10).

[0197] If the target person's facial feature vector is , the eigenvector and the feature vector in the face feature library The similarity is the largest, and the similarity is greater than the similarity threshold, then the feature vector As the matching feature vector. and identity information Correspondingly, the identity information As the identity of the target person, that is, the identity of the target person is .

[0198] Since the identity of the target person has been identified, the subsequent target perception process is no longer executed.

[0199] It should be noted here that if the eigenvector and the feature vector in the face feature library The similarity is the largest, but the similarity is less than the similarity threshold, which means that the identity of the target face cannot be recognized by using the face feature vector.

[0200] After identifying the target person’s identity After that, you need to further perform the following process.

[0201] 7.1) Tracking mark of target human body and the identity is The association relationship is written into the cache, as shown in the above formula (11).

[0202] 7.2) The facial feature vector of the target person Write into the facial feature library and identity In the corresponding field, as shown in the above formula (12), the face feature vector In the above formula (12), it is represented as the eigenvector .

[0203] Next, detection and identification The total number of current facial feature vectors in the corresponding field. If the total number of current feature vectors is greater than the total threshold, each feature vector in the field is deleted in the order of writing time until the total number of feature vectors in the field is no greater than the total threshold.

[0204] It's important to note that the target person's facial feature vector typically changes over time. Compared to earlier written facial feature vectors, the currently written facial feature vector is more conducive to identifying the target person. Therefore, if this field contains too many facial feature vectors, deleting the older ones can effectively save storage space without affecting recognition performance.

[0205] For example, in the above formula (12), by deleting the target feature vector written earlier , then with the identity The corresponding fields are shown in formula (13) above.

[0206] 7.3) Obtain the target human body feature vector by using the human body feature model , and the human feature vector Write the identity tag into the human feature database corresponding fields.

[0207] For example, the human feature model includes the Resnet18 model.

[0208] In the human body feature vector Write the identity tag into the human feature database After entering the corresponding field, further detect the total number of current human feature vectors in the field. If the total number of current human feature vectors is greater than the total threshold, delete each feature vector in the field in the order of writing time until the total number of feature vectors in the field is no greater than the total threshold.

[0209] Because the target person's body feature vector changes over time, the currently written body feature vector is more conducive to the target person's recognition than the previously written body feature vector. Therefore, if there are too many body feature vectors in this field, deleting the previously written body feature vectors can effectively save storage space without affecting the recognition effect.

[0210] 8) If the target person’s identity cannot be identified through the facial feature vector, use the human feature model to identify the target person’s human feature vector Next, calculate the human feature vector The feature similarity with each human feature vector in the human feature library.

[0211] It should be noted here that in the human feature database, identity information The storage format of the corresponding multiple human feature vectors is shown in the following formula (14).

[0212] (14).

[0213] If the target human body feature vector is , human feature vector and the feature vector in the human feature library The similarity is the largest, and the similarity is greater than the similarity threshold, then the feature vector As the matching feature vector. and identity information Correspondingly, the identity information As the identity of the target person, that is, the identity of the target person is .

[0214] It should be noted here that if the target person is facing away from the robot, or the target person's face is not facing the robot, in this case, the robot cannot effectively obtain the facial feature information of the target person, but can obtain the body feature information of the target person. Therefore, if the identity of the target person cannot be identified through facial recognition, the identity of the target person can be effectively identified through body feature recognition.

[0215] Since the identity of the target person has been identified, the subsequent target perception process is no longer executed.

[0216] It should be noted here that if the eigenvector and the feature vector in the human feature library The similarity is the largest, but the similarity is less than the similarity threshold, which means that the identity of the target face cannot be recognized by using the human feature vector.

[0217] After identifying the target person’s identity Then, the target human body is tracked and the identity is The association relationship is written into the cache, and the storage format is shown in the following formula (15).

[0218] (15).

[0219] In expression (15), The source of the identity, To identify the identity When , the similarity between the human feature vector of the target human body and the matching feature vector in the human feature library is calculated.

[0220] It's also important to note that compared to facial recognition, recognition based on body features is more susceptible to interference. For example, if two people wear similar clothing or walk with a similar gait, misidentification is likely to occur. Therefore, after successfully identifying a target person based on body features, there's no need to update the facial and body feature libraries.

[0221] 9) If the target person’s identity cannot be identified through the human body feature vector, use the human body attribute model to identify the target person’s human body attribute feature vector Next, calculate the human attribute feature vector The feature similarity with each human attribute feature vector in the human attribute library.

[0222] It should be noted here that in the human attribute library, identity information The storage format of the corresponding human attribute feature vector is shown in the following formula (16).

[0223] (16).

[0224] If the target person's body attribute feature vector is , human body attribute feature vector and the attribute feature vector in the human body attribute library The similarity is the largest, then the attribute feature vector Corresponding identity information As the identity of the target person, that is, the identity of the target person is .

[0225] It should also be noted that human attribute recognition is mainly used to determine some preset attributes of the current target human body, such as gender, age, accessories, body shape, etc. The attribute information of each target human body is written into the human attribute library during the identity entry process. Here, the human attribute feature vector Perform fuzzy matching with each human attribute feature vector in the human attribute library. For example, the human attribute feature vector includes 4 information bits, where the first information bit corresponds to gender, the second information bit corresponds to age, the third information bit corresponds to accessories, and the fourth information bit corresponds to body shape. If the target human’s human attribute feature vector and the attribute feature vector in the human body attribute library Compared with the above, if there are 3 information bits that are consistent, the human attribute feature vector can be determined. and the attribute feature vector in the human body attribute library The similarity is 3.

[0226] It should be noted that due to the low confidence of using human attribute feature vectors for identity recognition, the target human body will not be tracked. And the identity identified by the human attribute feature vector is The association relationship is written into the cache.

[0227] from Figure 8 As can be seen in the figure, the identity of the target person can be obtained through one of the four processing processes: caching, face recognition, human feature recognition, and human attribute recognition. Therefore, through the identity aggregation logic, while effectively identifying the identity of the target person, it can also ensure that a target person has only one identity.

[0228] 10) If the image includes multiple human subjects and the same identity is present among these subjects, only the one with the highest confidence is retained, and the others are deleted. Among the three identification methods mentioned above (identification by facial features, identification by body features, and identification by body attributes), the identity identified by facial features has the highest confidence, while the identity identified by body attributes has the lowest confidence.

[0229] If the image contains two target bodies, the tracking identifier of target body 1 is , the tracking identifier of target human 2 is , but the identity tags of target person 1 and target person 2 are both Target person 1's identity It is identified by facial features, and the identity of the target person 2 It is identified by human features. Since the identity identified by facial features has the highest confidence, the identity of target person 2 is deleted. This ensures that in one frame of image, multiple targets do not share the same identity.

[0230] In addition, due to the tracking mark of the target human body 2 and identity The association relationship has been written to the cache. To avoid the incorrect association relationship from causing erroneous effects on target identification, delete the tracking identifier from the cache. and identity The relationship between them.

[0231] For another example, if the image contains two target bodies, the tracking mark of target body 1 is , the tracking identifier of target human 2 is , but the identity tags of target person 1 and target person 2 are both Target person 1's identity It is identified by facial features, and the identity of the target person 2 It is identified by human attributes. Since the identity identified by facial features has the highest confidence, the identity of target person 2 is deleted. This ensures that in one frame of image, multiple targets do not share the same identity.

[0232] In addition, due to the tracking mark of the target human body 2 and identity The association is not written to the cache, so there is no need to delete the tracking identifier from the cache. and identity Operations on the association relationship.

[0233] By implementing the above-mentioned embodiments of the present disclosure, the following beneficial effects can be obtained.

[0234] (1) By using multiple models to identify the identity of the target from different dimensions, the effectiveness of identity recognition can be guaranteed as much as possible.

[0235] (2) Construct an online feature update mechanism so that the stored features can represent different situations corresponding to the same target at different times.

[0236] (3) Through identity aggregation logic, while effectively identifying the identity of the target person, it can also ensure that a target person has only one identity.

[0237] (4) Run high-frequency, low-computing algorithms on local, limited computing power, while run low-frequency, high-computing algorithms on the cloud. This ensures that even with limited computing resources, the robot can still provide basic human target perception capabilities and provide higher-level human semantic understanding with the help of cloud computing power.

[0238] In some embodiments, the functional units described above may be implemented as general-purpose processors, programmable logic controllers (PLCs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or any appropriate combination thereof, for performing the functions described in the present disclosure.

[0239] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0240] The description of the present disclosure is provided for purposes of illustration and description and is not intended to be exhaustive or to limit the disclosure to the disclosed form. Many modifications and variations will be apparent to those skilled in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present disclosure and to enable those skilled in the art to understand the present disclosure and design various embodiments with various modifications suitable for specific applications.

Claims

1. A target perception method, comprising: assigning tracking identifiers to targets included in a plurality of images to be processed; In ascending order of algorithm numbers, each of N target recognition algorithms is used to identify the target until an identity of the target is identified, where N is a natural number greater than 1, and the recognition confidence of the target recognition algorithm decreases as the algorithm number increases, and the target feature vectors used by the multiple target recognition algorithms are different from each other; In the case where the target includes M sub-targets, detecting whether there is an identical identity identifier among the identity identifiers of the M sub-targets, where M is a natural number greater than 1; If K sub-targets among the M sub-targets have the same identity identifier, then determine the algorithm serial number of the target recognition algorithm used to identify the identity identifier of each sub-target among the K sub-targets to obtain K algorithm serial numbers, ; Among the K algorithm numbers, the identities corresponding to K-1 algorithm numbers except the smallest algorithm number are deleted.

2. The target perception method according to claim 1, wherein: The identifying of the target comprises: In the process of identifying the target using the i-th target recognition algorithm, identifying the i-th target feature vector of the target, ; Determining whether the i-th information database includes identity identification information that matches the i-th target feature vector; If the identity identification information is included in the i-th information database, the identity identification information is used as the identity identification of the target.

3. The target perception method according to claim 2, wherein: The determining whether the i-th information database includes identity information that matches the i-th target feature vector includes: Calculating the similarity between the i-th target feature vector and each candidate feature vector in the i-th information library to obtain multiple similarities; Determining whether a maximum similarity among the multiple similarities is greater than a similarity threshold; In a case where the maximum similarity is greater than the similarity threshold, taking the candidate feature vector corresponding to the maximum similarity as the matching feature vector; The identity identification information corresponding to the matching feature vector is used as the identity identification information matched with the i-th target feature vector.

4. The target perception method according to claim 2, further comprising: The association relationship between the tracking identifier and the target identity identifier is written into a cache.

5. The target perception method according to claim 2, further comprising: When i is 1, the first target feature vector is written into the first field corresponding to the identity information in the first information library.

6. The target perception method according to claim 5, further comprising: Detecting the total number of current feature vectors in the first field; If the total number of the current feature vectors is greater than the total number threshold, each feature vector in the first field is deleted in sequence according to the order of writing time until the total number of feature vectors in the first field is no greater than the total number threshold.

7. The target perception method according to claim 5, further comprising: Get the j-th target feature vector of the target, ; The j-th target feature vector is written into the j-th field corresponding to the identity identification information in the j-th information library.

8. The target perception method according to claim 7, further comprising: Detecting the total number of current feature vectors in the j-th field; If the total number of the current feature vectors is greater than the total number threshold, each feature vector in the j-th field is deleted in sequence according to the order of writing time until the total number of feature vectors in the j-th field is no greater than the total number threshold.

9. The target perception method according to claim 1, wherein: The identifying of the target comprises: In the process of identifying the target using the Nth target recognition algorithm, identifying the Nth target feature vector of the target; Calculating the similarity between the Nth target feature vector and the feature vector of each candidate identity identification information in the Nth information database to obtain multiple similarities; The candidate identity identification information corresponding to the maximum similarity among the multiple similarities is used as the identity identification of the target.

10. The target perception method according to claim 1, further comprising: If the association between the identity identifier corresponding to the tth algorithm number among the K-1 algorithm numbers and the tracking identifier has been written into the cache, then the association is deleted from the cache. .

11. The target perception method according to claim 1, wherein: After assigning tracking identifiers to the targets included in the plurality of images to be processed, the method further includes: Detecting whether the cache includes identity information corresponding to the tracking identifier; If the cache includes identity information corresponding to the tracking identifier, using the identity information as the identity of the target; If the cache does not include the identity identification information corresponding to the tracking identifier, the target is identified by using each target identification algorithm in N target identification algorithms in ascending order of algorithm sequence number.

12. The target perception method according to claim 1, wherein: Each of the plurality of images to be processed includes a planar image and a depth image collected simultaneously, and both the planar image and the depth image include the target; The assigning of tracking identifiers to targets included in the plurality of images to be processed comprises: Detecting regional information of the target in each plane image; Detecting the plane position information of each key point of the target according to the area information; Acquire depth information of each key point according to the depth image corresponding to each planar image; Determining the three-dimensional spatial position information of each key point according to the plane position information of each key point and the depth information of each key point; The target is tracked according to the three-dimensional spatial position information, and the tracking identifier is assigned to the target according to the tracking result.

13. The target perception method according to claim 12, further comprising: Filtering the plurality of images to be processed according to preset rules to obtain a target image; The target image is sent to a server so that the server performs multimodal processing on the target image using a multimodal model.

14. The target perception method according to claim 1, wherein: When N is 3, The first target feature vector used by the first target recognition algorithm is the face feature vector of the target; The second target feature vector used by the second target recognition algorithm is the human feature vector of the target; The third target feature vector used by the third target recognition algorithm is the human attribute feature vector of the target.

15. The target perception method according to any one of claims 1 to 14, further comprising: Recording the identity information of the target, wherein recording the identity information of the target includes: Obtain the image to be detected; In a case where the target is included in the image to be detected, detecting whether a designated area of the target meets a predetermined condition; When the designated area of the target satisfies the predetermined condition, obtaining a target feature vector of the designated area of the target; Writing the identity assigned to the target and the first target feature vector of the designated area into a first information database; The identity assigned to the target and the Nth target feature vector of the target are written into the Nth information database.

16. A target sensing device, comprising: Memory; A processor is coupled to the memory, and the processor is configured to execute the target perception method according to any one of claims 1 to 15 based on instructions stored in the memory.

17. A robot comprising: The target sensing device according to claim 16; The image acquisition device is configured to acquire images and send the images to the target perception device.

18. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the target perception method as described in any one of claims 1 to 15 is implemented.

19. A computer program product comprising computer instructions, wherein when the computer instructions are executed by a processor, the target perception method according to any one of claims 1 to 15 is implemented.

Images