Target object recognition method, electronic device, computer storage medium, and program product

Abstract

An embodiment of the present application provides a target object identification method, an electronic device, a computer storage medium and a program product, wherein the target object identification method comprises: acquiring a video frame sequence and performing multi-target tracking on the video frame sequence to obtain at least one human body trajectory sequence; performing target object identification based on the human body trajectory sequence according to a preset multi-level cascade identification strategy to identify a target object; wherein the multi-level cascade identification strategy comprises at least two of a face recognition strategy, a pedestrian re-identification strategy and a multi-modal visual language model identification strategy which are sequentially cascaded. Through the embodiment of the present application, the overall robustness and coverage of identifying the target object are improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a target object recognition method, electronic device, computer storage medium, and computer program product. Background Technology

[0002] In scenarios such as retail store operations and management, optimizing customer service and accurately identifying and evaluating sales staff's customer service behavior are crucial for measuring store service quality and managing staff allocation. Currently, retail stores commonly deploy high-position image acquisition devices such as cameras to capture video streams, identify sales staff, and then evaluate their service.

[0003] However, existing methods for identifying sales staff based on video streams often suffer from poor image quality and frequent obstructions due to the high installation position of the high-position image acquisition equipment, resulting in tilted acquisition angles. This makes accurate identification of sales staff impossible. Consequently, it becomes difficult to accurately evaluate the sales staff's service. Similar problems exist in other similar scenarios. Summary of the Invention

[0004] In view of this, embodiments of this application provide a target object identification scheme to at least partially solve the above-mentioned problems.

[0005] According to a first aspect of the embodiments of this application, a target object recognition method is provided, comprising: acquiring a video frame sequence and performing multi-target tracking on the video frame sequence to obtain at least one human trajectory sequence; and performing target object recognition based on the human trajectory sequence according to a preset multi-level cascaded recognition strategy to identify the target object; wherein the multi-level cascaded recognition strategy includes at least two of the following: a face recognition strategy, a pedestrian re-identification strategy, and a multimodal visual language model recognition strategy cascaded sequentially.

[0006] According to a second aspect of the present application, an electronic device is provided, including: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus; the memory is used to store at least one executable instruction, which causes the processor to perform an operation corresponding to the method described in the first aspect.

[0007] According to a third aspect of the embodiments of this application, a computer storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0008] According to a fourth aspect of the embodiments of this application, a computer program product is provided, including computer instructions that instruct a computing device to perform an operation corresponding to the method described in the first aspect.

[0009] According to the solution provided in this application, by performing multi-target tracking on video frame sequences and using human trajectory sequences as processing units, a spatiotemporally continuous data foundation can be provided for subsequent target object recognition, avoiding the adverse effects of image quality fluctuations on the stability of recognition results in single video frames. Based on this, a multi-level cascaded recognition strategy is employed, cascading at least two of the following strategies in descending order of priority: face recognition, pedestrian re-identification, and multimodal visual language model recognition. Each level progressively degrades and complements the others. When face recognition fails, pedestrian re-identification and / or multimodal visual language model recognition provide effective degradation protection, improving the overall robustness and coverage of target object recognition (such as sales assistants in retail stores). Furthermore, if subsequent evaluation of sales assistant services is required, accurate evidence and data foundation can be provided. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0011] Figure 1 A schematic diagram of an exemplary system to which the embodiments of this application are applicable; Figure 2 This is a flowchart illustrating the steps of a target object identification method according to an embodiment of this application; Figure 3 This is a flowchart illustrating the steps of another target object identification method according to an embodiment of this application; Figure 4 This is a schematic diagram illustrating a target object identification process according to an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0012] To enable those skilled in the art to better understand the technical solutions in the embodiments of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art should fall within the protection scope of the embodiments of this application.

[0013] The specific implementation of the embodiments of this application will be further described below with reference to the accompanying drawings.

[0014] Figure 1 An exemplary system applicable to embodiments of this application is shown. For example... Figure 1 As shown, the system 100 may include a cloud server 102, a communication network 104, one or more user devices 106, and an image acquisition device 108. Figure 1 The example in the text refers to multiple user devices. In scenarios with high real-time requirements, the solution of this application embodiment can be implemented on the user device 106 side based on a lightweight multimodal visual language model; in scenarios without real-time requirements or with low real-time requirements, it can be implemented on the cloud server 102 based on a non-lightweight multimodal visual language model or a lightweight multimodal visual language model.

[0015] The cloud server 102 can be any suitable device for storing information, data, programs, and / or any other suitable type of content, including but not limited to distributed storage system devices, server clusters, computing cloud server clusters, etc. In some embodiments, the cloud server 102 can perform any suitable function. For example, when the target object recognition scheme of this application embodiment is implemented on the cloud server 102, the cloud server 102 can acquire a video frame sequence to be processed from the image acquisition device 108 or the user device 106, and perform multi-target tracking on the video frame sequence to obtain at least one human trajectory sequence; according to a preset multi-level cascaded recognition strategy, target object recognition is performed based on the human trajectory sequence to identify the target object; wherein, the multi-level cascaded recognition strategy includes at least two of the following: a face recognition strategy, a pedestrian re-recognition strategy, and a multimodal visual language model recognition strategy. As another example, in some embodiments, the cloud server 102 can send the target object recognition result to the user device 106.

[0016] In some embodiments, the communication network 104 can be any suitable combination of one or more wired and / or wireless networks. For example, the communication network 104 can include any one or more of the following: the Internet, an intranet, a wide area network (WAN), a local area network (LAN), a wireless network, a digital subscriber line (DSL) network, a frame relay network, an asynchronous transfer mode (ATM) network, a virtual private network (VPN), and / or any other suitable communication network. The user equipment 106 can be connected to the communication network 104 via one or more communication links (e.g., communication link 112), and the communication network 104 can be linked to the cloud server 102 via one or more communication links (e.g., communication link 114). The communication link can be any communication link suitable for transmitting data between the user equipment 106 and the cloud server 102, such as a network link, a dial-up link, a wireless link, a hardwired link, any other suitable communication link, or any suitable combination of such links.

[0017] User equipment 106 may include any one or more user devices suitable for interacting with a user. In some embodiments, when the target object recognition scheme of this application embodiment is implemented in user equipment 106, user equipment 106 can acquire a video frame sequence to be processed from image acquisition device 108, and perform multi-target tracking on the video frame sequence to obtain at least one human trajectory sequence; according to a preset multi-level cascaded recognition strategy, target object recognition is performed based on the human trajectory sequence to identify the target object; wherein, the multi-level cascaded recognition strategy includes at least two of the following: a face recognition strategy, a pedestrian re-identification strategy, and a multimodal visual language model recognition strategy cascaded sequentially. In some embodiments, user equipment 106 may include any suitable type of device. For example, in some embodiments, user equipment 106 may include a mobile device, tablet computer, laptop computer, desktop computer, wearable computer, and / or any other suitable type of user device.

[0018] Based on the above system, this application provides a target object recognition scheme, which will be described below through several embodiments.

[0019] Reference Figure 2 This document illustrates a flowchart of a target object identification method according to an embodiment of this application. The target object identification method of this embodiment includes the following steps: Step S202: Obtain the video frame sequence to be processed, and perform multi-target tracking on the video frame sequence to obtain at least one human trajectory sequence.

[0020] Taking a retail store as an example, in the actual operation of a retail store, there are usually multiple people in the store at the same time. They can be distinguished by their roles, including customers and sales assistants. It should be noted that although the multiple embodiments of this application use a retail store as an example scenario, those skilled in the art should understand that the solutions of the embodiments of this application can be applied to a variety of scenarios with different roles. Therefore, in the embodiments of this application, "service object" is used to refer to customers or roles similar to customers, and "target object" is used to refer to sales assistants or roles similar to sales assistants.

[0021] Furthermore, taking retail stores as an example, retail stores typically deploy image acquisition devices such as high-position cameras installed on the store ceiling or top of shelves to capture a global overhead view of the store's interior, covering customer entry and exit areas and sales service areas. These high-position cameras can continuously capture video streams within the store, and multiple video frames in the stream can form a video frame sequence. However, this is not limited to this; a video frame sequence can also be a collection of video frames arranged in chronological order after frame extraction from the video stream. In practical applications, downsampling can be performed on the video stream, that is, video frames can be extracted from the original video stream at equal intervals according to a preset frame rate. This effectively reduces the amount of data for subsequent processing and improves the overall system's processing efficiency while ensuring that key information is not lost. When target object recognition is required, such as when a target object recognition instruction is received, or when conditions in the video stream that trigger target object recognition (such as detecting a sales assistant or customer entering the store) are met, the video frame sequence corresponding to the video stream from the time the instruction is received or the triggering condition is met becomes the video frame sequence to be processed.

[0022] Since customers and sales staff typically move freely within the store space during the same time period, this embodiment employs multi-target tracking to continuously and stably track them within the video frame sequence for subsequent identification and analysis. Multi-target tracking detects all human targets appearing in the video frame sequence and establishes a correspondence between human targets in adjacent frames. This assigns a unique trajectory identifier to each continuously detected human and generates a continuous position record of that human in the time dimension of the video frame sequence. In this embodiment, multi-target tracking can be implemented using any suitable algorithm, including but not limited to ByteTrack, DeepSORT, and JDE; this embodiment does not impose any limitations on this.

[0023] Multi-target tracking can obtain at least one human trajectory sequence, each corresponding to a person in the store. It records the person's position information (human bounding box coordinates), cropped human image, and associated identity status at each moment in the video frame sequence. Each human trajectory sequence is distinguished by a unique trajectory identifier. Using the human trajectory sequence as the basic unit, the system processes the personnel corresponding to each trajectory sequence separately during subsequent identity recognition and service matching, thereby achieving effective management of scenarios with multiple people coexisting in the store.

[0024] By performing multi-target tracking on video frame sequences and obtaining at least one human trajectory sequence, the continuous video stream is converted into a structured human trajectory sequence, providing an ordered data foundation for subsequent target object recognition. At the same time, it effectively avoids the sensitivity of the scheme to the quality fluctuations of a single video frame and improves the robustness of the overall scheme.

[0025] Step S204: According to the preset multi-level cascaded recognition strategy, target object recognition is performed based on the human trajectory sequence to identify the target object.

[0026] Among them, the multi-level cascaded recognition strategy includes at least two of the following: face recognition strategy, pedestrian re-recognition strategy, and multimodal visual language model recognition strategy.

[0027] Because the high-positioned cameras are installed at a downward angle, the captured facial images suffer from angular distortion. Combined with variations in lighting conditions and occlusion by people, the quality of the captured facial images is generally low, and the proportion of clear, usable facial image frames in the entire human trajectory sequence is quite limited. Under these conditions, relying solely on facial recognition will result in a large number of faces in the trajectory failing to be successfully identified.

[0028] To address this, this application employs a multi-level cascaded recognition strategy, cascading multiple recognition methods according to priority to form a fault-tolerant recognition system. This approach balances accuracy with processing efficiency, improving the success rate of target object recognition and the robustness of the system. In this application embodiment, the face recognition strategy, pedestrian re-identification strategy, and multimodal visual language model recognition strategy within the multi-level cascaded recognition strategy are cascaded together and recognized sequentially in descending order of priority.

[0029] In one alternative approach, the multi-level cascaded recognition strategy includes the aforementioned three strategies: a face recognition strategy, a pedestrian re-identification strategy, and a multimodal visual language model recognition strategy, cascaded sequentially. According to the preset multi-level cascaded recognition strategy, target object recognition based on human trajectory sequences can include: first, performing target object recognition based on the human trajectory sequence using the face recognition strategy; if face recognition fails, then performing target object recognition based on the human trajectory sequence sequentially using the pedestrian re-identification strategy and the multimodal visual language model recognition strategy. In this approach, even if the face recognition strategy fails, the multimodal visual language model recognition strategy will still be executed regardless of whether the pedestrian re-identification strategy succeeds or fails. This is because the recognition accuracy of the pedestrian re-identification strategy is slightly lower than that of the face recognition strategy. Even if the pedestrian re-identification strategy succeeds, executing the multimodal visual language model recognition strategy can effectively improve recognition accuracy and ensure the accuracy of the recognition.

[0030] However, this is not the only option. In another alternative approach, target object recognition based on human trajectory sequences, according to a preset multi-level cascaded recognition strategy, can include: first, using a face recognition strategy to identify the target object based on the human trajectory sequence; if recognition fails, then using a pedestrian re-identification strategy to identify the target object based on the human trajectory sequence; if that fails, then using a multimodal visual language model recognition strategy to identify the target object based on the human trajectory sequence. In this approach, the multimodal visual language model recognition strategy is only activated when the pedestrian re-identification strategy fails. This is because, on the one hand, in many scenarios where recognition accuracy requirements are moderate, the recognition accuracy of the pedestrian re-identification strategy is sufficient to meet the usage needs; on the other hand, it can reduce latency, improve recognition efficiency, and also reduce the system energy consumption and computational costs brought about by target object recognition.

[0031] In practical applications, those skilled in the art can choose any of the above methods according to actual needs.

[0032] The identification strategies described above will be explained below.

[0033] Level 1: Facial recognition strategy.

[0034] Facial recognition is used for identity verification based on facial biometric features. In this embodiment, it is the highest priority recognition strategy in the three-level cascaded recognition process. According to the facial recognition strategy, target object recognition based on human trajectory sequences can be achieved by performing facial recognition on facial images within the human trajectory sequence, according to the facial recognition strategy.

[0035] In one example, when executing a face recognition strategy, the face region can be detected from the cropped human body image of the video frame of the human trajectory sequence, and the face feature vector can be extracted. Then, a similarity comparison is performed with the feature vector in a pre-registered target object (e.g., a salesperson) face database. The system has a standard image library for objects, which includes the aforementioned target object face database. The target object face database stores the face feature vectors of known target objects and their corresponding identity information. If a clear and valid face is detected in a frame, and its feature vector has a similarity exceeding a preset similarity threshold with the feature vector of a target object in the target object face database, and the coverage rate of valid face frames in the trajectory meets the set requirements, then the face recognition is considered successful, and the target object corresponding to the human trajectory sequence is directly confirmed as the corresponding target object, completing the identity marking. In an optional embodiment, the extraction of face feature vectors can be based on face recognition models such as ArcFace. This application embodiment does not limit the specific type of face recognition model; any model or algorithm capable of face feature extraction and similarity matching is acceptable.

[0036] Level 2: Pedestrian re-identification strategy.

[0037] If the first-level face recognition strategy fails to match, the second-level pedestrian re-identification strategy is initiated. According to the pedestrian re-identification strategy, target object recognition based on human trajectory sequences can be achieved by: performing appearance recognition on the human body's appearance within the human trajectory sequence according to the pedestrian re-identification strategy.

[0038] Pedestrian re-identification strategies can achieve cross-frame person identification based on appearance features (such as clothing color, style, body shape, etc.) without relying on facial information. They can perform effective identification even when facial images are unclear or unavailable.

[0039] In one alternative approach, the object standard image library further includes a target object appearance image library, which stores target object appearance images and their corresponding feature vectors. For example, this could be a target object uniform image library, storing target object uniform images, their corresponding feature vectors, and corresponding identification information. When implementing a pedestrian re-identification strategy, overall human appearance features can be extracted from video frames in a human trajectory sequence and compared with the appearance features in the target object uniform image library to confirm the person's identity.

[0040] In one alternative approach, according to a pedestrian re-identification strategy, the appearance recognition of human bodies in a human trajectory sequence may include: extracting human appearance images from the human trajectory sequence according to the pedestrian re-identification strategy; extracting appearance features from the human appearance images to obtain target appearance features; comparing the target appearance features with the appearance features corresponding to images in a standard image library based on dual threshold verification; and determining whether the target object recognition is successful based on the comparison results.

[0041] When extracting human appearance images from a human trajectory sequence, multiple cropped human images from the trajectory can be acquired to obtain richer appearance information, covering different poses and angles, thereby improving the representativeness of the extracted features. After obtaining the target appearance features, they are compared with the appearance features corresponding to the work clothes images of each target object stored in the target object appearance image library in the object standard image library (e.g., using cosine similarity) to obtain the similarity ranking results. In this embodiment, to further reduce the false matching rate, a dual threshold verification mechanism is adopted to verify the comparison results. The first threshold is the Top-1 similarity threshold, which requires that the similarity between the appearance features of the target object and the appearance features corresponding to the work clothes image with the highest similarity exceeds this threshold; the second threshold is the discrimination threshold between the Top-1 and Top-2 similarities, which requires that the similarity difference between the appearance features corresponding to the aforementioned work clothes image with the highest similarity and the appearance features corresponding to the work clothes image with the second highest similarity exceeds this threshold, to ensure that the recognition results have sufficient discriminability, exclude the fuzzy recognition situation caused by multiple target objects having close similarity, thereby reducing the false matching rate and improving the reliability of recognition. When both thresholds are met, pedestrian re-identification is deemed successful, and the identity of the target object in the current human trajectory sequence is identified by the identity tag corresponding to the work clothes image with the highest similarity. It should be noted that those skilled in the art can flexibly configure the aforementioned first and second thresholds according to actual needs; the specific threshold values ​​are not limited in this embodiment.

[0042] In another alternative approach, the aforementioned extraction of appearance features from a human body image to obtain target appearance features may include: extracting appearance features from the human body image to obtain candidate appearance features; performing clustering and denoising processing on the candidate appearance features; and performing averaging and normalization processing on the clustered and denoised appearance features to obtain the target appearance features. This method effectively filters features corresponding to noisy frames caused by occlusion, blurring, etc., retaining features from high-quality video frames. This allows for a more accurate representation of the true appearance distribution of the target object, improving the accuracy of subsequent comparisons and reducing the risk of misidentification. Through averaging and normalization processing, features from multiple frames can be fused into a more representative target appearance feature, effectively improving the quality and stability of the appearance features.

[0043] In this approach, candidate appearance features are sets of raw feature vectors obtained by extracting features from each frame of human appearance images using a pedestrian re-identification strategy, without undergoing quality screening. In multi-frame cropped human images, there may be low-quality video frames, such as frames with partial occlusion due to other people passing by, blurred frames due to motion blur, or distorted frames due to extreme shooting angles. The feature vectors corresponding to these low-quality video frames deviate significantly from those of normal video frames and can be considered noise. In this case, directly averaging all candidate appearance features would cause the final target appearance feature vector to deviate from the actual appearance distribution of the person, increasing the probability of mismatches.

[0044] Therefore, this method uses clustering denoising to screen candidate appearance features for quality control. Clustering denoising divides candidate appearance features into several groups using a clustering algorithm. The group with the most numerous and concentrated features corresponds to the features of normal quality frames, while the smaller and more dispersed groups correspond to the features of noisy frames. By retaining the main cluster (the group with the most concentrated features) and discarding the remaining noisy clusters, noisy frames can be filtered. In an optional approach, clustering denoising can be implemented using the DBSCAN density clustering algorithm. The DBSCAN algorithm does not require pre-specifying the number of clusters and can automatically identify and eliminate noise points, making it effective for denoising feature sets with unknown distributions. However, this is not a limitation; other clustering denoising algorithms are also applicable to the schemes of this application.

[0045] After clustering and denoising, the appearance features in the retained main clusters can be averaged to obtain an average feature that integrates information from multiple frames. This average feature is then transformed to a unit hypersphere using normalization techniques such as L2 normalization to obtain the target appearance feature. By averaging and integrating features from multiple frames, the target appearance feature has a greater coverage advantage than single-frame features, effectively reducing the risk of misidentification due to occasional anomalies in single-frame images. Normalization ensures that subsequent similarity comparisons are performed in a unified metric space, improving the reliability and comparability of the comparison results.

[0046] Level 3: Multimodal visual language model recognition strategy.

[0047] The multimodal visual language model recognition strategy, as a third-level strategy, can be considered a fallback strategy for target object recognition. According to this strategy, target object recognition based on human trajectory sequences can be achieved by: extracting keyframes from the human trajectory sequence and then using a multimodal visual language model to identify the target object based on these keyframes.

[0048] Multimodal visual language models possess both image understanding and natural language reasoning capabilities. They can perform semantic understanding of input image content and output reasoning conclusions in the form of natural language. They do not rely on precise biometric features (such as faces) or precise appearance feature matching; instead, they determine the role category of people in an image by understanding and inferring their overall semantic attributes.

[0049] In one alternative approach, target object recognition based on keyframes using a multimodal visual language model can include: recognizing target objects based on keyframes, appearance example images, and appearance example descriptions using the multimodal visual language model. By using keyframes, appearance example images, and appearance example descriptions together as input to the multimodal visual language model, multi-dimensional reference information is provided to the model. This allows it to perform comparative reasoning based on appearance examples, building upon image content understanding. Consequently, it makes more evidence-based judgments about the role or appearance attributes of the person being identified, thus improving the recognition accuracy of the multimodal visual language model.

[0050] Keyframes are representative video frames extracted from human trajectory sequences, used to provide image information of the person to be identified to the multimodal visual language model. Appearance example images are pre-prepared reference images describing the typical appearance of the target object (e.g., a salesperson), such as images of the salesperson's dress code. Appearance example descriptions are textual descriptions of the typical appearance features of the target object, such as textual descriptions of the color, style, and logo of the salesperson's uniform. By providing keyframes, appearance example images, and appearance example descriptions together to the multimodal visual language model, the model can combine the example information to compare and infer the appearance of the person to be identified, thereby obtaining a more accurate inference result, i.e., the target object recognition result.

[0051] Optionally, target object recognition based on keyframes, appearance sample images, and appearance sample descriptions using a multimodal visual language model can include: performing role recognition using a multimodal visual language model based on keyframes, appearance sample images, and appearance sample descriptions; and determining the target object based on the role recognition results. Role recognition refers to the multimodal visual language model determining the role of the person to be identified in the scene, such as determining whether they are a salesperson, a customer, or another person. Based on the conclusion of role recognition, it can be determined whether the person corresponding to the human trajectory sequence belongs to the target object that needs to be identified, as well as their corresponding category and identifier. By performing role recognition using a multimodal visual language model, the role of the person to be identified can be determined semantically, and the target object can be determined based on the role recognition results. Role recognition transforms target object identification into semantic classification, eliminating the need for precise feature matching, broadening the applicability of the third-level identification strategy, and further improving the ability to determine identity in scenarios where the first two levels of identification strategies fail.

[0052] Through the aforementioned three-tiered cascaded recognition system, the recognition coverage is progressively expanded while maintaining recognition accuracy. Specifically, the face recognition strategy covers scenarios where valid face images can be obtained; the pedestrian re-identification strategy covers scenarios where faces are unavailable but clothing features are identifiable; and the multimodal visual language model recognition strategy further covers scenarios where the first two levels fail. The cascading and synergistic cooperation of these three recognition strategies improves the overall robustness and coverage of target object recognition in high-position camera scenarios.

[0053] Through the above process, the target object of each human trajectory sequence in the video frame sequence was identified, and the personnel and their identities belonging to the registered target objects were identified, providing an accurate basis and effective data foundation for subsequent processing.

[0054] According to this embodiment, by performing multi-target tracking on video frame sequences and using human trajectory sequences as processing units, a spatiotemporally continuous data foundation can be provided for subsequent target object recognition, avoiding the adverse effects of image quality fluctuations in a single video frame on the stability of the recognition results. Based on this, following a multi-level cascaded recognition strategy, at least two of the following strategies—face recognition, pedestrian re-identification, and multimodal visual language model recognition—are cascaded sequentially in descending order of priority. Each level progressively degrades and complements the others. When the face recognition strategy proves ineffective, the pedestrian re-identification strategy and / or the multimodal visual language model recognition strategy provide effective degradation protection, significantly improving the overall robustness and coverage of target object recognition (such as sales assistants in retail stores). Furthermore, if subsequent evaluation of sales assistant services is required, accurate evidence and data foundation can be provided.

[0055] Based on the above target object identification, optionally, further service matching can be performed. Based on this, refer to... Figure 3 The flowchart illustrates the steps of another target object identification method according to an embodiment of this application.

[0056] The target object identification method in this embodiment includes the following steps: Step S302: Obtain the video frame sequence to be processed, and perform multi-target tracking on the video frame sequence to obtain at least one human trajectory sequence.

[0057] For detailed implementation of this step, please refer to... Figure 2 The descriptions of the corresponding parts in the illustrated embodiments will not be repeated here.

[0058] Step S304: Based on the human trajectory sequence, identify the service recipients of human bodies that cross the preset judgment line or enter the preset judgment area.

[0059] To further capture and record specific service behaviors, one alternative approach is to use a service triggering mechanism to automatically detect the entry events of service recipients (i.e., customers).

[0060] Specifically, in this embodiment, the video frame sequence can be continuously analyzed in real time using streaming processing to detect the relationship between the spatial location of the human trajectory sequence and the decision line or decision area.

[0061] In one alternative approach, the decision line can be a virtual line segment pre-marked in the coordinate system of the video frame, typically placed at the store entrance to define the spatial boundary between the inside and outside of the store. The decision area can be a virtual area pre-marked in the coordinate system of the video frame, similar to the decision line, also used to define the event boundary when a service object enters a specific spatial range. Both the decision line and the decision area are specific implementations of spatial trigger boundaries. Those skilled in the art can choose either form based on the actual store layout and camera installation location, or design other equivalent forms as needed. This application embodiment does not impose any limitations on this.

[0062] Based on this, if spatial position analysis of human trajectory sequences detects that a human trajectory sequence moves from the outside area to the inside area of ​​the store, and its position trajectory crosses the judgment line, then the person corresponding to this human trajectory sequence can be determined as a newly entered service recipient (usually someone entering the store seeking service, typically a customer in a retail store scenario). Their entry time and corresponding trajectory identifier are recorded, forming an entry event. Alternatively, if spatial position analysis of human trajectory sequences detects that a human trajectory sequence enters a preset judgment area, and its position trajectory falls within the judgment area, then the person corresponding to this human trajectory sequence can be determined as a newly entered service recipient. Their entry time and corresponding trajectory identifier are recorded, forming an entry event.

[0063] In one alternative approach, the determination of service recipients can be combined with the results of target object identification. For example, if a human trajectory sequence is not identified as a registered target object (such as not being identified as a registered salesperson) in the target object identification process, then the person corresponding to the human trajectory sequence can be identified as a service recipient; conversely, if they have been identified as a target object, they will not be treated as a service recipient. This application does not limit the specific method for determining service recipients. That is, in... Figure 2 In step S204 of the illustrated embodiment, when identifying target objects based on human trajectory sequences according to a preset multi-level cascaded identification strategy, not only are target objects identified, but also non-target objects, such as service objects in this embodiment, are identified, thereby saving implementation costs and improving implementation efficiency.

[0064] Step S306: If a human body that crosses a preset judgment line or enters a preset judgment area is a service object, then service matching is performed on the target object based on the service object and the identified target object.

[0065] Once a service object's entry event is detected, the service matching process for the target object can be triggered to determine the target object to provide services to that service object.

[0066] Optionally, the above-mentioned service matching for target objects may include: if multiple target objects are identified, then selecting the target object that provides services to the service object from among the multiple target objects based on the spatiotemporal co-occurrence relationship between the service object and the multiple target objects. Through spatiotemporal co-occurrence relationships, in scenarios where multiple target objects coexist, the target object that actually provides services to the service object can be filtered out based on objective data, achieving automatic and accurate matching between service objects and target objects, and avoiding the inaccuracies caused by random allocation or manual designation.

[0067] Spatiotemporal co-occurrence relationships indicate the simultaneous appearance and spatial proximity of service objects and target objects in both the temporal and spatial dimensions of a video frame sequence. In real-world service scenarios, target objects providing reception services to service objects are often present simultaneously with the service objects in time and maintain a relatively close spatial distance, exhibiting behaviors such as proactively approaching and providing accompaniment and guidance. Other target objects not involved in the reception tend to maintain a relatively greater distance from the service objects or operate in different areas. Therefore, by quantitatively analyzing the degree of spatiotemporal co-occurrence between service objects and various target objects, we can objectively identify the target objects most likely to undertake reception services.

[0068] Optionally, selecting the target object to provide services to the service object from among the multiple target objects based on the spatiotemporal co-occurrence relationship between the service object and multiple target objects may include: selecting the target object to provide services to the service object from among the multiple target objects based on the spatiotemporal co-occurrence parameters between the service object and multiple target objects. The spatiotemporal co-occurrence parameters include at least one of the following: the number of co-occurring video frames, the maximum spatial distance, and the average spatial distance.

[0069] Spatiotemporal co-occurrence parameters are a quantitative description of the degree of co-occurrence between service objects and target objects in both time and space dimensions, and are a numerical representation of spatiotemporal co-occurrence relationships.

[0070] in: The number of co-occurring video frames indicates the number of frames in which the human trajectory sequence of a service recipient and the human trajectory sequence of a target object appear simultaneously within a time window after an in-store event. A higher number of co-occurring video frames indicates a greater degree of temporal association between the target object and the service recipient, and a greater likelihood of providing service.

[0071] Maximum spatial distance indicates the maximum distance between the center points of the human bounding boxes of a service object and a target object across all video frames in which they co-occur. Maximum spatial distance reflects the upper limit of spatial proximity between the target object and the service object throughout the entire co-occurrence period. A smaller maximum spatial distance indicates that the target object consistently remains within a close spatial range of the service object, making it more likely to provide service.

[0072] Average spatial distance indicates the average distance between the center points of the human bounding boxes of the service object and the target object across all video frames where they co-occur. Average spatial distance reflects the average spatial proximity of the target object and the service object throughout the entire co-occurrence period, providing a comprehensive assessment of the service behavior in the spatial dimension.

[0073] In one alternative approach, the spatiotemporal co-occurrence parameters may simultaneously include the number of co-occurring video frames, the maximum spatial distance, and the average spatial distance. This allows for a comprehensive evaluation of each target object, such as a comprehensive score, and the selection of the target object with the highest score as the target object to which reception services are provided. In practical applications, those skilled in the art can flexibly set the weight of each spatiotemporal co-occurrence parameter in the comprehensive score, as well as the specific calculation method of each parameter, according to actual needs; the embodiments of this application do not impose such limitations.

[0074] By using the aforementioned spatiotemporal co-occurrence parameters, target objects can be evaluated from multiple dimensions, thereby more comprehensively and accurately assessing the reception potential of each target object, effectively addressing complex scenarios where multiple target objects are present in the store at the same time, and improving the accuracy of matching service objects with target objects.

[0075] For commonly used high-position cameras, due to perspective effects during overhead shots, people closer to the camera (i.e., in the center or near the frame) appear larger with wider bounding boxes, while people farther away (at the edge or farthest point) appear smaller with narrower bounding boxes. In this situation, using a fixed pixel distance threshold to determine spatial proximity between two people is inaccurate because the same physical distance corresponds to different pixel distances in different areas of the video frame. This fixed threshold cannot accurately reflect the actual degree of spatial proximity, leading to misjudgments.

[0076] Therefore, embodiments of this application employ a dynamic adaptive distance threshold determination strategy based on the width of the human body bounding box. Based on this, in one optional approach, if the spatiotemporal co-occurrence parameters include maximum spatial distance and / or average spatial distance, then selecting the target object to provide services to the service object from among the multiple target objects based on the spatiotemporal co-occurrence parameters of the service object and multiple target objects may include: dynamically determining a first distance threshold and / or a second distance threshold based on the width of the human body bounding box used during real-time detection; if the maximum spatial distance between the service object and the multiple target objects is less than the first distance threshold, and / or the average spatial distance is less than the second distance threshold, then the target object whose distance is less than the first distance threshold and / or the second distance threshold is determined as the target object to provide services to the service object.

[0077] The width of the human body bounding box describes the size of the human body image in a video frame. It reflects the actual proportional size of the detected human body in the image and is directly related to the imaging distance of the person in the image and the degree of perspective scaling. Therefore, dynamically determining the distance threshold based on the width of the human body bounding box allows the scale of the threshold to adaptively match the imaging size of the detected person, thereby transforming a fixed pixel distance threshold in the image coordinate system into an adaptive distance threshold that adapts to the actual physical space. The human body bounding box can be obtained based on a multi-target tracking process.

[0078] For example, the first distance threshold can indicate a threshold for determining whether the maximum spatial distance between the service object and the target object meets the spatial proximity condition; the second distance threshold can indicate a threshold for determining whether the average spatial distance between the service object and the target object meets the spatial proximity condition. Both the first and second distance thresholds are dynamically determined based on the width of the human body bounding box, allowing the thresholds to automatically adjust with changes in the imaging scene, accurately reflecting the degree of proximity in actual physical space at different shooting distances and regional locations. In practical applications, those skilled in the art can flexibly set the conversion relationship between the first and second distance thresholds and the width of the human body bounding box according to the actual scene and installation parameters; this application embodiment does not impose any limitations on this.

[0079] By dynamically determining the distance threshold based on the width of the human body bounding box, the interference of perspective distortion from high-position cameras on spatial distance judgment is effectively solved. This ensures that the spatial proximity judgment of the target object has a consistent physical meaning among people in different locations, thereby improving the accuracy and generalization ability of spatiotemporal co-occurrence service matching.

[0080] Furthermore, in some service scenarios, such as retail stores, there may be no target object (e.g., a sales associate) or only one target object (e.g., a sales associate). In such cases, to avoid invalid processing and improve the overall efficiency of the system, one optional approach to matching the target object for services may include: if the standard object image library is empty, then it is determined that there is no target object to provide services; if the standard object image library includes a single object image, then the target object corresponding to the image is determined as the target object to provide services to the service object.

[0081] As mentioned earlier, the object standard image library is a pre-established database that stores information on registered target objects (such as sales staff). Examples include the aforementioned target object face database and / or target object appearance image database. If the object standard image library is empty, it means that no target object has completed registration. Therefore, the system cannot identify any person in the store as a known target object. Even if a service recipient is detected entering the store, effective service matching cannot be performed. The system determines this entry as an event of "no target object providing service" and records the corresponding event information. If the object standard image library contains information on only a single target object, meaning only one target object has completed registration, then when a service recipient enters the store, the only target object identified in the store can be that single target object. The system can directly identify this single target object as the target object providing service to the service recipient without initiating subsequent identification processes, thus simplifying the processing logic.

[0082] This ensures that even under boundary conditions where the target object's registration information is incomplete, a reasonable processing result can still be given, improving the robustness and reliability of the solution and avoiding system anomalies caused by the lack of boundary case handling.

[0083] Furthermore, optionally, after completing the target object identification and service matching, structured service event results can be output to support the consumption and application of possible downstream tasks.

[0084] The structured service event results can include event type information, a list of identity markers of the target objects providing the service, and a confidence score. The event type indicates the service status of the current in-store event, and can include, for example, three types: "In-store reception" (i.e., a service object was detected entering the store, and a target object providing service was successfully matched), "In-store reception" (i.e., a service object was detected entering the store, but no target object was matched), and "No event" (i.e., no in-store service object event was detected). The confidence score is a quantitative assessment of the reliability of the service matching results, which can be used as a credibility reference for downstream tasks when consuming results.

[0085] Optionally, a target display frame can be selected from the video frames during the service process as the image basis for downstream tasks. The selection of the target display frame can comprehensively consider multiple image quality factors, such as facial clarity and human body integrity, to ensure that the selected frame can clearly display the appearance information of the target object providing the service, meeting the image quality requirements of downstream tasks. In practical applications, those skilled in the art can flexibly set specific evaluation indicators and weights for the selection of the target display frame according to actual needs; this application embodiment does not impose any limitations on this.

[0086] This embodiment, based on effective identification of the target object, achieves automatic detection of the service object's entry into the store by real-time detection of the video stream, using the service object crossing the judgment line or entering the judgment area as the trigger condition. This triggers the service matching process for the target object. Therefore, service response can be initiated immediately upon the service object's entry into the store, reducing the perception delay of service events and ensuring timely fulfillment of the service needs of the service object.

[0087] The following is for reference Figure 4 The processing flow of the above target object recognition method is illustrated by a specific example.

[0088] Figure 4 In the example shown, a retail store is set up with high-position cameras deployed, and the registration and database construction of sales staff (including a facial database of sales staff and a database of images of sales staff uniforms) are completed in advance.

[0089] The target object identification process in this example includes: Step S402: Begin.

[0090] The process begins, preparing to perform target object identification. In this example, it specifically involves the identification of sales staff and the subsequent service matching process.

[0091] Step S404: Initialize Pipeline by loading the model and object standard library.

[0092] The system preloads the algorithm models required for the multi-level cascaded recognition strategy (including face recognition, pedestrian re-identification, multimodal visual language model, etc.) into memory, and establishes a standard image library of objects (such as the target object face library, which is the sales guide face library in this example, and the target object appearance image library, which is the sales guide appearance uniform library in this example).

[0093] Step S406: Multi-object tracking using CV (computer vision) algorithm with ByteTracker algorithm.

[0094] For example, the video frame sequence to be processed in the video stream can be obtained, and the ByteTracker algorithm can be used to perform multi-target tracking on the video frame sequence to obtain at least one spatiotemporally continuous human trajectory sequence, which can be used as the basic analysis unit for subsequent identity recognition and service matching.

[0095] Step S408: Store Entry Event Detection This example uses the inshop_line crossing detection method. Real-time detection is performed on the video frame sequence. If a customer's (service recipient's) human trajectory is detected to cross the preset inshop_line, an entry event is recorded as a condition to trigger target object service matching.

[0096] Step S410: Face detection and recognition.

[0097] In this example, the ArcFace model is used for face detection and recognition. As the highest priority first-level cascaded strategy, face images are extracted from human trajectory sequences, and the ArcFace model is used to extract face features and compare them with images in a standard image library (such as a shopping guide face library).

[0098] Step S412: Determine whether the face recognition is successful; if successful, proceed to step S414; if unsuccessful, proceed to step S416.

[0099] The identity of the current person is determined based on the similarity result of the comparison. If the match is successful, proceed to step A414; if the match fails, a downgraded identification strategy is triggered, and proceed to step S416.

[0100] Step S414: Mark as a sales guide and record the identity identifier face_match_id. Skip subsequent identity verification and proceed to step S436.

[0101] If the facial recognition is successful, the corresponding person is directly identified as the target (salesperson) and their identity is recorded. Since the first-level strategy has been effective, subsequent identification is skipped, and the process proceeds to step S436.

[0102] Step S416: Determine whether ReID is enabled; if enabled, proceed to step S418; if not enabled, proceed to step S426.

[0103] After face recognition fails, the system checks whether the second-level ReID (Person Re-identification) function is enabled. If yes, proceed to step S418; otherwise, downgrade directly to the third-level recognition strategy and proceed to step S426.

[0104] Step S418: ReID feature extraction.

[0105] In this example, ReID feature extraction is performed using the OSNetAIN model. Following the ReID strategy, human appearance images are extracted from human trajectory sequences, and candidate appearance features are extracted using the OSNetAIN model.

[0106] Step S420: Feature library matching.

[0107] In this example, the DBSCAN denoising + Top-N search method is used. The candidate appearance features are first clustered and denoised using the DBSCAN algorithm to filter low-quality video frames. Then, the processed target appearance features are compared with the appearance features in the salesperson uniform library in the object standard image library, which is a comparison based on dual threshold verification.

[0108] Step S422: Determine whether the ReID match is successful; if the match is successful, proceed to step S424; if the match fails, proceed to step S426.

[0109] Based on the results of the dual threshold verification, determine whether the appearance recognition was successful. If the match is successful, proceed to step S424; if it still fails, proceed to step S426.

[0110] Step S424: The auxiliary marker is the shopping guide, the identity identifier osnet_match_id is recorded, and then proceed to step S426.

[0111] If pedestrian re-identification is successful, the person in the current trajectory is marked as the corresponding target object, i.e., the sales guide, and the matching identifier is recorded. To distinguish it from the identifier marked by face recognition, the identifier matched by ReID is recorded as osnet_match_id. Then, proceed to step S426.

[0112] In this example, although the sales associate is identified by appearance, to make the identification more accurate, VLM is used again for identity analysis and confirmation in step S426. However, those skilled in the art should understand that in practical applications, after this step, the process can directly proceed to step S432.

[0113] Step S426: VLM identity analysis.

[0114] In this example, VLM (Virtual Language Model) is used as a fallback strategy to extract keyframes from the human trajectory sequence and combine them with appearance sample images and sample descriptions, which are then input into VLM for semantic-level role reasoning.

[0115] Step S428: Store entry priority strategy.

[0116] That is, first analyze the people entering the store. During the VLM analysis, combine the store entry behavior of the tracked people to make comprehensive strategy assistance judgments. On the one hand, it can avoid misclassifying service recipients as target recipients; on the other hand, it can realize role identification (in this example, roles include: sales assistant, customer, and other personnel), so as to prepare for different treatments for different roles in the future.

[0117] Step S430: VLM determines whether it is a sales guide; if it is determined to be a sales guide, proceed to step S432; if it is determined to be not a sales guide, proceed to step S434.

[0118] Based on the role recognition results output by VLM, determine the role category to which the current person belongs.

[0119] Step S432: Mark as a shopping guide.

[0120] If VLM determines that it meets the characteristics of a sales guide, it marks its identity as a sales guide and proceeds to step S436.

[0121] Step S434: Mark as customer.

[0122] If VLM determines that the user does not belong to the sales guide role, it will identify the user as an unregistered service recipient, which is a customer in this example, and proceed to step S436.

[0123] Step S436: Reception behavior decision.

[0124] After the personnel identification is completed, the system initiates the reception service matching process for the target, namely the sales guide.

[0125] Step S438: Determine if it is a multi-sales scenario. If yes, proceed to step S440; otherwise, proceed to step S442.

[0126] Check if there are multiple sales associates currently identified. If there are multiple sales associates, proceed to step S440; if there is only a single sales associate or no sales associate, proceed to step S442.

[0127] Step S440: Spatial Relationship Filtering.

[0128] Based on the spatiotemporal co-occurrence relationship between the customer and multiple sales associates, the sales associate who actually provides the service is selected from among the multiple sales associates, and then the process proceeds to step S444.

[0129] Step S442: Output directly, proceed to step S444.

[0130] If it is determined that there is only one sales guide or no sales guide, then directly identify the single sales guide as the service provider, or determine that there is no sales guide to provide services, simplify the logic and proceed to step S444.

[0131] Step S444: Select the target display frame.

[0132] From the human trajectory sequence corresponding to the service process, the target display frame is extracted by comprehensively evaluating indicators such as facial clarity and human body integrity to support the consumption of downstream tasks.

[0133] Step S446: Output the event report EventResult.

[0134] The final service matching result is encapsulated into a structured service event result for output, which includes the event type event_type (e.g., there is a customer service representative entering the store) and the staff_uid_list of the sales guides who provided the service.

[0135] Step S448: End the process.

[0136] This example demonstrates how a recognition system is constructed based on human trajectory sequences obtained from multi-target tracking, using a three-level cascaded recognition strategy. Spatiotemporal co-occurrence analysis is used as the basis for service matching, achieving end-to-end automated complete link processing from video stream input to service event output. This effectively improves the overall robustness and coverage of target object recognition.

[0137] Furthermore, in the aforementioned embodiments, various parallel methods can be employed as needed to improve recognition efficiency, reduce processing latency, and meet low-latency requirements when performing target object recognition. For example, when tracking targets in a video frame sequence, a frame-sampling strategy can be used to reduce the number of video frames and improve tracking efficiency. As another example, in service object recognition, the video frames corresponding to the human trajectory sequence can be downsampled before determining whether a human body crosses a preset judgment line or enters a preset judgment area for service object recognition. Furthermore, multi-threading can be enabled to improve efficiency in multi-target tracking, target object recognition, and service object recognition. For yet another example, multi-threading can be enabled for parallel inference when using a VLM recognition strategy to improve inference speed and efficiency. Finally, when using a VLM recognition strategy, parallel processing of the VLM model can be achieved through concurrent VLM requests. Furthermore, in some embodiments, the process of the target object recognition method of the aforementioned multiple embodiments can be implemented through a Pipeline. In this case, the Pipeline instance pool can be reused to avoid cold start overhead and realize end-to-end real-time processing from video input to reception event output, meeting the requirements of low-latency tasks, such as the instant pop-up of shopping guide badges.

[0138] Reference Figure 5 This document illustrates a schematic diagram of an electronic device according to an embodiment of this application. The specific embodiments of this application do not limit the specific implementation of the electronic device.

[0139] like Figure 5 As shown, the electronic device may include: a processor 502, a communications interface 504, a memory 506, and a communications bus 508.

[0140] in: The processor 502, communication interface 504, and memory 506 communicate with each other via communication bus 508.

[0141] Communication interface 504 is used to communicate with other electronic devices or servers.

[0142] The processor 502 is used to execute program 510, which can specifically execute the relevant steps in the above-described embodiment of the target object recognition method.

[0143] Specifically, program 510 may include program code that includes computer operation instructions.

[0144] The processor 502 may be a CPU, a GPU (Graphics Processing Unit), an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The electronic device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or they may be processors of different types, such as one or more CPUs and one or more ASICs.

[0145] Memory 506 is used to store program 510. Memory 506 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0146] Program 510 may include multiple computer instructions. Specifically, program 510 may use multiple computer instructions to cause processor 502 to perform the operation corresponding to the target object recognition method described in any of the foregoing multiple method embodiments.

[0147] The specific implementation of each step in program 510 can be found in the corresponding steps and units described in the above method embodiments, and has corresponding beneficial effects, which will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.

[0148] This application also provides a computer storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in any of the foregoing method embodiments. The computer storage medium includes, but is not limited to, compact disc read-only memory (CD-ROM), random access memory (RAM), floppy disk, hard disk, or magneto-optical disk.

[0149] This application also provides a computer program product, including computer instructions that instruct a computing device to perform an operation corresponding to any of the target object identification methods in the above-described multiple method embodiments.

[0150] Furthermore, it should be noted that the user-related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to sample data used for training the model, data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0151] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of this application can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this application.

[0152] The methods described in the embodiments of this application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code downloaded over a network that is originally stored in a remote recording medium or a non-transitory machine-readable medium and will be stored in a local recording medium. Thus, the methods described herein can be stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an Application Specific Integrated Circuit (ASIC) or a Field Programmable Gate Array (FPGA)). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., Random Access Memory (RAM), Read-Only Memory (ROM), Flash Memory, etc.) capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the methods shown herein.

[0153] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0154] The above embodiments are only used to illustrate the embodiments of this application, and are not intended to limit the embodiments of this application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this application, and the patent protection scope of the embodiments of this application should be defined by the claims.

Claims

1. A method for identifying a target object, comprising: Acquire a video frame sequence and perform multi-target tracking on the video frame sequence to obtain at least one human trajectory sequence; According to the preset multi-level cascaded recognition strategy, the target object is identified based on the human trajectory sequence. The multi-level cascaded recognition strategy includes at least two of the following: a face recognition strategy, a pedestrian re-recognition strategy, and a multimodal visual language model recognition strategy, which are cascaded sequentially.

2. The method according to claim 1, wherein, The multi-level cascaded recognition strategy includes a face recognition strategy, a pedestrian re-identification strategy, and a multimodal visual language model recognition strategy cascaded sequentially; the step of identifying the target object based on the human trajectory sequence according to the preset multi-level cascaded recognition strategy includes: First, target object recognition is performed based on the human body trajectory sequence according to the face recognition strategy; if face recognition fails, target object recognition is performed based on the human body trajectory sequence according to the pedestrian re-identification strategy and the multimodal visual language model recognition strategy in sequence. or, First, target object recognition is performed based on the human body trajectory sequence according to the face recognition strategy; if recognition fails, target object recognition is performed based on the human body trajectory sequence according to the pedestrian re-recognition strategy; if recognition fails, target object recognition is performed based on the human body trajectory sequence according to the multimodal visual language model recognition strategy.

3. The method according to claim 2, wherein, According to the face recognition strategy, target object identification is performed based on the human body trajectory sequence, including: performing face recognition on the face images in the human body trajectory sequence according to the face recognition strategy; According to the pedestrian re-identification strategy, target object identification is performed based on the human body trajectory sequence, including: appearance recognition of the human body appearance in the human body trajectory sequence according to the pedestrian re-identification strategy; According to the multimodal visual language model recognition strategy, target object recognition is performed based on the human trajectory sequence, including: extracting keyframes from the human trajectory sequence, and performing target object recognition based on the keyframes using a multimodal visual language model.

4. The method according to claim 3, wherein, The step of performing appearance recognition on the human body appearance in the human body trajectory sequence according to the pedestrian re-identification strategy includes: According to the pedestrian re-identification strategy, extract human appearance images from the human trajectory sequence; The appearance features of the human body image are extracted to obtain the target appearance features; The target appearance features are compared with the appearance features corresponding to the images in the object standard image library based on dual threshold verification. Based on the comparison results, determine whether the target object identification was successful.

5. The method according to claim 4, wherein, The step of extracting appearance features from the human body appearance image to obtain target appearance features includes: The appearance features of the human body image are extracted to obtain candidate appearance features; Clustering and denoising processing is performed on the candidate appearance features; The appearance features after clustering and denoising are averaged and normalized to obtain the target appearance features.

6. The method according to claim 3, wherein, The step of identifying target objects based on the keyframes using a multimodal visual language model includes: Based on the keyframes, appearance sample images, and appearance sample descriptions, target object recognition is performed using a multimodal visual language model based on the keyframes.

7. The method according to claim 6, wherein, The step of identifying target objects based on the keyframes, appearance sample images, and appearance sample descriptions, using a multimodal visual language model based on the keyframes, includes: Based on the keyframes, appearance sample images, and appearance sample descriptions, character recognition is performed using a multimodal visual language model; Based on the results of role recognition, the target object is determined.

8. The method according to any one of claims 1-7, wherein, After acquiring the video frame sequence and performing multi-target tracking on the video frame sequence to obtain at least one human trajectory sequence, the method further includes: Based on the human body trajectory sequence, service recipients are identified for human bodies that cross a preset judgment line or enter a preset judgment area.

9. The method according to claim 8, wherein, The method further includes: If a human body that crosses a preset judgment line or enters a preset judgment area is a service target, then service matching for the target target is performed based on the service target and the identified target object.

10. The method according to claim 9, wherein, The process of matching services for the target object includes: If multiple target objects are identified, then a target object for providing services to the service object is selected from among the multiple target objects based on the spatiotemporal co-occurrence relationship between the service object and the multiple target objects.

11. The method according to claim 10, wherein, The step of selecting a target object to provide services to the service object from among the multiple target objects based on the spatiotemporal co-occurrence relationship between the service object and multiple target objects includes: Based on the spatiotemporal co-occurrence parameters of the service object and multiple target objects, a target object is selected from the multiple target objects to provide services to the service object; the spatiotemporal co-occurrence parameters include at least one of the following: the number of co-occurring video frames, the maximum spatial distance, and the average spatial distance.

12. The method according to claim 11, wherein, If the spatiotemporal co-occurrence parameters include maximum spatial distance and / or average spatial distance, then the step of selecting a target object to provide services to the service object from among the multiple target objects based on the spatiotemporal co-occurrence parameters of the service object and multiple target objects includes: The first distance threshold and / or the second distance threshold are dynamically determined based on the width of the human body bounding box used during the real-time detection. If the maximum spatial distance between the service object and the plurality of target objects is less than the first distance threshold, and / or the average spatial distance is less than the second distance threshold, then the target objects that are less than the first distance threshold and / or the second distance threshold are determined as the target objects to provide services to the service object.

13. The method according to claim 9, wherein, The process of matching services for the target object also includes: If the standard object library is empty, then it is determined that there is no target object to provide services. If the object standard image library includes a single object image, then the target object corresponding to the image is determined as the target object to provide services to the service object.

14. An electronic device comprising: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform the operation corresponding to the method as described in any one of claims 1-13.

15. A computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of claims 1-13.

16. A computer program product comprising computer instructions that instruct a computing device to perform an operation corresponding to any one of the methods described in claims 1-13.

Images