An edge collaborative reasoning interaction method for pedestrian re-identification scenarios

Abstract

This invention provides an edge-collaborative inference interaction method for pedestrian re-identification scenarios. The method includes: a video tracking cluster extracting features from video frame data to form pedestrian feature data packets with header information; a message middleware employing a load-balanced task forwarding strategy to prioritize allocating pedestrian feature data packets to idle edge devices for processing; edge devices in the pedestrian re-identification cluster performing pedestrian re-identification processing on the pedestrian feature data packets to generate pedestrian re-identification results, forming pedestrian ReID data packets with header information, and sending the pedestrian ReID data packets back to the corresponding message queue of the message middleware; and edge devices in the video tracking cluster continuously monitoring the message queue of pedestrian re-identification results in the message middleware, reading the pedestrian ReID data packets with corresponding header information, obtaining the pedestrian re-identification results, and completing the pedestrian re-identification interaction process. This invention aims to achieve high-precision pedestrian re-identification applications through collaborative inference of edge devices.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically to an edge collaborative reasoning interaction method for pedestrian re-identification scenarios applied in an edge computing environment. Background Technology

[0002] Person re-identification (ReID) technology, a key technology in computer vision, primarily addresses the problem of person identification and retrieval across different cameras and scenes. This technology is crucial for various applications, including video surveillance, public safety, and personal identity verification. However, in edge computing environments, the limited processing power of edge devices typically restricts the deployment of lightweight deep learning models. This limits the model's parameter size, consequently affecting ReID accuracy and making it difficult to meet the demands of complex and ever-changing video surveillance environments.

[0003] To further improve ReID accuracy, leveraging cloud computing centers to handle complex computational tasks is a common and feasible solution. Compared to resource-constrained edge computing, the cloud can deploy more complex deep learning models with larger parameter scales. With the powerful feature extraction and pattern recognition capabilities of these models, the accuracy of pedestrian re-identification can be significantly improved. However, this centralized cloud-based computing model has significant drawbacks. First, cloud computing services typically involve continuous rental fees for computing resources, making high computing costs almost unavoidable. Second, large-scale video surveillance data requires continuous transmission between local and cloud environments, impacting the system's real-time responsiveness and placing higher demands on network bandwidth. Furthermore, uploading large amounts of potentially sensitive video surveillance data to the cloud poses potential security and privacy risks, presenting additional challenges to protecting privacy data.

[0004] Currently, the computing resources available for edge devices are insufficient to independently support complex deep learning models, and the ReID accuracy they can achieve is also inadequate for application requirements. While existing cloud processing models can provide powerful computing capabilities, they suffer from shortcomings in terms of real-time performance, bandwidth and storage pressure, and privacy protection.

[0005] Furthermore, in monitoring scenarios requiring rapid identification, such as emergency response, cloud processing latency and potential security risks are key factors limiting their application. Although some distributed methods based on edge computing already exist, these technologies often focus on improving pedestrian re-identification accuracy, lacking consideration for practical application scenarios. Summary of the Invention

[0006] To address the problems existing in the prior art, this invention provides an edge collaborative reasoning interaction method for pedestrian re-identification scenarios. This method not only solves the problem of insufficient accuracy when using only edge devices, but also avoids the shortcomings in real-time performance, bandwidth, storage, and privacy protection caused by local and cloud data interaction.

[0007] The present invention achieves the above objectives through the following technical solutions:

[0008] An edge-collaborative reasoning interaction method for pedestrian re-identification scenarios includes:

[0009] The video tracking cluster takes the video frame data obtained from feature extraction, i.e. pedestrian features, attaches header information that uniquely identifies the transmitting device, forms a pedestrian feature data packet with header information, and sends the pedestrian feature data packet to the message middleware;

[0010] The message middleware receives and caches the pedestrian feature data packets. Based on the real-time status of each edge device in the pedestrian re-identification cluster, it adopts a load-balanced task forwarding strategy to prioritize the allocation of the pedestrian feature data packets to idle edge devices for processing.

[0011] The edge devices in the pedestrian re-identification cluster process pedestrian feature data packets for pedestrian re-identification, generate pedestrian re-identification results, and package the results into data packets, namely pedestrian ReID data, while attaching the header information to form pedestrian ReID data packets with header information. The pedestrian ReID data packets are then sent back to the message queue corresponding to the message middleware.

[0012] The edge devices in the video tracking cluster continuously listen to the message queue of pedestrian re-identification results in the message middleware, identify and read the corresponding pedestrian ReID data packets with header information based on the header information, obtain the pedestrian re-identification results, and complete the pedestrian re-identification interaction process.

[0013] According to the present invention, an edge collaborative reasoning interaction method for pedestrian re-identification scenarios is provided. The video tracking cluster is composed of multiple edge devices with different computing capabilities. Each of these edge devices establishes a data link with at least one smart camera to receive and process video streams from the smart camera in real time.

[0014] Each edge device in the video tracking cluster is configured to analyze the received video stream frame by frame to extract pedestrian features, and attach header information that uniquely identifies the edge device to the extracted pedestrian feature data to form a pedestrian re-identification task data packet to be completed.

[0015] The edge devices of the video tracking cluster continuously poll the pedestrian re-identification result message queue in the message middleware while sending pedestrian feature data to the message middleware, so as to identify and receive the pedestrian re-identification result corresponding to the pedestrian feature data sent by the message middleware based on the header information.

[0016] In this video tracking cluster, each edge device also synchronizes the received pedestrian re-identification results with the original video frames and displays the identification results in the monitoring video, thereby realizing real-time tracking and re-identification of pedestrians.

[0017] According to the edge collaborative reasoning interaction method for pedestrian re-identification scenarios provided by the present invention, each edge device in the video tracking cluster is configured with an agent based on the "producer-consumer" model to realize the sending of pedestrian re-identification tasks and the receiving of identification results;

[0018] Before sending the video frame data as a pedestrian re-identification task to the message middleware, the agent first adds a header information to the video frame data. This header information contains at least the unique identifier of its corresponding edge device to ensure that the subsequent pedestrian re-identification results can be accurately returned to the original edge device based on the identifier.

[0019] After the task is issued, the agent starts a listening process. This process continuously listens to the pedestrian re-identification result message queue in the message middleware. By comparing the header information of the pedestrian ReID data packet in the message queue with the unique identifier of the local device, it identifies and receives the pedestrian re-identification result corresponding to the pedestrian feature data sent locally.

[0020] After receiving the pedestrian re-identification result, the agent transmits it to the edge device for processing, including synchronous display with the original video frame or for further analysis and decision-making, thereby realizing closed-loop management of the pedestrian re-identification task.

[0021] According to the present invention, an edge collaborative reasoning interaction method for pedestrian re-identification scenarios is provided, wherein the pedestrian re-identification cluster is composed of multiple edge devices with different computing capabilities, and each edge device is configured to provide pedestrian re-identification services.

[0022] After receiving pedestrian feature data forwarded through message middleware, each edge device in the pedestrian re-identification cluster performs similarity matching using the locally stored pedestrian feature library or the pedestrian feature information acquired in real time, in order to identify the pedestrian identity or trajectory that matches the received pedestrian feature data.

[0023] After obtaining the pedestrian re-identification result, the edge device in the pedestrian re-identification cluster appends the same header information as the original pedestrian feature data to the result. This header information contains at least the unique identifier of the original sending edge device.

[0024] According to the present invention, an edge collaborative reasoning interaction method for pedestrian re-identification scenarios is provided, wherein the pedestrian re-identification process includes:

[0025] The similarity between the pedestrian feature data to be identified and the pedestrian features stored in the database is calculated. The similarity is calculated using Euclidean distance as the similarity metric. The degree of similarity between the pedestrian to be identified and the pedestrians in the database is evaluated by calculating the Euclidean distance between the feature vectors.

[0026] Based on the calculated similarity, the pedestrians in the database are sorted to generate a list of pedestrian matching results arranged from high to low similarity. This list contains the identity or trajectory information of several pedestrians who are most similar to the pedestrian to be identified.

[0027] According to the edge collaborative reasoning interaction method for pedestrian re-identification scenarios provided by the present invention, the following further steps are performed:

[0028] Locality-Sensitive Hash (LSH) algorithm is used for pedestrian matching and retrieval. In the process of pedestrian matching and retrieval, the pedestrian feature vector to be matched is first mapped to the corresponding hash bucket through a hash function. Then, a search is performed in the same hash bucket as the query pedestrian feature vector to find pedestrian feature vectors similar to the query pedestrian feature vector.

[0029] The hash function should satisfy the locality sensitivity property, that is, for any two pedestrian feature vectors x and y, if the distance d(x,y) between them is less than or equal to a preset distance threshold d1, then the probability that they are hashed to the same bucket is at least p1; if the distance d(x,y) between them is greater than or equal to a preset distance threshold d2 (d2>d1), then the probability that they are hashed to the same bucket is at most p2, where p1 and p2 are constants between 0 and 1.

[0030] According to the present invention, an edge collaborative reasoning interaction method for pedestrian re-identification scenarios is provided. The message middleware is a data stream processing platform with a caching mechanism. It adopts a "publish-subscribe" model to realize asynchronous communication between the video tracking cluster and the pedestrian re-identification cluster. Two topics are defined in the message middleware: a pedestrian feature data topic and a pedestrian re-identification result topic. The pedestrian feature data topic is used to send the video frame data to be identified to the idle edge device. The pedestrian re-identification result topic is used to return the pedestrian re-identification result to the corresponding edge device in the video tracking cluster based on the header information.

[0031] According to the present invention, an edge collaborative reasoning interaction method for pedestrian re-identification scenarios is provided. In this method, the edge device in the video tracking cluster acts as a publisher, which publishes the extracted pedestrian feature data along with the header information of the device's unique identifier to the corresponding topic. The edge device in the pedestrian re-identification cluster acts as a subscriber, which subscribes to one or more topics of interest in order to receive and process pedestrian feature data from these topics and perform pedestrian re-identification tasks.

[0032] In the "publish-subscribe" model, there is no direct communication link between publishers and subscribers. They exchange data asynchronously through message brokers and defined topics, thus achieving a loosely coupled communication method.

[0033] An edge collaborative reasoning interaction method for pedestrian re-identification scenarios provided by the present invention further includes:

[0034] Pedestrian feature data topic maintenance: A pedestrian feature data topic is set in the message middleware to receive and store video frame data with header information published by edge devices in the video tracking cluster; the header information includes at least the timestamp of the video frame, the device's unique identifier, and possible video frame quality evaluation indicators.

[0035] Video tracking cluster publishing operation: Edge devices in the video tracking cluster continuously extract pedestrian features from surveillance videos and publish the extracted pedestrian features along with header information to the pedestrian feature data topic of the message middleware for the pedestrian re-identification cluster to subscribe to.

[0036] An edge collaborative reasoning interaction method for pedestrian re-identification scenarios provided by the present invention further includes:

[0037] Subscription request of pedestrian re-identification cluster: Devices in the pedestrian re-identification cluster send subscription requests to the message middleware to subscribe to the pedestrian feature data topic so as to receive pedestrian feature data from the video tracking cluster in real time or on demand;

[0038] Publication of pedestrian re-identification results: After receiving pedestrian feature data, the devices in the pedestrian re-identification cluster perform pedestrian re-identification processing, and concatenate the processing results with the original header information to form complete pedestrian re-identification result data, which is then published to the pedestrian re-identification result topic in the message middleware;

[0039] Subscription results of video tracking cluster: Devices in the video tracking cluster send subscription requests to the message middleware as needed, requesting to subscribe to the pedestrian re-identification results corresponding to specific header information in the pedestrian re-identification result topic, so as to obtain pedestrian re-identification information related to their own surveillance video.

[0040] Therefore, compared to existing technologies, this invention achieves efficient edge collaborative reasoning, fully utilizing the computing resources of edge device clusters while maintaining high-precision ReID performance. By leveraging the efficient data interaction capabilities of message middleware, functional modules such as pedestrian detection, feature extraction, target tracking, and visualization are integrated. Therefore, this invention has the following beneficial effects:

[0041] 1. Optimized computing resource utilization: This invention employs a load-balanced task forwarding strategy, prioritizing the dispatch of tasks (video frames) to idle edge devices for processing. By dynamically distributing video frames across the entire cluster for parallel processing, the computing resources of edge devices are fully utilized, while effectively handling large-scale data flows to prevent traffic peaks from impacting system performance.

[0042] 2. Real-time data processing: This invention designs a message middleware-based collaborative interaction method for inference, enabling data transmission between edge device clusters. This allows computational tasks to be executed efficiently, and pedestrian re-identification results to be returned to the edge devices in an orderly and accurate manner. Collaboration between edge devices reduces inference latency, thus meeting real-time requirements.

[0043] 3. Data security and privacy protection: The video surveillance data of this invention is transmitted only between edge devices and does not need to be sent out, effectively avoiding the risk of privacy leakage.

[0044] 4. System scalability and reliability: The message middleware of this invention based on the "publish-subscribe" model helps reduce the coupling between system modules and allows two clusters to flexibly add or remove edge devices, thus improving the scalability and reliability of the entire system.

[0045] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0046] Figure 1 This is a flowchart of an embodiment of an edge collaborative reasoning interaction method for pedestrian re-identification scenarios according to the present invention.

[0047] Figure 2 This is a schematic diagram of an embodiment of an edge collaborative reasoning interaction method for pedestrian re-identification scenarios according to the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0049] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0050] See Figure 1 and Figure 2 This embodiment provides an edge collaborative reasoning interaction method for pedestrian re-identification scenarios, including the following steps:

[0051] In step S1, the video tracking cluster adds header information that uniquely identifies the sending device to the video frame data obtained from feature extraction, i.e. pedestrian features, to form a pedestrian feature data packet with header information, and sends the pedestrian feature data packet to the message middleware.

[0052] In step S2, the message middleware receives and caches pedestrian feature data packets. Based on the real-time status of each edge device in the pedestrian re-identification cluster, it adopts a load-balanced task forwarding strategy to prioritize the allocation of pedestrian feature data packets to idle edge devices for processing.

[0053] Step S3: The edge devices in the pedestrian re-identification cluster process the pedestrian feature data packets for pedestrian re-identification, generate pedestrian re-identification results, and package the results into data packets, namely pedestrian ReID data, while attaching header information to form pedestrian ReID data packets with header information. The pedestrian ReID data packets are then sent back to the message queue corresponding to the message middleware.

[0054] Step S4: The edge devices in the video tracking cluster continuously listen to the message queue of the pedestrian re-identification results in the message middleware, identify and read the corresponding pedestrian ReID data packets with header information according to the header information, obtain the pedestrian re-identification results, and complete the pedestrian re-identification interaction process.

[0055] In this embodiment, the video tracking cluster consists of multiple edge devices with different computing capabilities. Each of these edge devices establishes a data link with at least one smart camera to receive and process video streams from the smart camera in real time.

[0056] Each edge device in the video tracking cluster is configured to analyze the received video stream frame by frame to extract pedestrian features, and attach header information that uniquely identifies the edge device to the extracted pedestrian feature data to form a pedestrian re-identification task data packet to be completed.

[0057] In this process, the edge devices of the video tracking cluster continuously poll the pedestrian re-identification result message queue in the message middleware while sending pedestrian feature data to the message middleware, so as to identify and receive the pedestrian re-identification result corresponding to the pedestrian feature data sent by the message middleware based on the header information.

[0058] In this video tracking cluster, each edge device also synchronizes the received pedestrian re-identification results with the original video frames and displays the identification results in the monitoring video, thereby achieving real-time tracking and re-identification of pedestrians.

[0059] In this embodiment, each edge device in the video tracking cluster is configured with an agent based on the "producer-consumer" model to send pedestrian re-identification tasks and receive identification results.

[0060] Before sending video frame data as a pedestrian re-identification task to the message middleware, the agent adds a header to the video frame data. This header must contain at least the unique identifier of the corresponding edge device to ensure that the subsequent pedestrian re-identification results can be accurately returned to the original edge device based on this identifier. After the task is sent, the agent starts a listening process that continuously monitors the pedestrian re-identification result message queue in the message middleware. By comparing the header information of the pedestrian ReID data packets in the message queue with the unique identifier of the local device, the agent identifies and receives the pedestrian re-identification results corresponding to the pedestrian feature data sent locally. After receiving the pedestrian re-identification results, the agent passes them to the edge device for processing, including synchronous display with the original video frames or for further analysis and decision-making, thus achieving closed-loop management of the pedestrian re-identification task.

[0061] In this embodiment, the pedestrian re-identification cluster consists of multiple edge devices with different computing capabilities. Each edge device is configured to provide pedestrian re-identification services. After receiving pedestrian feature data forwarded through message middleware, each edge device in the pedestrian re-identification cluster performs similarity matching using a locally stored pedestrian feature library or real-time acquired pedestrian feature information to identify the pedestrian identity or trajectory that matches the received pedestrian feature data. After obtaining the pedestrian re-identification result, the edge device in the pedestrian re-identification cluster appends the same header information as the original pedestrian feature data to the result. This header information contains at least the unique identifier of the original sending edge device.

[0062] In this embodiment, the pedestrian re-identification process includes: calculating the similarity between the pedestrian feature data to be identified and the pedestrian features stored in the database, wherein the similarity calculation uses Euclidean distance as the similarity metric, and the degree of similarity between the pedestrian to be identified and the pedestrians in the database is evaluated by calculating the Euclidean distance between feature vectors; and sorting the pedestrians in the database according to the calculated similarity to generate a list of pedestrian matching results arranged from high to low similarity, wherein the list contains the identity or trajectory information of several pedestrians most similar to the pedestrian to be identified.

[0063] Optionally, in order to improve the efficiency of pedestrian matching and retrieval, especially in retrieval in a large-scale pedestrian feature database, this embodiment also performs the following: pedestrian matching and retrieval using a locality-sensitive hashing algorithm. In the process of pedestrian matching and retrieval, the pedestrian feature vector to be matched is first mapped to the corresponding hash bucket through a hash function, and then a search is performed in the same hash bucket as the query pedestrian feature vector to find pedestrian feature vectors similar to the query pedestrian feature vector.

[0064] The hash function should satisfy the locality sensitivity property, that is, for any two pedestrian feature vectors x and y, if the distance d(x,y) between them is less than or equal to a preset distance threshold d1, then the probability that they are hashed to the same bucket is at least p1; if the distance d(x,y) between them is greater than or equal to a preset distance threshold d2 (d2>d1), then the probability that they are hashed to the same bucket is at most p2, where p1 and p2 are constants between 0 and 1.

[0065] The construction of the hash bucket includes: using the Locality Sensitive Hash (LSH) algorithm to construct the hash bucket. Specifically, a type of hash function is designed such that similar pedestrian feature vectors (i.e., feature vectors that are close to each other) in the original space have a higher probability of falling into the same hash bucket after being mapped by the hash function; while dissimilar pedestrian feature vectors (i.e., feature vectors that are far apart) have a lower probability of falling into the same hash bucket after being mapped by the hash function.

[0066] In this embodiment, the message middleware is a data stream processing platform with a caching mechanism. It employs a publish-subscribe pattern to achieve asynchronous communication between the video tracking cluster and the pedestrian re-identification cluster. Two topics are defined in the message middleware: a pedestrian feature data topic and a pedestrian re-identification result topic. The pedestrian feature data topic is used to send the video frame data to be identified to an idle edge device, while the pedestrian re-identification result topic is used to return the pedestrian re-identification result to the corresponding edge device in the video tracking cluster based on the header information. Thus, the message middleware is a data stream processing platform with a caching mechanism used to achieve parallel asynchronous message passing between the two clusters. Specifically, the message middleware maintains two topics: pedestrian feature data and pedestrian re-identification result. The former is responsible for sending the video frame data to be identified to an idle edge device that can provide pedestrian re-identification services, while the latter returns the pedestrian re-identification result to the corresponding edge device in the video tracking cluster based on the header information.

[0067] In this embodiment, the edge devices in the video tracking cluster act as publishers, publishing the extracted pedestrian feature data along with header information containing the device's unique identifier to the corresponding topics; the edge devices in the pedestrian re-identification cluster act as subscribers, subscribing to one or more topics of interest in order to receive and process pedestrian feature data from these topics and perform pedestrian re-identification tasks.

[0068] In the "publish-subscribe" model, there is no direct communication link between publishers and subscribers. They exchange data asynchronously through message brokers and defined topics, thus achieving a loosely coupled communication method.

[0069] In this embodiment, the method further includes: maintaining the pedestrian feature data topic: setting a pedestrian feature data topic in the message middleware to receive and store video frame data with header information published by edge devices in the video tracking cluster; the header information includes at least the timestamp of the video frame, the device's unique identifier, and possible video frame quality evaluation metrics.

[0070] Video tracking cluster publishing operation: Edge devices in the video tracking cluster continuously extract pedestrian features from surveillance videos and publish the extracted pedestrian features along with header information to the pedestrian feature data topic of the message middleware for the pedestrian re-identification cluster to subscribe to.

[0071] In this embodiment, it also includes:

[0072] Subscription requests from the pedestrian re-identification cluster: Devices in the pedestrian re-identification cluster send subscription requests to the message middleware to subscribe to the pedestrian feature data topic so as to receive pedestrian feature data from the video tracking cluster in real time or on demand.

[0073] The release of pedestrian re-identification results: After receiving pedestrian feature data, the devices in the pedestrian re-identification cluster perform pedestrian re-identification processing, and then concatenate the processing results with the original header information to form complete pedestrian re-identification result data, which is then published to the pedestrian re-identification result topic in the message middleware.

[0074] Subscription results of video tracking cluster: Devices in the video tracking cluster send subscription requests to the message middleware as needed, requesting to subscribe to the pedestrian re-identification results corresponding to specific header information in the pedestrian re-identification result topic, so as to obtain pedestrian re-identification information related to their own surveillance video.

[0075] Specifically, the video tracking cluster in this embodiment consists of multiple edge devices with varying computing capabilities. These edge devices typically establish data links with smart cameras, receive video data streams in real time, and extract features frame by frame. At this point, all pedestrian feature data can be considered as pedestrian re-identification tasks to be completed. Therefore, the video tracking cluster will continuously generate new pedestrian re-identification tasks. During this process, each edge device uses a proxy based on a "producer-consumer" model to send tasks and receive identification results. Before sending a task to the message middleware, the proxy adds a header to the video frame data, containing the unique identifier of its corresponding device, to ensure that the pedestrian re-identification result can be accurately returned to the original device. After the task is sent, the proxy will continuously monitor the message queue in the message middleware until it finds a pedestrian re-identification result whose header information matches the device identifier. Furthermore, the feature extraction function involved in the video tracking cluster can be implemented using different technologies depending on the application scenario. Taking deep learning technology as an example, a convolutional neural network (CNN) can be trained using a self-annotated private dataset to automatically extract pedestrian features from video frame images, or a CNN model pre-trained on a large-scale dataset such as ImageNet can be used directly for pedestrian feature extraction.

[0076] Specifically, a Convolutional Neural Network (CNN) was trained using a self-annotated private dataset. This CNN was configured to automatically and efficiently extract pedestrian features from video frame images. The private dataset contained precise annotations of information such as the location, size, and identity of pedestrian targets to ensure that the trained CNN model could be applied to specific video tracking scenarios.

[0077] One approach involves directly using pre-trained CNN models on publicly available large-scale datasets, such as ImageNet, and applying these models to extract pedestrian features from video frames through transfer learning or fine-tuning. This method leverages the rich feature representation capabilities learned by the pre-trained model on large-scale datasets, enabling rapid extraction of pedestrian features with minimal or no adjustments to new scenes.

[0078] Of course, in either method, the extracted pedestrian features will be appended with header information containing the edge device's unique identifier, so that task distribution, processing and result feedback can be carried out in the message middleware, ensuring the accurate execution of the pedestrian re-identification task and the correct feedback of the results.

[0079] Specifically, the message middleware in this embodiment uses a "publish-subscribe" model for data interaction to achieve asynchronous communication between the video tracking cluster and the pedestrian re-identification cluster. In this model, publishers and subscribers do not have direct contact but interact through topics. Specifically, the message middleware maintains two topics: pedestrian feature data and pedestrian re-identification results. Edge devices in the video tracking cluster continuously "publish" video frame data with header information to the pedestrian feature data topic in the message middleware, while the pedestrian re-identification cluster requests to "subscribe" to the pedestrian feature data from the message middleware. Additionally, devices in the pedestrian re-identification cluster concatenate the header information and the pedestrian re-identification results and "publish" them to the pedestrian re-identification results topic in the message middleware, so that devices in the video tracking cluster can continuously request to "subscribe" to pedestrian re-identification results with specific header information from the message middleware.

[0080] Specifically, the pedestrian re-identification cluster in this embodiment consists of multiple edge devices with varying computing capabilities. These edge devices typically deploy a pedestrian re-identification service, which subscribes to pedestrian feature data in a message middleware to measure the similarity of pedestrian features and perform pedestrian matching. Generally, the pedestrian re-identification task first needs to calculate the similarity between the pedestrian to be identified and the pedestrian features in the database. Subsequently, the results are sorted by similarity and a list of pedestrian matching results is output. The pedestrian re-identification service design in this embodiment can be implemented using different technologies depending on the application scenario. For example, Euclidean distance can be used for similarity measurement; Locality Sensitive Hashing (LSH) can be used for pedestrian matching and retrieval.

[0081] In practical applications, during a complete edge collaborative inference interaction, each of the two edge device clusters only needs to focus on a single task: pedestrian feature extraction or pedestrian re-identification. Through a task forwarding strategy with load balancing via message middleware, the two clusters can efficiently interact and collaborate to complete complex pedestrian re-identification tasks. To verify the effectiveness of this method, a video tracking cluster consisting of four edge devices and a pedestrian re-identification cluster consisting of two edge devices were used. The experiment simulated high-load conditions that might be encountered in real-world applications, with each device in the video tracking cluster sending 1000 pedestrian feature recognition tasks to the pedestrian re-identification cluster concurrently, for a total of 4000 tasks. Experimental results show that under high-concurrency testing, the two devices in the pedestrian re-identification cluster each processed approximately 2000 tasks, effectively achieving load balancing within the cluster. Simultaneously, the average response latency was only 33 milliseconds, meeting real-time requirements.

[0082] In summary, this embodiment achieves efficient edge collaborative inference, fully utilizing the computing resources of the edge device cluster while maintaining high-precision ReID performance. It leverages the efficient data interaction capabilities of message middleware to integrate functional modules such as pedestrian detection, feature extraction, target tracking, and visualization.

[0083] Furthermore, this embodiment employs a load-balanced task forwarding strategy, prioritizing the dispatch of tasks (video frames) to be processed to idle edge devices. By dynamically distributing video frames across the entire cluster for parallel processing, the computing resources of edge devices are fully utilized, while effectively handling large-scale data flows to prevent traffic peaks from impacting system performance.

[0084] Furthermore, this embodiment designs a message middleware-based inference collaborative interaction method to realize data transmission between edge device clusters, enabling efficient execution of computing tasks and orderly and accurate return of pedestrian re-identification results to the edge devices. Collaboration between edge devices reduces inference latency and meets real-time requirements.

[0085] Furthermore, in this embodiment, the video surveillance data is transmitted only between edge devices without being sent out, effectively avoiding the risk of privacy leakage.

[0086] Furthermore, the message middleware based on the "publish-subscribe" model in this embodiment helps reduce the coupling between system modules and allows two clusters to flexibly add or remove edge devices, thereby improving the scalability and reliability of the entire system.

[0087] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0088] The above embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Any non-substantial changes and substitutions made by those skilled in the art based on the present invention shall fall within the scope of protection claimed by the present invention.

Claims

1. An edge-collaborative reasoning interaction method for pedestrian re-identification scenarios, characterized in that, Includes the following steps: The video tracking cluster takes the video frame data obtained from feature extraction, i.e. pedestrian features, attaches header information that uniquely identifies the transmitting device, forms a pedestrian feature data packet with header information, and sends the pedestrian feature data packet to the message middleware; The message middleware receives and caches the pedestrian feature data packets. Based on the real-time status of each edge device in the pedestrian re-identification cluster, it adopts a load-balanced task forwarding strategy to prioritize the allocation of the pedestrian feature data packets to idle edge devices for processing. The edge devices in the pedestrian re-identification cluster process pedestrian feature data packets for pedestrian re-identification, generate pedestrian re-identification results, and package the results into data packets, namely pedestrian ReID data, while attaching the header information to form pedestrian ReID data packets with header information. The pedestrian ReID data packets are then sent back to the message queue corresponding to the message middleware. The edge devices in the video tracking cluster continuously listen to the message queue of pedestrian re-identification results in the message middleware, identify and read the corresponding pedestrian ReID data packets with header information according to the header information, obtain the pedestrian re-identification results, and complete the pedestrian re-identification interaction process. The video tracking cluster consists of multiple edge devices with different computing capabilities. Each of these edge devices establishes a data link with at least one smart camera to receive and process video streams from the smart camera in real time. Each edge device in the video tracking cluster is configured to analyze the received video stream frame by frame to extract pedestrian features, and attach header information that uniquely identifies the edge device to the extracted pedestrian feature data to form a pedestrian re-identification task data packet to be completed. The edge devices of the video tracking cluster continuously poll the pedestrian re-identification result message queue in the message middleware while sending pedestrian feature data to the message middleware, so as to identify and receive the pedestrian re-identification result corresponding to the pedestrian feature data sent by the message middleware based on the header information. Each edge device in the video tracking cluster will also synchronize the received pedestrian re-identification results with the original video frames and display the identification results in the monitoring video, thereby realizing real-time tracking and re-identification of pedestrians; The message middleware is a data stream processing platform with a caching mechanism. It adopts a "publish-subscribe" model to realize asynchronous communication between the video tracking cluster and the pedestrian re-identification cluster. Two topics are defined in the message middleware: the pedestrian feature data topic and the pedestrian re-identification result topic. The pedestrian feature data topic is used to send the video frame data to be identified to the idle edge device, and the pedestrian re-identification result topic is used to return the pedestrian re-identification result to the corresponding edge device in the video tracking cluster based on the header information. In the "publish-subscribe" model, there is no direct communication link between publishers and subscribers. They exchange data asynchronously through message middleware and defined topics, thereby achieving a loosely coupled communication method.

2. The method according to claim 1, characterized in that: Each edge device in the video tracking cluster is configured with an agent based on the "producer-consumer" model to send pedestrian re-identification tasks and receive identification results; Before sending the video frame data as a pedestrian re-identification task to the message middleware, the agent first adds a header information to the video frame data. This header information contains at least the unique identifier of its corresponding edge device to ensure that the subsequent pedestrian re-identification results can be accurately returned to the original edge device based on the identifier. After the task is issued, the agent starts a listening process. This process continuously listens to the pedestrian re-identification result message queue in the message middleware. By comparing the header information of the pedestrian ReID data packet in the message queue with the unique identifier of the local device, it identifies and receives the pedestrian re-identification result corresponding to the pedestrian feature data sent locally. After receiving the pedestrian re-identification result, the agent transmits it to the edge device for processing, including synchronous display with the original video frame or for further analysis and decision-making, thereby realizing closed-loop management of the pedestrian re-identification task.

3. The method according to claim 1, characterized in that: The pedestrian re-identification cluster consists of multiple edge devices with different computing capabilities, and each edge device is configured to provide pedestrian re-identification services. After receiving pedestrian feature data forwarded through message middleware, each edge device in the pedestrian re-identification cluster performs similarity matching using the locally stored pedestrian feature library or the pedestrian feature information acquired in real time, in order to identify the pedestrian identity or trajectory that matches the received pedestrian feature data. After obtaining the pedestrian re-identification result, the edge device in the pedestrian re-identification cluster appends the same header information as the original pedestrian feature data to the result. This header information contains at least the unique identifier of the original sending edge device.

4. The method according to claim 1, characterized in that, The pedestrian re-identification process includes: The similarity between the pedestrian feature data to be identified and the pedestrian features stored in the database is calculated. The similarity is calculated using Euclidean distance as the similarity metric. The degree of similarity between the pedestrian to be identified and the pedestrians in the database is evaluated by calculating the Euclidean distance between the feature vectors. Based on the calculated similarity, the pedestrians in the database are sorted to generate a list of pedestrian matching results arranged from high to low similarity. This list contains the identity or trajectory information of several pedestrians who are most similar to the pedestrian to be identified.

5. The method according to claim 1, characterized in that, Also execute: Locality-Sensitive Hash (LSH) algorithm is used for pedestrian matching and retrieval. In the process of pedestrian matching and retrieval, the pedestrian feature vector to be matched is first mapped to the corresponding hash bucket through a hash function. Then, a search is performed in the same hash bucket as the query pedestrian feature vector to find pedestrian feature vectors similar to the query pedestrian feature vector. The hash function should satisfy the locality sensitivity property, that is, for any two pedestrian feature vectors x and y, if the distance d(x,y) between them is less than or equal to a preset distance threshold d1, then the probability that they are hashed to the same bucket is at least p1; if the distance d(x,y) between them is greater than or equal to a preset distance threshold d2, then the probability that they are hashed to the same bucket is at most p2, where p1 and p2 are constants between 0 and 1.

6. The method according to claim 1, characterized in that: In the video tracking cluster, edge devices act as publishers, publishing extracted pedestrian feature data along with header information containing the device's unique identifier to the corresponding topics. In the pedestrian re-identification cluster, edge devices act as subscribers, subscribing to one or more topics of interest in order to receive and process pedestrian feature data from these topics and perform pedestrian re-identification tasks.

7. The method according to claim 1, characterized in that, Also includes: Pedestrian feature data topic maintenance: A pedestrian feature data topic is set in the message middleware to receive and store video frame data with header information published by edge devices in the video tracking cluster; the header information includes at least the timestamp of the video frame and the device's unique identifier; Video tracking cluster publishing operation: Edge devices in the video tracking cluster continuously extract pedestrian features from surveillance videos and publish the extracted pedestrian features along with header information to the pedestrian feature data topic of the message middleware for the pedestrian re-identification cluster to subscribe to.

8. The method according to claim 7, characterized in that, Also includes: Subscription request of pedestrian re-identification cluster: Devices in the pedestrian re-identification cluster send subscription requests to the message middleware to subscribe to the pedestrian feature data topic so as to receive pedestrian feature data from the video tracking cluster in real time or on demand; Publication of pedestrian re-identification results: After receiving pedestrian feature data, the devices in the pedestrian re-identification cluster perform pedestrian re-identification processing, and concatenate the processing results with the original header information to form complete pedestrian re-identification result data, which is then published to the pedestrian re-identification result topic in the message middleware; Subscription results of video tracking cluster: Devices in the video tracking cluster send subscription requests to the message middleware as needed, requesting to subscribe to the pedestrian re-identification results corresponding to specific header information in the pedestrian re-identification result topic, so as to obtain pedestrian re-identification information related to their own surveillance video.

Images