Edge collaborative reasoning interaction method for pedestrian re-recognition scene

Abstract

The invention provides a pedestrian re-recognition scene-oriented edge collaborative reasoning interaction method, which comprises the following steps that: a video tracking cluster forms a pedestrian feature data packet with head information from video frame data obtained by feature extraction; the message-oriented middleware adopts a load-balanced task forwarding strategy to preferentially distribute the pedestrian feature data packet to the edge device in an idle state for processing; the edge device in the pedestrian re-identification cluster performs pedestrian re-identification processing on the pedestrian feature data packet, generates a pedestrian re-identification result, forms a pedestrian ReID data packet with head information, and sends the pedestrian ReID data packet back to a message queue corresponding to the message middleware; and the edge device in the video tracking cluster continuously monitors the message queue of the pedestrian re-identification result in the message-oriented middleware, reads the pedestrian ReID data packet attached with the corresponding header information, obtains the pedestrian re-identification result, and completes the interaction process of pedestrian re-identification. The invention aims to realize high-precision pedestrian re-identification application through reasoning cooperation of edge equipment.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to an edge collaborative reasoning interaction method for a pedestrian re-identification scenario applied in an edge computing environment. Background Art

[0002] As one of the key technologies in the field of computer vision, pedestrian re-identification (ReID) technology mainly solves the problem of pedestrian recognition and retrieval across cameras and scenarios. This technology is crucial for multiple application scenarios such as video surveillance, public safety, and personal identity verification. However, in edge computing environments, due to the limited processing power of edge devices, only lightweight deep learning models can usually be deployed. This limits the parameter scale of the model to a certain extent, which in turn affects the accuracy of ReID and makes it difficult to meet the needs of complex and changing video surveillance environments.

[0003] In order to further improve the accuracy of ReID, it is a common and feasible solution to use cloud computing centers to handle complex computing tasks. Compared with the resource-constrained edge, the cloud can deploy deep learning models with more complex structures and larger parameter scales. With the powerful feature extraction and pattern recognition capabilities of the model, the accuracy of pedestrian re-identification can be greatly improved. However, this centralized computing model based on the cloud has obvious shortcomings. First, cloud computing services usually involve continuous computing resource rental fees, and high computing costs are almost inevitable. Secondly, large-scale video surveillance data needs to be continuously transmitted between the local and the cloud, which not only affects the real-time response capability of the system but also puts higher requirements on network bandwidth. In addition, uploading a large amount of video surveillance data that may contain sensitive information to the cloud poses potential security and privacy leakage risks, which brings additional challenges to the protection of privacy data.

[0004] At present, the computing resources of edge devices are difficult to independently carry complex deep learning models, and the ReID accuracy they can achieve cannot meet application requirements. Although the existing cloud processing model can provide powerful computing power, it is insufficient in terms of real-time performance, bandwidth and storage pressure, and privacy protection.

[0005] In addition, in monitoring scenarios that require rapid identification, such as emergency response, the latency and potential security risks of cloud processing have become key factors restricting its application. Although there are some distributed methods based on edge computing, these technologies often focus on how to improve the accuracy of pedestrian re-identification and lack consideration of actual application scenarios. Summary of the invention

[0006] In response to the problems existing in the above-mentioned prior art, the present invention provides an edge collaborative reasoning interaction method for pedestrian re-identification scenarios. This method not only solves the problem of insufficient accuracy when only edge devices are used, but also avoids the deficiencies in real-time, bandwidth, storage, and privacy protection brought about by the interaction between local and cloud data.

[0007] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0008] An edge collaborative reasoning interaction method for pedestrian re-identification scenarios, including:

[0009] The video tracking cluster attaches the video frame data obtained by feature extraction, i.e., pedestrian features, to the header information that uniquely identifies the sending device to form a pedestrian feature data packet with the header information, and sends the pedestrian feature data packet to the message middleware;

[0010] The message middleware receives and caches the pedestrian feature data packet, and adopts a load-balanced task forwarding strategy according to the real-time status of each edge device in the pedestrian re-identification cluster, and preferentially allocates the pedestrian feature data packet to the edge device in an idle state for processing;

[0011] The edge device in the pedestrian re-identification cluster performs pedestrian re-identification processing on the pedestrian feature data packet, generates a pedestrian re-identification result, and packages the result into a data packet, i.e., pedestrian ReID data, and attaches the header information to form a pedestrian ReID data packet with header information, and sends the pedestrian ReID data packet back to the message queue corresponding to the message middleware;

[0012] The edge devices in the video tracking cluster continuously monitor the message queue of pedestrian re-identification results in the message middleware, identify and read the corresponding pedestrian ReID data packet with header information according to the header information, obtain the pedestrian re-identification results, and complete the interactive process of pedestrian re-identification.

[0013] According to an edge collaborative reasoning interaction method for pedestrian re-identification scenarios provided by the present invention, the video tracking cluster is composed of multiple edge devices with different computing capabilities, and each of these edge devices establishes a data link with at least one smart camera to receive and process video streams from the smart camera in real time;

[0014] Each edge device of the video tracking cluster is configured to analyze the received video stream frame by frame to extract pedestrian features, and attach the extracted pedestrian feature data to header information that uniquely identifies the edge device to form a pedestrian re-identification task data packet to be completed;

[0015] Wherein, while sending the pedestrian feature data to the message middleware, the edge device of the video tracking cluster continuously polls the pedestrian re-identification result message queue in the message middleware to identify and receive the pedestrian re-identification result corresponding to the pedestrian feature data sent by it according to the header information;

[0016] Among them, each edge device of the video tracking cluster also synchronizes the received pedestrian re-identification results with the original video frames, and displays the recognition results in the monitoring video, thereby realizing real-time tracking and re-identification of pedestrians.

[0017] According to an edge collaborative reasoning interaction method for pedestrian re-identification scenarios provided by the present invention, each edge device in the video tracking cluster is configured with an agent based on the "producer-consumer" model to implement the sending of pedestrian re-identification tasks and the receiving of recognition results;

[0018] Before sending the video frame data as a person re-identification task to the message middleware, the proxy first adds a header information in front of the video frame data, and the header information at least contains a unique identifier of the corresponding edge device to ensure that the subsequent person re-identification result can be accurately returned to the original edge device based on the identifier;

[0019] After the task is issued, the agent starts a monitoring process, which continuously monitors the pedestrian re-identification result message queue in the message middleware, and identifies and receives the pedestrian re-identification result corresponding to the pedestrian feature data sent locally by comparing the header information of the pedestrian ReID data packet in the message queue with the unique identifier of the local device;

[0020] After receiving the pedestrian re-identification result, the agent passes it to the edge device for processing, including synchronous display with the original video frame or for further analysis and decision-making, thereby realizing closed-loop management of the pedestrian re-identification task.

[0021] According to an edge collaborative reasoning interaction method for a pedestrian re-identification scenario provided by the present invention, the pedestrian re-identification cluster is composed of a plurality of edge devices with different computing capabilities, and each edge device is configured to provide a pedestrian re-identification service;

[0022] After receiving the pedestrian feature data forwarded by the message middleware, each edge device in the pedestrian re-identification cluster uses the locally stored pedestrian feature library or the pedestrian feature information obtained in real time to perform similarity matching to identify the pedestrian identity or trajectory that matches the received pedestrian feature data;

[0023] After obtaining the pedestrian re-identification result, the edge device in the pedestrian re-identification cluster attaches the same header information as the original pedestrian feature data to the result, and the header information at least contains the unique identifier of the original sender edge device.

[0024] According to an edge collaborative reasoning interaction method for a pedestrian re-identification scenario provided by the present invention, the pedestrian re-identification process includes:

[0025] Calculate the similarity between the feature data of the pedestrian to be identified and the features of the pedestrians stored in the database, wherein the similarity is calculated using the Euclidean distance as a similarity metric, and the similarity between the pedestrian to be identified and the pedestrians in the database is evaluated by calculating the Euclidean distance between the feature vectors;

[0026] According to the calculated similarity, the pedestrians in the database are sorted to generate a pedestrian matching result list arranged from high to low according to the similarity, wherein the list contains the identities or trajectory information of several pedestrians that are most similar to the pedestrian to be identified.

[0027] According to an edge collaborative reasoning interaction method for pedestrian re-identification scenarios provided by the present invention, the following steps are also performed:

[0028] The locality sensitive hashing algorithm is used for pedestrian matching and retrieval. In the process of pedestrian matching and retrieval, the pedestrian feature vector to be matched is first mapped to the corresponding hash bucket through the hash function, and then the hash bucket with the same pedestrian feature vector as the query is searched to find the pedestrian feature vector similar to the query pedestrian feature vector.

[0029] Among them, the hash function should satisfy the local sensitivity property, that is, for any two pedestrian feature vectors x and y, if the distance d(x, y) between the two is less than or equal to the preset distance threshold d1, then the probability that they are hashed to the same bucket is at least p1; if the distance d(x, y) between the two is greater than or equal to the preset distance threshold d2 (d2>d1), then the probability that they are hashed to the same bucket is at most p2, where p1 and p2 are constants between 0 and 1.

[0030] According to an edge collaborative reasoning interaction method for pedestrian re-identification scenarios provided by the present invention, the message middleware is a data stream processing platform with a caching mechanism, and a "publish-subscribe" mode is adopted to realize asynchronous communication between a video tracking cluster and a pedestrian re-identification cluster. Two topics are defined in the message middleware, namely, a pedestrian feature data topic and a pedestrian re-identification result topic. The pedestrian feature data topic is used to send video frame data to be identified to idle edge devices, and the pedestrian re-identification result topic is used to return the pedestrian re-identification result to the corresponding edge device in the video tracking cluster based on the header information.

[0031] According to an edge collaborative reasoning interaction method for pedestrian re-identification scenarios provided by the present invention, the edge device in the video tracking cluster acts as a publisher, and publishes the extracted pedestrian feature data with the header information of the device unique identifier to the corresponding topic; the edge device in the pedestrian re-identification cluster acts as a subscriber, and subscribes to one or more topics of interest to receive and process the pedestrian feature data from these topics and perform pedestrian re-identification tasks;

[0032] In the "publish-subscribe" mode, there is no direct communication link between publishers and subscribers. They exchange data asynchronously through message middleware and defined topics, thus achieving a loosely coupled communication mode.

[0033] According to the present invention, an edge collaborative reasoning interaction method for pedestrian re-identification scenarios also includes:

[0034] Maintenance of pedestrian feature data topic: A pedestrian feature data topic is set in the message middleware to receive and store video frame data with header information published by edge devices in the video tracking cluster; the header information at least includes the timestamp of the video frame, the device unique identifier, and possible video frame quality assessment indicators;

[0035] Publishing operation of the video tracking cluster: The edge devices in the video tracking cluster continuously extract pedestrian features from the surveillance video, and publish the extracted pedestrian features together with the header information to the pedestrian feature data topic of the message middleware for subscription by the pedestrian re-identification cluster.

[0036] According to the present invention, an edge collaborative reasoning interaction method for pedestrian re-identification scenarios also includes:

[0037] Subscription request of the pedestrian re-identification cluster: The devices in the pedestrian re-identification cluster send subscription requests to the message middleware to subscribe to the pedestrian feature data topic, so as to receive pedestrian feature data from the video tracking cluster in real time or on demand;

[0038] Publishing of person re-identification results: After receiving the person feature data, the devices in the person re-identification cluster perform person re-identification processing and concatenate the processing results with the original header information to form complete person re-identification result data, which is then published to the person re-identification result topic in the message middleware;

[0039] Subscription results of the video tracking cluster: Devices in the video tracking cluster send subscription requests to the message middleware as needed, requesting to subscribe to the pedestrian re-identification results corresponding to specific header information in the pedestrian re-identification result topic, so as to obtain pedestrian re-identification information related to their own surveillance videos.

[0040] It can be seen that compared with the prior art, the present invention fully utilizes the computing resources of the edge device cluster by realizing efficient edge collaborative reasoning while maintaining high-precision ReID performance. With the efficient data interaction capability of the message middleware, the functional modules such as pedestrian detection, feature extraction, target tracking and visualization display are integrated. Therefore, the present invention has the following beneficial effects:

[0041] 1. Optimize computing resource utilization: The present invention adopts a load-balanced task forwarding strategy to send the tasks to be processed (video frames) to idle edge devices for processing first. By dynamically distributing video frames in the entire cluster for parallel processing, the computing resources of edge devices can be fully utilized, and large-scale data flows can be effectively handled to prevent traffic peaks from affecting system performance.

[0042] 2. Real-time data processing: This paper designs a reasoning collaborative interaction method based on message middleware to realize data transmission between edge device clusters, so that computing tasks can be executed efficiently and pedestrian re-identification results can be returned to edge devices in an orderly and accurate manner. The reasoning delay can be reduced through the collaboration between edge devices to meet the real-time requirements.

[0043] 3. Data security and privacy protection: The video surveillance data of the present invention is only transmitted between edge devices without being sent outward, which effectively avoids the risk of privacy leakage.

[0044] 4. System scalability and reliability: The message middleware based on the "publish-subscribe" mode of the present invention helps to reduce the coupling between system modules and allows the two clusters to flexibly add or reduce edge devices, thereby improving the scalability and reliability of the entire system.

[0045] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a flowchart of an embodiment of an edge collaborative reasoning interaction method for pedestrian re-identification scenarios of the present invention.

[0047] Figure 2 It is a schematic diagram of an embodiment of an edge collaborative reasoning interaction method for a pedestrian re-identification scenario of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0050] See also Figure 1 and Figure 2 , this embodiment provides an edge collaborative reasoning interaction method for pedestrian re-identification scenarios, including the following steps:

[0051] Step S1, the video tracking cluster attaches header information that uniquely identifies the sending device to the video frame data obtained by feature extraction, namely, pedestrian features, to form a pedestrian feature data packet with header information, and sends the pedestrian feature data packet to the message middleware.

[0052] In step S2, the message middleware receives and caches pedestrian feature data packets, and adopts a load-balanced task forwarding strategy based on the real-time status of each edge device in the pedestrian re-identification cluster, and preferentially allocates pedestrian feature data packets to idle edge devices for processing.

[0053] In step S3, the edge device in the pedestrian re-identification cluster performs pedestrian re-identification processing on the pedestrian feature data packet, generates a pedestrian re-identification result, and packages the result into a data packet, namely, pedestrian ReID data, and attaches header information to form a pedestrian ReID data packet with header information, and sends the pedestrian ReID data packet back to the message queue corresponding to the message middleware.

[0054] In step S4, the edge device in the video tracking cluster continuously monitors the message queue of the pedestrian re-identification results in the message middleware, identifies and reads the corresponding pedestrian ReID data packet with header information according to the header information, obtains the pedestrian re-identification results, and completes the interactive process of pedestrian re-identification.

[0055] In this embodiment, the video tracking cluster is composed of multiple edge devices with different computing capabilities. Each of these edge devices establishes a data link with at least one smart camera to receive and process the video stream from the smart camera in real time.

[0056] Among them, each edge device in the video tracking cluster is configured to analyze the received video stream frame by frame to extract pedestrian features, and attach the extracted pedestrian feature data to the header information that uniquely identifies the edge device to form a pedestrian re-identification task data packet to be completed.

[0057] Among them, the edge device of the video tracking cluster continuously polls the pedestrian re-identification result message queue in the message middleware while sending the pedestrian feature data to the message middleware, so as to identify and receive the pedestrian re-identification result corresponding to the pedestrian feature data sent by it according to the header information.

[0058] Among them, each edge device of the video tracking cluster will also synchronize the received pedestrian re-identification results with the original video frames, and display the recognition results in the surveillance video, thereby realizing real-time tracking and re-identification of pedestrians.

[0059] In this embodiment, each edge device in the video tracking cluster is configured with an agent based on the "producer-consumer" model to implement the sending of pedestrian re-identification tasks and the receiving of recognition results.

[0060] Among them, before the agent sends the video frame data as a pedestrian re-identification task to the message middleware, it first adds a header information in front of the video frame data. The header information at least contains the unique identifier of its corresponding edge device to ensure that the subsequent pedestrian re-identification results can be accurately returned to the original edge device based on the identifier; after the task is issued, the agent starts a listening process, which continuously listens to the pedestrian re-identification result message queue in the message middleware, and identifies and receives the pedestrian re-identification results corresponding to the pedestrian feature data sent locally by comparing the header information of the pedestrian ReID data packet in the message queue with the unique identifier of the local device; after receiving the pedestrian re-identification result, the agent passes it to the edge device for processing, including synchronous display with the original video frame or for further analysis and decision-making, to achieve closed-loop management of the pedestrian re-identification task.

[0061] In this embodiment, the pedestrian re-identification cluster is composed of multiple edge devices with different computing capabilities, and each edge device is configured to provide pedestrian re-identification services; after receiving the pedestrian feature data forwarded through the message middleware, each edge device in the pedestrian re-identification cluster uses the locally stored pedestrian feature library or the pedestrian feature information obtained in real time to perform similarity matching to identify the pedestrian identity or trajectory that matches the received pedestrian feature data; after obtaining the pedestrian re-identification result, the edge device in the pedestrian re-identification cluster attaches the same header information as the original pedestrian feature data to the result, and the header information at least contains the unique identifier of the original sender edge device.

[0062] In this embodiment, the pedestrian re-identification process includes: calculating the similarity between the feature data of the pedestrian to be identified and the pedestrian features stored in the database, wherein the similarity is calculated using the Euclidean distance as a similarity metric, and the similarity between the pedestrian to be identified and the pedestrians in the database is evaluated by calculating the Euclidean distance between the feature vectors; according to the calculated similarity, the pedestrians in the database are sorted, and a list of pedestrian matching results arranged in descending order according to the similarity is generated, wherein the list contains several pedestrian identities or trajectory information that are most similar to the pedestrian to be identified.

[0063] Optionally, in order to improve the efficiency of pedestrian matching and retrieval, especially retrieval in a large-scale pedestrian feature database, the present embodiment also performs: using a local sensitive hashing algorithm for pedestrian matching and retrieval. During the pedestrian matching and retrieval process, the pedestrian feature vector to be matched is first mapped to a corresponding hash bucket through a hash function, and then a search is performed in the same hash bucket as the query pedestrian feature vector to find a pedestrian feature vector similar to the query pedestrian feature vector.

[0064] Among them, the hash function should satisfy the local sensitivity property, that is, for any two pedestrian feature vectors x and y, if the distance d(x, y) between the two is less than or equal to the preset distance threshold d1, then the probability that they are hashed to the same bucket is at least p1; if the distance d(x, y) between the two is greater than or equal to the preset distance threshold d2 (d2>d1), then the probability that they are hashed to the same bucket is at most p2, where p1 and p2 are constants between 0 and 1.

[0065] Among them, the construction of hash buckets includes: using a local sensitive hashing algorithm to construct hash buckets, specifically designing a type of hash function so that similar pedestrian feature vectors in the original space (i.e., feature vectors with close distances) have a higher probability of falling into the same hash bucket after being mapped by the hash function; while dissimilar pedestrian feature vectors (i.e., feature vectors with far distances) have a lower probability of falling into the same hash bucket after being mapped by the hash function.

[0066] In this embodiment, the message middleware is a data stream processing platform with a cache mechanism, and the "publish-subscribe" mode is used to realize asynchronous communication between the video tracking cluster and the pedestrian re-identification cluster. Two topics are defined in the message middleware, namely the pedestrian feature data topic and the pedestrian re-identification result topic. The pedestrian feature data topic is used to send the video frame data to be identified to the idle edge device, and the pedestrian re-identification result topic is used to return the pedestrian re-identification result to the corresponding edge device in the video tracking cluster based on the header information. It can be seen that the message middleware is a data stream processing platform with a cache mechanism, which is used to realize parallel asynchronous message transmission between two clusters. Among them, the message middleware maintains two topics (Topic), namely pedestrian feature data and pedestrian re-identification results. The former is responsible for sending the video frame data to be identified to the idle edge device that can provide pedestrian re-identification services, and the latter returns the pedestrian re-identification results to the corresponding edge device in the video tracking cluster based on the header information.

[0067] In this embodiment, the edge device in the video tracking cluster acts as a publisher, publishing the extracted pedestrian feature data along with the header information of the device's unique identifier to the corresponding topic; the edge device in the pedestrian re-identification cluster acts as a subscriber, subscribing to one or more topics of interest in order to receive and process the pedestrian feature data from these topics and perform pedestrian re-identification tasks.

[0068] In the "publish-subscribe" mode, there is no direct communication link between publishers and subscribers. They exchange data asynchronously through message middleware and defined topics, thus achieving a loosely coupled communication mode.

[0069] In this embodiment, it also includes: maintenance of pedestrian feature data topics: setting up pedestrian feature data topics in the message middleware to receive and store video frame data with header information published by edge devices in the video tracking cluster; the header information contains at least the timestamp of the video frame, the device unique identifier, and possible video frame quality assessment indicators.

[0070] Publishing operation of the video tracking cluster: The edge devices in the video tracking cluster continuously extract pedestrian features from the surveillance video, and publish the extracted pedestrian features together with the header information to the pedestrian feature data topic of the message middleware for subscription by the pedestrian re-identification cluster.

[0071] In this embodiment, it also includes:

[0072] Subscription request of the pedestrian re-identification cluster: The devices in the pedestrian re-identification cluster send subscription requests to the message middleware, requesting to subscribe to the pedestrian feature data topic, so as to receive pedestrian feature data from the video tracking cluster in real time or on demand.

[0073] Publishing of pedestrian re-identification results: After receiving the pedestrian feature data, the devices in the pedestrian re-identification cluster perform pedestrian re-identification processing and splice the processing results with the original header information to form complete pedestrian re-identification result data, which is then published to the pedestrian re-identification result topic in the message middleware.

[0074] Subscription results of the video tracking cluster: Devices in the video tracking cluster send subscription requests to the message middleware as needed, requesting to subscribe to the pedestrian re-identification results corresponding to specific header information in the pedestrian re-identification result topic, so as to obtain pedestrian re-identification information related to their own surveillance videos.

[0075] Specifically, the video tracking cluster of this embodiment is composed of multiple edge devices with different computing capabilities. These edge devices usually establish a data link with the smart camera, receive the video data stream in real time, and extract features frame by frame. At this time, all pedestrian feature data can be regarded as pedestrian re-identification tasks to be completed. Therefore, the video tracking cluster will continuously generate new pedestrian re-identification tasks. In this process, each edge device implements task sending and recognition result reception through an agent based on the "producer-consumer" model. Before sending the task to the message middleware, the agent will add a header information before the video frame data, containing the unique identifier of its corresponding device, to ensure that the pedestrian re-identification result can be accurately returned to the original device. After the task is issued, the agent will continue to listen to the message queue in the message middleware until a pedestrian re-identification result with the header information consistent with the device identifier is found. In addition, the feature extraction function involved in the video tracking cluster can be implemented using different technologies according to different application scenarios. Taking deep learning technology as an example, a convolutional neural network (CNN) can be trained using a self-labeled private dataset to automatically extract pedestrian features from video frame images, or a CNN model that has been pre-trained on public large-scale datasets such as ImageNet can be directly used to extract pedestrian features.

[0076] A convolutional neural network (CNN) is trained using a self-annotated private dataset. The CNN is configured to automatically and efficiently extract pedestrian features from video frames. The private dataset contains precise annotations of the location, size, and identity of pedestrian targets to ensure that the trained CNN model is applicable to specific video tracking scenarios.

[0077] Among them, the pre-trained CNN model is directly used in public large-scale datasets, such as ImageNet, and the model is applied to the pedestrian feature extraction in video frame images through transfer learning or fine-tuning. This method utilizes the rich feature representation capabilities learned by the pre-trained model on large-scale datasets, and can achieve rapid extraction of pedestrian features with little or no adjustment to new scenes.

[0078] Of course, in either method, the extracted pedestrian features will be attached with header information containing the unique identifier of the edge device to facilitate subsequent task distribution, processing and result feedback in the message middleware, ensuring the accurate execution of the pedestrian re-identification task and the correct feedback of the results.

[0079] Specifically, the message middleware of this embodiment adopts the "publish-subscribe" mode for data interaction to realize asynchronous communication between the video tracking cluster and the pedestrian re-identification cluster. In this mode, there is no direct contact between the publisher and the subscriber, but data interaction is carried out through topics. Specifically, two topics are maintained in the message middleware, namely the pedestrian feature data and the pedestrian re-identification result topic. The edge devices in the video tracking cluster will continue to "publish" the video frame data with header information to the pedestrian feature data topic of the message middleware, and the pedestrian re-identification cluster will request the message middleware to "subscribe" to the pedestrian feature data. In addition, the devices in the pedestrian re-identification cluster will splice the header information and the pedestrian re-identification results, and "publish" them to the pedestrian re-identification result topic in the message middleware, so that the devices in the video tracking cluster will continue to request the message middleware to "subscribe" to the pedestrian re-identification results with specific header information.

[0080] Specifically, the pedestrian re-identification cluster of this embodiment is composed of multiple edge devices with different computing capabilities. These edge devices usually deploy pedestrian re-identification services, and "subscribe" to pedestrian feature data in the message middleware to achieve similarity measurement and pedestrian matching of pedestrian features. Usually, the pedestrian re-identification task first needs to calculate the similarity between the pedestrian to be identified and the pedestrian features in the database. Then, sort by similarity and output a list of pedestrian matching results. The pedestrian re-identification service design of this embodiment can be implemented using different technologies according to different application scenarios. For example, Euclidean distance is used for similarity measurement; Locality Sensitive Hashing technology is used for pedestrian matching and retrieval.

[0081] In practical applications, in a complete edge collaborative reasoning interaction process, the two edge device clusters only need to focus on a single task, namely pedestrian feature extraction or pedestrian re-identification. Through the task forwarding strategy of message middleware load balancing, the two clusters can efficiently exchange data and collaborate to complete complex pedestrian re-identification tasks. In order to verify the effectiveness of this method, four edge devices are used to form a video tracking cluster, and two edge devices are used to form a pedestrian re-identification cluster. The experiment simulates the high load situation that may be encountered in practical applications, allowing each device in the video tracking cluster to send 1000 pedestrian feature recognition tasks to the pedestrian re-identification cluster with high concurrency and continuous, totaling 4000 times. The experimental results show that in a high-concurrency test environment, the two devices in the pedestrian re-identification cluster processed about 2000 tasks respectively, effectively achieving load balancing within the cluster. At the same time, the average response delay is only 33 milliseconds, which can meet the real-time requirements.

[0082] In summary, this embodiment fully utilizes the computing resources of the edge device cluster by achieving efficient edge collaborative reasoning while maintaining high-precision ReID performance. With the efficient data interaction capability of the message middleware, functional modules such as pedestrian detection, feature extraction, target tracking, and visualization are integrated.

[0083] Furthermore, this embodiment adopts a load-balanced task forwarding strategy to send the tasks (video frames) to be processed to the idle edge devices for processing. By dynamically distributing the video frames in the entire cluster for parallel processing, the computing resources of the edge devices can be fully utilized, and large-scale data flows can be effectively handled to prevent traffic peaks from affecting system performance.

[0084] Furthermore, this embodiment designs a reasoning collaborative interaction method based on message middleware to realize data transmission between edge device clusters, so that computing tasks can be executed efficiently and pedestrian re-identification results can be returned to edge devices in an orderly and accurate manner. The reasoning delay can be reduced through the collaboration between edge devices to meet the real-time requirements.

[0085] Furthermore, the video surveillance data of this embodiment is only transmitted between edge devices without being sent externally, which effectively avoids the risk of privacy leakage.

[0086] Furthermore, the message middleware based on the "publish-subscribe" mode in this embodiment helps to reduce the coupling between system modules and allows the two clusters to flexibly add or reduce edge devices, thereby improving the scalability and reliability of the entire system.

[0087] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0088] The above-mentioned embodiments are only preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantial changes and substitutions made by technicians in this field on the basis of the present invention shall fall within the scope of protection required by the present invention.

Claims

1. An edge collaborative reasoning interaction method for pedestrian re-identification scenarios, characterized in that: The following steps are involved: The video tracking cluster attaches the video frame data obtained by feature extraction, i.e., pedestrian features, to the header information that uniquely identifies the sending device to form a pedestrian feature data packet with the header information, and sends the pedestrian feature data packet to the message middleware; The message middleware receives and caches the pedestrian feature data packet, and adopts a load-balanced task forwarding strategy according to the real-time status of each edge device in the pedestrian re-identification cluster, and preferentially allocates the pedestrian feature data packet to the edge device in an idle state for processing; The edge device in the pedestrian re-identification cluster performs pedestrian re-identification processing on the pedestrian feature data packet, generates a pedestrian re-identification result, and packages the result into a data packet, i.e., pedestrian ReID data, and attaches the header information to form a pedestrian ReID data packet with header information, and sends the pedestrian ReID data packet back to the message queue corresponding to the message middleware; The edge devices in the video tracking cluster continuously monitor the message queue of pedestrian re-identification results in the message middleware, identify and read the corresponding pedestrian ReID data packet with header information according to the header information, obtain the pedestrian re-identification results, and complete the interactive process of pedestrian re-identification.

2. The method according to claim 1, characterized in that: The video tracking cluster is composed of a plurality of edge devices with different computing capabilities, each of which establishes a data link with at least one smart camera to receive and process video streams from the smart camera in real time; Each edge device of the video tracking cluster is configured to analyze the received video stream frame by frame to extract pedestrian features, and attach the extracted pedestrian feature data to header information that uniquely identifies the edge device to form a pedestrian re-identification task data packet to be completed; Wherein, while sending the pedestrian feature data to the message middleware, the edge device of the video tracking cluster continuously polls the pedestrian re-identification result message queue in the message middleware to identify and receive the pedestrian re-identification result corresponding to the pedestrian feature data sent by it according to the header information; Among them, each edge device of the video tracking cluster also synchronizes the received pedestrian re-identification results with the original video frames, and displays the recognition results in the monitoring video, thereby realizing real-time tracking and re-identification of pedestrians.

3. The method according to claim 2, characterized in that: Each edge device in the video tracking cluster is configured with a proxy based on the "producer-consumer" model to implement the sending of pedestrian re-identification tasks and the receiving of recognition results; Before sending the video frame data as a person re-identification task to the message middleware, the proxy first adds a header information in front of the video frame data, and the header information at least contains a unique identifier of the corresponding edge device to ensure that the subsequent person re-identification result can be accurately returned to the original edge device based on the identifier; After the task is issued, the agent starts a monitoring process, which continuously monitors the pedestrian re-identification result message queue in the message middleware, and identifies and receives the pedestrian re-identification result corresponding to the pedestrian feature data sent locally by comparing the header information of the pedestrian ReID data packet in the message queue with the unique identifier of the local device; After receiving the pedestrian re-identification result, the agent passes it to the edge device for processing, including synchronous display with the original video frame or for further analysis and decision-making, thereby realizing closed-loop management of the pedestrian re-identification task.

4. The method according to claim 1, characterized in that: The pedestrian re-identification cluster is composed of multiple edge devices with different computing capabilities, each edge device is configured to provide pedestrian re-identification services; After receiving the pedestrian feature data forwarded by the message middleware, each edge device in the pedestrian re-identification cluster uses the locally stored pedestrian feature library or the pedestrian feature information obtained in real time to perform similarity matching to identify the pedestrian identity or trajectory that matches the received pedestrian feature data; After obtaining the pedestrian re-identification result, the edge device in the pedestrian re-identification cluster attaches the same header information as the original pedestrian feature data to the result, and the header information at least contains the unique identifier of the original sender edge device.

5. The method according to claim 1, characterized in that The pedestrian re-identification process includes: Calculate the similarity between the feature data of the pedestrian to be identified and the features of the pedestrians stored in the database, wherein the similarity is calculated using the Euclidean distance as a similarity metric, and the similarity between the pedestrian to be identified and the pedestrians in the database is evaluated by calculating the Euclidean distance between the feature vectors; According to the calculated similarity, the pedestrians in the database are sorted to generate a pedestrian matching result list arranged from high to low according to the similarity, wherein the list contains the identities or trajectory information of several pedestrians that are most similar to the pedestrian to be identified.

6. The method according to claim 1, characterized in that Also execute: The locality sensitive hashing algorithm is used for pedestrian matching and retrieval. In the process of pedestrian matching and retrieval, the pedestrian feature vector to be matched is first mapped to the corresponding hash bucket through the hash function, and then the hash bucket with the same pedestrian feature vector as the query is searched to find the pedestrian feature vector similar to the query pedestrian feature vector. Among them, the hash function should satisfy the local sensitivity property, that is, for any two pedestrian feature vectors x and y, if the distance d(x, y) between the two is less than or equal to the preset distance threshold d1, then the probability that they are hashed to the same bucket is at least p1; if the distance d(x, y) between the two is greater than or equal to the preset distance threshold d2 (d2>d1), then the probability that they are hashed to the same bucket is at most p2, where p1 and p2 are constants between 0 and 1.

7. The method according to claim 5, characterized in that: The message middleware is a data stream processing platform with a caching mechanism. It adopts the "publish-subscribe" mode to realize asynchronous communication between the video tracking cluster and the pedestrian re-identification cluster. Two topics are defined in the message middleware, namely the pedestrian feature data topic and the pedestrian re-identification result topic. The pedestrian feature data topic is used to send the video frame data to be identified to the idle edge device, and the pedestrian re-identification result topic is used to return the pedestrian re-identification result to the corresponding edge device in the video tracking cluster based on the header information.

8. The method according to claim 7, characterized in that: The edge devices in the video tracking cluster act as publishers and publish the extracted pedestrian feature data with the header information of the device's unique identifier to the corresponding topic. The edge devices in the person re-identification cluster act as subscribers and subscribe to one or more topics of interest to receive and process the pedestrian feature data from these topics and perform the person re-identification task. In the "publish-subscribe" mode, there is no direct communication link between publishers and subscribers. They exchange data asynchronously through message middleware and defined topics, thus achieving a loosely coupled communication mode.

9. The method according to claim 7, characterized in that: Also includes: Maintenance of pedestrian feature data topic: A pedestrian feature data topic is set in the message middleware to receive and store video frame data with header information published by edge devices in the video tracking cluster; the header information at least includes the timestamp of the video frame, the device unique identifier, and possible video frame quality assessment indicators; Publishing operation of the video tracking cluster: The edge devices in the video tracking cluster continuously extract pedestrian features from the surveillance video, and publish the extracted pedestrian features together with the header information to the pedestrian feature data topic of the message middleware for subscription by the pedestrian re-identification cluster.

10. The method according to claim 9, characterized in that Also includes: Subscription request of the pedestrian re-identification cluster: The devices in the pedestrian re-identification cluster send subscription requests to the message middleware to subscribe to the pedestrian feature data topic, so as to receive pedestrian feature data from the video tracking cluster in real time or on demand; Publishing of person re-identification results: After receiving the person feature data, the devices in the person re-identification cluster perform person re-identification processing and concatenate the processing results with the original header information to form complete person re-identification result data, which is then published to the person re-identification result topic in the message middleware; Subscription results of the video tracking cluster: Devices in the video tracking cluster send subscription requests to the message middleware as needed, requesting to subscribe to the pedestrian re-identification results corresponding to specific header information in the pedestrian re-identification result topic, so as to obtain pedestrian re-identification information related to their own surveillance videos.

Images