A cross-camera multi-target tracking system and method based on tracking chains

Abstract

The application relates to a tracking chain-based cross-camera multi-target tracking system and method, which comprises an edge visual perception module, a cross-view fusion multi-target tracking module and a tracking chain module; the edge visual perception module comprises an edge camera, a target detector, a coordinate transformer and a depth estimation network; the cross-view fusion multi-target tracking module adopts a cross-view identity recognition algorithm to judge multiple targets and uses a global multi-target association algorithm to associate all current frame target sets with historical target sets; the tracking chain module comprises an intelligent contract and a consensus algorithm; the intelligent contract receives target storage transaction requests, target update transaction requests, target leaving transaction requests and target deletion transaction requests; and according to different types of transaction requests, transactions are initiated, and the corresponding transactions are packaged and verified by the consensus algorithm. The application uses a blockchain technology to complete a multi-target tracking task in a cross-camera scene, and realizes safe credibility and privacy protection while ensuring efficiency.

Description

Technical Field

[0001] This invention relates to a cross-camera multi-target tracking system and method based on a tracking chain, which uses blockchain technology to achieve multi-target tracking in cross-camera scenarios, and belongs to the field of information technology. Background Technology

[0002] With the rapid development of smart cities, cameras have covered every corner of the city, and the massive amounts of video data they generate have immeasurable value. Cross-camera multi-target tracking technology has become a research hotspot in recent years, with broad application prospects in fields such as autonomous driving and video surveillance. Compared with single-camera multi-target tracking, cross-camera multi-target tracking technology not only requires accurate trajectory correlation between each camera, but also places great emphasis on trajectory fusion.

[0003] Current cross-camera multi-target tracking technologies require the aggregation of raw video data collected from different devices over a network. First, this places stringent demands on hardware, easily leading to bandwidth congestion, high latency, and scalability issues. Second, it faces serious privacy and security problems; the security and robustness of computation are limited by single points of failure, which is intolerable in real-world industrial environments. Third, data leakage may occur during storage, transmission, and sharing, causing a series of serious problems for data owners and administrators. Because video data often carries a large amount of private information, data owners are unwilling to upload raw videos, directly rendering centralized computing unusable. These challenges are the main obstacles encountered by cross-camera multi-target tracking technology in real-world industrial environments. Summary of the Invention

[0004] The technical problem solved by this invention is to overcome the shortcomings of existing technologies and provide a cross-camera multi-target tracking system and method based on a tracking chain. This system uses blockchain technology to complete multi-target tracking tasks in cross-camera scenarios, achieving security, reliability, and privacy protection while ensuring efficiency.

[0005] The present invention adopts the following technical solution: In a first aspect, the present invention provides a cross-camera multi-target tracking system based on a tracking chain, comprising: an edge visual perception module, a cross-view fusion multi-target tracking module, and a tracking chain module, wherein: The edge vision perception module consists of edge cameras, a target detector, a coordinate transformer, and a depth estimation network. The edge cameras are responsible for acquiring raw video images frame by frame. The intrinsic and extrinsic parameters of each edge camera are known, and each edge camera has undergone pixel-level calibration. Each edge camera is considered an edge node, and all edge nodes together form an edge network. The target detector is responsible for extracting the target set of the current frame from the raw video images acquired by each edge camera. The target set of the current frame includes the position information of each target in the raw video image and its detection confidence. Using the position information of each target in the raw video image... The system extracts the original sub-plots of each target from the location information in the image; the depth estimation network is responsible for extracting the depth information of the original video image of the current frame; the coordinate transformer combines the location information of each target in the original video image, the depth information of the original video image of the current frame, and the intrinsic and extrinsic parameters of the edge camera to obtain the three-dimensional coordinates of each target in the real world; the system sends the identifier, three-dimensional coordinates in the real world, detection confidence, original sub-plot, tracking status, intrinsic and extrinsic parameters of the corresponding edge camera, and current frame number of each target in the target set of the current frame to the tracking chain module, and initiates a target storage transaction request; The cross-view fusion multi-target tracking module obtains the current frame target set from the tracking chain module. All targets in the current frame target set are in a "to be tracked" state. For the current frame target set, based on each target's 3D coordinates in the real world, original sub-tiles, and the intrinsic and extrinsic parameters of the corresponding edge camera, a cross-view identity recognition algorithm is used to determine the identity of multiple targets. If multiple targets from different edge cameras are determined to be the same target, a target merging transaction request is initiated to the tracking chain module to obtain the target's cross-view fusion appearance embedding and 3D stereo detection box. After processing the current frame target set, the module obtains the historical target set from the tracking chain module, and based on the corresponding target's cross-view fusion appearance embedding and 3D stereo detection box... The body detection frame uses a global multi-target association algorithm to associate all current frame target sets with historical target sets, resulting in successfully associated target sets and unassociated target sets. The successfully associated target sets are sent to the tracking chain module, and a target update transaction request is initiated. Each target in the unassociated target set is marked as lost. If the number of times a target in the "to be tracked" state is marked as lost exceeds a set first threshold, or the number of times a target in the "left" state is marked as lost exceeds a set second threshold, a target deletion transaction request is initiated to the tracking chain module. If the number of times a target in the "tracking" state is marked as lost exceeds a set third threshold, a target departure transaction request is initiated to the tracking chain module. The tracking chain module, comprising smart contracts and a consensus algorithm, is built on each edge node. The smart contract is responsible for receiving target storage transaction requests initiated by the edge visual perception module, as well as target update, target departure, and target deletion transaction requests initiated by the cross-view fusion multi-target tracking module. Based on the received target set, the smart contract initiates transactions according to different types of transaction requests. The consensus algorithm then packages and verifies the corresponding transactions. After verification, the data is stored on the blockchain so that the cross-view fusion multi-target tracking module can obtain the current frame target set and the historical target set.

[0006] Specifically, in the edge visual perception module, the coordinate transformer is implemented as follows: (1) Based on the position information of the target in the original video image, calculate the pixel center coordinates of the target; (2) Based on the depth information of the current frame and the position information of the target in the original video image, obtain the depth sub-map of the target; the depth sub-map of the target contains both the depth information of the target and the depth information of the background clutter, and the background clutter needs to be processed; according to the pixel center coordinates of the target and the depth sub-map of the target, the depth of the target relative to the edge camera is obtained by weighted summation, and the depth information of the target after removing the background clutter is obtained. (3) Using the pixel center coordinates of the target and the intrinsic parameters of the edge camera, the position coordinates of the target in the edge camera are obtained. Based on the position coordinates of the target in the edge camera and the intrinsic parameters of the edge camera, the depth information of the target after removing background clutter is obtained, and the axial distance between the target and the edge camera in the world coordinate system is calculated. (4) Based on the axial distance between the target and the edge camera in the world coordinate system, the intrinsic and extrinsic parameters of the edge camera, and the pixel center coordinates of the target, the three-dimensional coordinates of each target in the real world are obtained through matrix transformation.

[0007] Specifically, in the cross-view fusion multi-target tracking module, the cross-view identity recognition algorithm is implemented as follows: (1) For each target in the target set of the current frame, the original sub-map of the target is input into a convolutional neural network with attention mechanism to extract the appearance mask of the target. Then the original sub-map of the target and the appearance mask of the target are merged to obtain the feature-enhanced sub-map of the target. (2) Based on the feature enhancement sub-map of the target, the intrinsic and extrinsic parameters of the corresponding edge camera are combined and transformed by matrix multiplication to obtain the transformed target feature enhancement sub-map; (3) Based on all targets in the current frame target set and the corresponding transformed target feature enhanced sub-maps, use a feature extraction network to extract the appearance embedding of each target, and use a clustering algorithm to classify the appearance embedding of the target to obtain the appearance classification of each target in the current frame target set; (4) If multiple targets belong to the same appearance category and the Euclidean distance of the three-dimensional coordinates of each target in the real world is less than or equal to the set fourth threshold, then the multiple targets are determined to be the same target.

[0008] Specifically, in the cross-view fusion multi-target tracking module, the global multi-target association algorithm is implemented as follows: (1) For each target in the historical target set, the Kalman filter is used to predict the three-dimensional detection box of the target to obtain the predicted three-dimensional detection box; (2) For all current frame target sets, sort them according to the detection confidence of each target. If the detection confidence of the target exceeds the set fifth threshold, add the target to the high-scoring target set. If the detection confidence of the target is between the fifth threshold and the sixth threshold, add the target to the low-scoring target set. (3) Based on the high-scoring target set and the historical target set, a first cost matrix is ​​constructed, where each element in the first cost matrix represents the weighted sum of the intersection-union ratio of the three-dimensional detection boxes between the two targets and the similarity of the fused appearance embedding. (4) Based on the first cost matrix, the Hungarian matching algorithm is used to associate the high-scoring target set with the historical target set to obtain the target set that was successfully associated for the first time, the high-scoring target set that failed to be associated for the first time, and the historical target set that failed to be associated for the first time. (5) Based on the set of low-scoring targets and the set of historical targets that failed the first association, a second cost matrix is ​​constructed. Each element in the second cost matrix represents the weighted sum of the intersection-union ratio of the three-dimensional detection boxes between the two targets and the similarity of the fused appearance embedding. (6) Based on the second cost matrix, the Hungarian matching algorithm is used to associate the set of low-scoring targets with the set of historical targets that failed the first association, to obtain the set of targets that succeeded in the second association, the set of low-scoring targets that failed in the second association, and the set of historical targets that failed in the second association, respectively. The set of targets that were successfully associated for the first time and the set of targets that were successfully associated for the second time are collectively referred to as the set of targets that were successfully associated. The set of high-scoring targets that failed to be associated for the first time, the set of low-scoring targets that failed to be associated for the second time, and the set of historical targets that failed to be associated for the second time are collectively referred to as the set of targets that failed to be associated.

[0009] Specifically, in the tracking chain module, the implementation of smart contract-initiated transactions is as follows: (1) If it is a target storage transaction, the target's identity identifier is used as the key, and the target's three-dimensional coordinates in the real world, detection confidence, original sub-plot, tracking status, and the intrinsic and extrinsic parameters of the corresponding edge camera and the current frame number are written into the block and committed. (2) If it is a target merging transaction, the three-dimensional coordinates of multiple identical targets in the real world are projected and transformed to obtain the three-dimensional stereo detection box of the target; in addition, the appearance embeddings of multiple identical targets are merged to obtain the cross-view fusion appearance embedding; a new identity identifier is generated as the key, and the three-dimensional stereo detection box of the target, the cross-view fusion appearance embedding, the detection confidence, the tracking status, and the intrinsic and extrinsic parameters of the corresponding edge camera and the current frame number are written into the new block and submitted; (3) If it is a target update transaction, for each association pair <current frame target, corresponding historical target> in the successfully associated target set, use the 3D stereo detection box of the current frame target and the Kalman filter to update the 3D stereo detection box of the corresponding historical target; based on the cross-view fusion appearance embedding of the current frame target, use feature merging to update the cross-view fusion appearance embedding of the corresponding historical target, and modify the tracking status of the corresponding historical target to tracking; finally, use the identity identifier of the corresponding historical target as the key, and write the updated 3D stereo detection box, the updated cross-view fusion appearance embedding, the detection confidence, the tracking status, the intrinsic and extrinsic parameters of the corresponding edge camera, and the current frame number as the values ​​into the new block and submit; (4) If the target leaves the transaction, the tracking status of the corresponding target is changed to "leaving". Then, the identity identifier of the corresponding target is used as the key, and the three-dimensional detection box, cross-view fusion appearance embedding, detection confidence, updated tracking status, intrinsic and extrinsic parameters of the corresponding edge camera, and current frame number are written into the new block and submitted. (5) If it is a target deletion transaction, the value with the corresponding target's identity identifier as the key will be deleted, this process will be written into a new block and committed.

[0010] Specifically, the consensus algorithm in the tracking chain module is implemented as follows: (1) Each edge node uses its own private key, random seed, reputation value and workload as input to generate a hash value and proof according to a verifiable random function. The random seed is the hash value of the latest block on the blockchain. The random seed is the same for all edge nodes. The reputation value is used to measure the reputation of the edge node. It is the number of times each edge node has successfully packaged data. The workload is the amount of transaction data submitted by each node. (2) Each edge node broadcasts the generated hash value, proof, reputation value and workload to all other edge nodes in the edge network. Other edge nodes receive the hash value, proof, reputation value and workload sent from the edge node and use a verifiable random function to authenticate the edge node. If the edge node is successfully authenticated, the edge node is added to the candidate list. (3) From the candidate list, select the edge node with the smallest hash value as the packaging edge node of the current frame. The packaging edge node is responsible for packaging and submitting the corresponding transactions submitted by other edge nodes. (4) The packaging edge node broadcasts the new block to the edge network. If an edge node is not a packaging edge node, it receives the new block sent by the packaging edge node, performs transaction verification, and if the transaction is valid, broadcasts the verification message to the edge network. (5) If the number of verification messages received by the packaging edge node exceeds the set seventh threshold, it proves that the transactions in the new block have been verified by at least the same number of edge nodes as the seventh threshold. The packaging edge node adds the new block to the blockchain and broadcasts the synchronization message to other nodes in the edge network.

[0011] Secondly, the present invention provides a cross-camera multi-target tracking method based on a tracking chain, which is implemented as follows: (1) The original video images are acquired frame by frame using edge cameras. The intrinsic and extrinsic parameters of each edge camera are known, and each edge camera is calibrated at the pixel level. Each edge camera is regarded as an edge node, and all edge nodes together form an edge network. (2) Extract the target set of the current frame from the original video images collected by each edge camera through the target detector. The target set of the current frame contains the position information of each target in the original video image and the detection confidence. Extract the original sub-plot of each target based on the position information of each target in the original video image. (3) Use a depth estimation network to extract the original depth information of the current frame image; (4) Combining the position information of each target in the original video image, the original depth information of the current frame image, and the intrinsic and extrinsic parameters of the edge camera, the three-dimensional coordinates of each target in the real world are obtained through a coordinate transformer; (5) Send the identity identifier, three-dimensional coordinates in the real world, detection confidence, original sub-plot, tracking status, intrinsic and extrinsic parameters of the corresponding edge camera, and current frame number of each target in the current frame target set to the tracking chain, and initiate a target storage transaction request; (6) Obtain the target set of the current frame from the tracking chain. The tracking status of all targets in the target set of the current frame is pending tracking. For the target set of the current frame, based on the three-dimensional coordinates of each target in the real world, the original sub-plot, the intrinsic and extrinsic parameters of the corresponding edge camera, a cross-view identity recognition algorithm is used to judge multiple targets. If multiple targets collected from different edge cameras are judged to be the same target, a target merging transaction request is initiated to the tracking chain to obtain the cross-view fusion appearance embedding and three-dimensional detection box of the target. (7) After processing the current frame target set, obtain the historical target set from the tracking chain. Based on the cross-view fusion appearance embedding and three-dimensional detection box of the corresponding target, use the global multi-target association algorithm to associate all current frame target sets with the historical target sets to obtain the target set with successful association and the target set with failed association. Send the target set with successful association to the tracking chain and initiate a target update transaction request. Mark each target in the target set with failed association as lost. If the number of times a target in the target state to be tracked is marked as lost exceeds the set first threshold, or the number of times a target in the target state to have left is marked as lost exceeds the set second threshold, then initiate a target deletion transaction request to the tracking chain. If the number of times a target in the target state to be tracked is marked as lost exceeds the set third threshold, then initiate a target leave transaction request to the tracking chain. (8) The tracking chain built on each edge node includes smart contracts and consensus algorithms. The smart contracts are responsible for receiving the target storage transaction requests, target update transaction requests, target leave transaction requests and target deletion transaction requests. The smart contracts initiate transactions according to different types of transaction requests based on the received target set. Then the consensus algorithm packages and verifies the corresponding transactions. After verification, the data is stored on the blockchain for obtaining the target set.

[0012] Thirdly, the present invention also provides an electronic device, including a processor and a memory, wherein: the memory is used to store a computer program; the processor is used to execute the computer program stored in the memory, wherein when the computer program is executed, it implements the system or method described above.

[0013] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the system or method described above.

[0014] The advantages of this invention compared to the prior art are: (1) The present invention provides a cross-camera multi-target tracking system and method based on a tracking chain, which includes an edge visual perception module, a cross-view fusion multi-target tracking module and a tracking chain module. The target information collected by the edge visual perception module and the target information associated by the cross-view fusion multi-target tracking module are stored on the blockchain in the form of transactions, realizing trusted cooperation and data sharing between different nodes in cross-camera scenarios. Compared with the traditional centralized system, it ensures the security and reliability of data and reduces the risk of privacy leakage.

[0015] (2) The present invention designs a coordinate transformer. Based on the target's position information, the depth information of the original video image, and the camera's intrinsic and extrinsic parameters, the coordinate transformer uses matrix transformation and similar triangles to calculate the three-dimensional coordinates of each target in the real world. Compared with the two-dimensional position information of traditional planar tracking, the tracking accuracy is improved.

[0016] (3) This invention designs a cross-view identity recognition algorithm. The cross-view identity recognition algorithm utilizes the original sub-maps of the target, uses a convolutional neural network with an attention mechanism to extract the appearance mask of the target, and combines the camera extrinsic parameters to obtain the target feature-enhanced sub-maps through matrix transformation. Based on the target feature-enhanced sub-maps, a feature extraction network is used to obtain the appearance embedding of the target, and then a clustering algorithm is used to classify the target's appearance. The Euclidean distance of the target in the three-dimensional coordinates of the real world is combined to comprehensively judge the identity of the target, which solves the problem of large differences in the appearance of the target in different view scenes and improves the accuracy of identity recognition.

[0017] (4) The present invention designs a global multi-target association algorithm, which uses the weighted sum of the intersection-union ratio of the three-dimensional detection boxes between two targets and the similarity of the fused appearance embedding to calculate the cost matrix and use the Hungarian algorithm to perform multiple associations, providing higher-dimensional position and appearance information and improving tracking accuracy.

[0018] (5) The present invention initiates transactions through smart contracts. For different transaction types, the target information is processed and stored on the blockchain and shared with other nodes in the network, thus ensuring data transmission efficiency while achieving security, reliability and privacy protection.

[0019] (6) This invention designs a consensus algorithm that uses each node’s private key, random seed, workload and reputation value to generate hash value and proof, ensuring the randomness and reliability of the election results, and ensuring the reliability of transactions through verification messages. It realizes the consistency of transaction decisions among multiple nodes in a distributed scenario, avoids problems caused by single point of failure, and has the characteristics of resisting Byzantine attacks. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This invention provides an overall architecture diagram of a cross-camera multi-target tracking system based on a tracking chain, as shown in an embodiment of the invention. Figure 2 This is a flowchart of the coordinate transformer in an embodiment of the present invention; Figure 3 This is a flowchart of the cross-view identity recognition algorithm in an embodiment of the present invention; Figure 4 This is a flowchart of the global multi-objective association algorithm in an embodiment of the present invention; Figure 5 This is a flowchart illustrating the smart contract transaction initiation process in an embodiment of the present invention; Figure 6 This is a flowchart of the consensus algorithm in an embodiment of the present invention; Figure 7 A flowchart of a cross-camera multi-target tracking method based on a tracking chain is provided for an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0023] like Figure 1 As shown in the figure, a cross-camera multi-target tracking system based on a tracking chain is provided in an embodiment of the present invention, including: an edge visual perception module, a cross-view fusion multi-target tracking module, and a tracking chain module.

[0024] The specific implementation steps for each of the above modules are as follows: 1. Edge visual perception module: It consists of an edge camera, an object detector, a coordinate transformer, and a depth estimation network.

[0025] Each edge camera has been calibrated at the pixel level. The intrinsic parameters of the edge cameras include the lens focal length. Edge camera pixels Length in direction Edge camera pixels Length in direction and the center coordinates of the edge camera The above All units are millimeters. The extrinsic parameters of the edge camera include its real-world installation location. and installation angle ,in The unit is millimeters. The unit is radians. The symbol for a 3D vector is a commonly used notation in linear algebra. Each edge camera is considered an edge node, and all edge nodes together form an edge network.

[0026] This module utilizes an edge camera to perceive the position and depth information of the target from the original video image based on a target detector and a depth estimation network. It then obtains the target's three-dimensional coordinates in the real world through a coordinate transformer, providing higher-precision position coordinates for tracking calculations.

[0027] The implementation process of the above edge vision perception module is as follows: (1) First, the edge camera captures the original video images frame by frame. Then, the raw video images captured by each edge camera are used with an object detector. Extract the target set of the current frame The current frame target set Includes each target Location information in the original video image and detection confidence level ,in The coordinates of the top-left corner of each target. The coordinates of the upper right corner of each target, the All units are pixels. Indicates the first One target, corresponding to .

[0028] Based on the location information of each target in the original video image Extract the original sub-pattern for each target ; (2) Use a depth estimation network to extract the depth information of the original video image of the current frame. ,in In the image The depth of the location, in meters. and These represent the height and width of the image, respectively, both in pixels; (3) Combine the location information of each target in the original video image Depth information of the original video image in the current frame The intrinsic and extrinsic parameters of the edge camera are used to obtain the three-dimensional coordinates of each target in the real world through a coordinate transformer. .

[0029] like Figure 2 As shown, the coordinates of each target in the real world are obtained through the coordinate transformer. The specific implementation steps are as follows: (3-1) According to the objective Location information in the original video image Find the pixel center coordinates of the target. ,in , .

[0030] (3-2) Depth information based on the original video image of the current frame and target Location information in the original video image The depth sub-pattern of the target is obtained according to the following formula. :

[0031] , The depth sub-map of the target contains both the target's depth information and the depth information of background clutter. The background clutter needs to be processed (since the depth sub-map extracts coarse-grained depth information of the target, which includes both background and target depth information, the background depth information needs to be removed). Based on the target's pixel center coordinates and the target's depth sub-map, a weighted summation method is used to obtain the target's depth relative to the edge camera, resulting in the target after removing background clutter. depth information :

[0032] , (4) Using the pixel center coordinates of the target And the internal reference of edge cameras and Obtain the target's position coordinates in the edge camera. ,in Based on the target's position coordinates in the edge camera and edge camera insider information and The target after removing background clutter depth information The target was calculated. Axial distance between the edge camera and the world coordinate system ;

[0033] , (5) Based on the axial distance between the target and the edge camera in the world coordinate system Intrinsic and extrinsic parameters of the edge camera, and pixel center coordinates of the target. The three-dimensional coordinates of each target in the real world are obtained through matrix transformation. The calculation method is as follows:

[0034] , (6) Identify the identity of each target in the detection set. 3D coordinates in the real world Detection confidence Original sub-plots Tracking status The intrinsic and extrinsic parameters of the corresponding edge camera, and the current frame number. The request is sent to the tracking chain module, and a target storage transaction request is initiated. The tracking status... Includes: pending tracking, tracking in progress, departed, and deleted.

[0035] 2. Cross-view fusing multi-target tracking module: This module consists of a cross-view identity recognition algorithm and a global multi-target association algorithm. The cross-view identity recognition algorithm is responsible for identifying targets from different edge camera perspectives, and the global multi-target association algorithm associates the current frame's target set with the historical target set, thereby achieving multi-target tracking across camera scenes. The specific implementation is as follows: (1) Obtain the target set of the current frame from the tracking chain module The current frame target set All targets in the current frame are in a tracking state of "to be tracked"; for the target set in the current frame... Based on each goal Three-dimensional coordinates in the real world Original sub-plots Based on the intrinsic and extrinsic parameters of the corresponding edge cameras, a cross-view identity recognition algorithm is used to identify multiple targets. If multiple targets from different edge cameras are identified as the same target, a target merging transaction request is initiated to the tracking chain module to obtain the cross-view fused appearance embedding of the target. and 3D detection frame ; like Figure 3 As shown, specifically, the above cross-view identity recognition algorithm is implemented using the following steps: (1-1) For each target in the target set of the current frame , target original sub-plot The appearance mask of the target is extracted by feeding it into a convolutional neural network with an attention mechanism. The appearance mask To be consistent with the original sub-plot Boolean matrices of the same size. Then, the original sub-patterns of the target and... Target appearance mask Merge the maps to obtain the feature-enhanced sub-maps of the target. ; (1-2) Target-based feature-enhanced sub-plots By combining the intrinsic and extrinsic parameters of the corresponding edge cameras, matrix multiplication is used to transform the target feature enhanced sub-maps. ;

[0036] , (1-3) Based on all targets in the current frame target set and their corresponding transformed target feature enhancement sub-patterns The appearance embedding of each target is extracted using a feature extraction network. Clustering algorithms are used to classify the appearance embeddings of the targets, resulting in appearance classifications for each target in the current frame's target set. ; (1-4) If the target and Both belong to the same appearance category, and the target and Euclidean distance in three-dimensional coordinates in the real world Less than or equal to the set fourth threshold If the value ranges from 5 to 10, then the multiple targets are determined to be the same target. and target Euclidean distance in three-dimensional coordinates in the real world The calculation is as follows:

[0037] .

[0038] (2) Process the target set of the current frame. Then, obtain the historical target set from the tracking chain module. Cross-view fusion appearance embedding based on corresponding targets and 3D detection frame A global multi-target association algorithm is used to analyze all target sets in the current frame. With historical target set Perform associations to obtain the set of targets that were successfully associated and the set of targets that were not associated. like Figure 4As shown, specifically, the global multi-objective association algorithm described above is implemented using the following steps: (2-1) Targeting the historical target set Each goal The Kalman filter is used to predict the 3D stereo detection box of the target, resulting in the predicted 3D stereo detection box. ; (2-2) For all current frame target sets Sort according to the detection confidence of each target. If the target Detection confidence Exceeding the set fifth threshold (The value is usually 0.7), then the target is added to the high-scoring target set. If the detection confidence of the target is within the fifth threshold and the sixth threshold If the value is between 0.3 and 3, then the target is added to the set of low-scoring targets. ; Assuming there are 12 targets in the current frame's target set, with corresponding detection confidence scores of 0.98, 0.99, 0.89, 0.23, 0.73, 0.92, 0.34, 0.44, 0.45, 0.83, 0.95, and 0.49, then the 7 targets with confidence scores of 0.98, 0.99, 0.89, 0.73, 0.92, 0.83, and 0.95 are added to the high-scoring target set. The four targets with confidence levels of 0.34, 0.44, 0.45, and 0.49 were added to the low-scoring target set. .

[0039] (2-3) Based on the aforementioned high-resolution target set With historical target set Construct the first cost matrix The first cost matrix Each element in the expression represents a weighted sum of the intersection-union ratio (IoU) of the 3D bounding boxes between two targets and the similarity of the fused appearance embedding. The calculation method is as follows: , , The weights are 0.2-0.8. Indicate target and The intersection-union ratio of the three-dimensional detection boxes between them. Indicate target and The similarity between appearance embeddings is fused and calculated using cosine distance.

[0040] , (2-4) Based on the first cost matrix The Hungarian matching algorithm was used to analyze the high-scoring target set. With historical target set Perform associations to obtain the target sets that were successfully associated for the first time. The set of high-scoring targets that failed to be associated for the first time The set of historical targets that failed to be associated for the first time ; Assuming the high-scoring target set contains 7 high-scoring targets and the historical target set contains 6 historical targets, the first cost matrix... As shown in Table 1 below, the element in the first row and first column represents the weighted sum of the intersection-union ratio and the fusion appearance embedding similarity of the three-dimensional detection boxes between historical target 1 and high-scoring target 1, which is 0.21. The larger the value, the greater the probability that the two targets are the same target.

[0041] After executing the Hungarian matching algorithm, the set of successfully associated targets is... For the set of high-scoring goals that failed to be associated for the first time, see [(historical goal 1, high-score goal 4), (historical goal 3, high-score goal 5), (historical goal 4, high-score goal 2), (historical goal 5, high-score goal 6)]. For [High Score Target 1, High Score Target 3, High Score Target 7], the set of historical targets that failed to be associated for the first time. For [Historical Goal 2, Historical Goal 6].

[0042] Table 1 , (2-5) Based on the low-scoring target set Set of historical targets that failed to be associated for the first time Construct the second cost matrix Each element in the second cost matrix represents a weighted sum of the intersection-union ratio of the three-dimensional detection boxes between two targets and the similarity of the fused appearance embedding.

[0043] , (2-6) Based on the second cost matrix The Hungarian matching algorithm was used to analyze the low-scoring target set. Set of historical targets that failed to be associated for the first time By performing association, we can obtain the target sets that were successfully associated twice. Low-scoring target set where secondary association fails The set of historical targets that failed in the second-order association ; Assuming there are 4 low-scoring targets in the low-scoring target set, based on the assumptions in (2-4), the set of historical targets that failed to be associated for the first time... For [Historical Objective 2, Historical Objective 6], the second cost matrix As shown in Table 2 below, the definitions of the elements in Table 2 are the same as those in Table 1.

[0044] After executing the Hungarian matching algorithm, the target set that was successfully associated twice. This is the set of low-scoring objectives (historical objective 2, low-scoring objective 4) that failed to achieve a secondary association. This is a set of historical targets that failed to achieve secondary association, including [low-scoring target 1, low-scoring target 2, and low-scoring target 3]. For [Historical Goal 6].

[0045] Table 2 , (2-7) The set of targets that were successfully associated for the first time The target set of successful secondary association This is collectively referred to as the set of targets that were successfully associated, and the set of high-scoring targets that failed to be associated on the first attempt. Low-scoring target set where secondary association fails The set of historical targets that failed in the second-order association This is collectively referred to as the target set where association failed.

[0046] Based on the assumptions in (2-4) and (2-6), the set of targets that were successfully associated is [(historical target 1, high-scoring target 4), (historical target 3, high-scoring target 5), (historical target 4, high-scoring target 2), (historical target 5, high-scoring target 6), (historical target 2, low-scoring target 4)], and the set of targets that were not successfully associated is [high-scoring target 1, high-scoring target 3, high-scoring target 7, low-scoring target 1, low-scoring target 2, low-scoring target 3, historical target 6].

[0047] (2-8) Send the successfully associated target set to the tracking chain module and initiate a target update transaction request; mark each target in the failed associated target set as lost. If the number of times a target in the target status of pending tracking is marked as lost exceeds a set first threshold... (Value range is 30-60), or the number of times a target whose status is "left" has been marked as lost exceeds the set second threshold. (Value range is 60-120), then a target deletion transaction request is initiated to the tracking chain module; if the number of times the target in the tracking state has been marked as lost exceeds the set third threshold. (If the value ranges from 30 to 60), then a target departure transaction request is sent to the tracking chain module.

[0048] 3. Tracking Chain Module: This module comprises smart contracts and a consensus algorithm, built upon each edge node. The smart contracts receive target storage transaction requests initiated by the edge visual perception module, as well as target update, target departure, and target deletion transaction requests initiated by the cross-view fusion multi-target tracking module. Based on the received target set, the smart contracts initiate transactions according to different types of requests. The consensus algorithm then packages and verifies the corresponding transactions. After verification, the data is stored on the blockchain, allowing the cross-view fusion multi-target tracking module to obtain the current frame target set and historical target sets. This module is responsible for storing target information on the blockchain and sharing it with other nodes in the network, ensuring data transmission efficiency while maintaining security, reliability, and privacy.

[0049] like Figure 5 As shown, in particular, the specific implementation method of the smart contract initiating the transaction is as follows, and the transaction is divided into 5 categories: target storage transaction, target merging transaction, target update transaction, target leaving transaction and target deletion transaction.

[0050] (1) If it is a target storage transaction, the target's identity identifier As a key, the target's three-dimensional coordinates in the real world Detection confidence Original sub-plots Tracking status And the intrinsic and extrinsic parameters of the corresponding edge camera, and the current frame number. As a value, it is written into the block and committed; (2) If it is a target merger transaction, the three-dimensional coordinates of multiple identical targets in the real world are projected and transformed to obtain the three-dimensional detection box of the target. Furthermore, embedding the appearance of multiple identical targets Feature merging is performed to obtain cross-view fused appearance embedding. Generate new identity identifiers As a key, the three-dimensional detection bounding box of the target is... Cross-view fusion appearance embedding Detection confidence Tracking status And the intrinsic and extrinsic parameters of the corresponding edge camera, and the current frame number. As a value, write it into the new block and commit; (3) If it is a target update transaction, for each associated pair in the set of successfully associated targets <current frame target , corresponding historical goals Using the 3D stereo detection bounding box of the target in the current frame, the corresponding 3D stereo detection bounding box of the historical target is updated using a Kalman filter. Cross-view fusion appearance embedding based on the target in the current frame The cross-view fusion appearance embedding of the corresponding historical target is updated by merging features. And track the corresponding historical target status. Change to "Tracking in progress"; finally, change the identifier of the corresponding historical target. As a key, the updated 3D detection box corresponding to the historical target is used. Updated cross-view fusion appearance embedding Detection confidence Tracking status The intrinsic and extrinsic parameters of the corresponding edge camera, and the current frame number. As a value, write it into the new block and commit; (4) If the target leaves the transaction, change the tracking status of the corresponding target to "left", and then change the identity identifier of the corresponding target. As a key, the three-dimensional detection box of the corresponding target is... Cross-view fusion appearance embedding Detection confidence Updated tracking status The intrinsic and extrinsic parameters of the corresponding edge camera, and the current frame number. As a value, write it into the new block and commit; (5) If it is a target deletion transaction, it will be based on the identity identifier of the corresponding target. Delete the value for the key, write this process to a new block, and commit.

[0051] like Figure 6 As shown, in particular, the specific implementation of the above consensus algorithm is as follows: (1) Traverse all edge nodes, each edge node Each using its own private key Random seed Reputation value and workload As input, based on a verifiable random function Generate hash value and proof , wherein the random seed It is the hash value of the latest block on the blockchain; for all edge nodes, the random seed is... They are the same; the reputation value The workload is used to measure the credibility of edge nodes; it is the number of times each edge node successfully packages data. The amount of transaction data submitted for each edge node; (2) The hash value generated by each edge node ,prove Reputation value and workload The hash is broadcast to all other edge nodes in the edge network, and the other edge nodes receive the hash value sent from that edge node. ,prove Reputation value and workload Use a verifiable random function Authenticate the edge node; if the edge node is successfully authenticated, add the edge node to the candidate list. (3) From the candidate list, select the edge node with the smallest hash value as the packaging edge node of the current frame. The packaging edge node is responsible for packaging and submitting the corresponding transactions submitted by other edge nodes. (4) The packaging edge node broadcasts the new block to the edge network. If an edge node is not a packaging edge node, it receives the new block sent by the packaging edge node, performs transaction verification, and if the transaction is valid, broadcasts the verification message to the edge network. (5) If the number of verification messages received by the packaging edge node exceeds the set seventh threshold (The value range is not less than the number of edge nodes minus 1 divided by 3), which proves that the transactions in the new block have reached at least the number of the seventh threshold. Upon verification by the same edge node, the packaging edge node adds the new block to the blockchain and broadcasts a synchronization message to other nodes in the edge network.

[0052] like Figure 7 As shown, another embodiment of the present invention also provides a cross-camera multi-target tracking method based on a tracking chain, specifically implemented as follows: (1) Acquire raw video images frame by frame using an edge camera. The intrinsic and extrinsic parameters of each edge camera are known, and each edge camera has been calibrated at the pixel level; each edge camera is regarded as an edge node, and all edge nodes together form an edge network. (2) Extract the target set of the current frame from the original video images captured by each edge camera using the target detector. The current frame target set contains the location information of each target in the original video image. and detection confidence level The original sub-patterns of each target are extracted based on their location information in the original video image. ; (3) Use a depth estimation network to extract the depth information of the original video image of the current frame. ; (4) Combine the location information of each target in the original video image Depth information of the original video image in the current frame The intrinsic and extrinsic parameters of the edge camera are used to obtain the three-dimensional coordinates of each target in the real world through a coordinate transformer. ; (5) Identify the identity of each target in the current frame target set. 3D coordinates in the real world Detection confidence Original sub-plots Tracking status The intrinsic and extrinsic parameters of the corresponding edge camera, and the current frame number. Send to the tracking chain and initiate a target storage transaction request; (6) Obtain the target set of the current frame from the tracking chain. The tracking status of all targets in the current frame target set is "to be tracked". For the current frame target set, the tracking status is based on the three-dimensional coordinates of each target in the real world. Original sub-plots Based on the intrinsic and extrinsic parameters of the corresponding edge cameras, a cross-view identity recognition algorithm is used to identify multiple targets. If multiple targets from different edge cameras are identified as the same target, a target merging transaction request is initiated to the tracking chain to obtain the cross-view fused appearance embedding of the target. and 3D detection frame ; (7) After processing the current frame target set Retrieve historical target set from the root chain Cross-view fusion appearance embedding based on corresponding targets and 3D detection frame A global multi-target association algorithm is used to associate all current frame target sets with historical target sets, resulting in successfully associated target sets and unassociated target sets. The successfully associated target sets are sent to the tracking chain, and a target update transaction request is initiated. Each target in the unassociated target set is marked as lost. If the number of times a target in the "to be tracked" state is marked as lost exceeds a set first threshold, or the number of times a target in the "left" state is marked as lost exceeds a set second threshold, a target deletion transaction request is initiated to the tracking chain. If the number of times a target in the "tracking" state is marked as lost exceeds a set third threshold, a target leave transaction request is initiated to the tracking chain. (8) The tracking chain built on each edge node includes smart contracts and consensus algorithms. The smart contracts are responsible for receiving the target storage transaction requests, target update transaction requests, target leave transaction requests and target deletion transaction requests. The smart contracts initiate transactions according to different types of transaction requests based on the received target set. Then the consensus algorithm packages and verifies the corresponding transactions. After verification, the data is stored on the blockchain for obtaining the target set.

[0053] The specific implementation process of each part in the above system is similar to the method process provided in the embodiments of the present invention.

[0054] Based on the same inventive concept, another embodiment of the present invention provides an electronic device (including a computer, server, smartphone, etc.) comprising a memory and a processor, wherein the memory stores a computer program configured to be executed by the processor, and the computer program includes steps and systems for performing the methods and systems provided above in the embodiments of the present invention.

[0055] Based on the same inventive concept, another embodiment of the present invention also provides a computer-readable storage medium (such as ROM / RAM, disk, optical disk), which stores a computer program. When the computer program is executed by a computer, it implements the various steps and systems of the methods provided in the above embodiments of the present invention.

[0056] The above embodiments are provided merely for the purpose of describing the present invention and are not intended to limit the scope of the invention. The scope of the invention is defined by the appended claims. Various equivalent substitutions and modifications made without departing from the spirit and principles of the invention should be covered within the scope of the invention.

Claims

1. A cross-camera multi-target tracking system based on a tracking chain, characterized in that, The system includes: an edge visual perception module, a cross-view fusion multi-target tracking module, and a tracking chain module, wherein: The edge vision perception module consists of edge cameras, a target detector, a coordinate transformer, and a depth estimation network. The edge cameras are responsible for acquiring raw video images frame by frame. The intrinsic and extrinsic parameters of each edge camera are known, and each edge camera has undergone pixel-level calibration. Each edge camera is considered an edge node, and all edge nodes together form an edge network. The target detector is responsible for extracting the target set of the current frame from the raw video images acquired by each edge camera. The target set of the current frame includes the position information of each target in the raw video image and its detection confidence. Using the position information of each target in the raw video image... The system extracts the original sub-plots of each target from the location information in the image; the depth estimation network is responsible for extracting the depth information of the original video image of the current frame; the coordinate transformer combines the location information of each target in the original video image, the depth information of the original video image of the current frame, and the intrinsic and extrinsic parameters of the edge camera to obtain the three-dimensional coordinates of each target in the real world; the system sends the identifier, three-dimensional coordinates in the real world, detection confidence, original sub-plot, tracking status, intrinsic and extrinsic parameters of the corresponding edge camera, and current frame number of each target in the target set of the current frame to the tracking chain module, and initiates a target storage transaction request; The cross-view fusion multi-target tracking module obtains the current frame target set from the tracking chain module. All targets in the current frame target set are in a "to be tracked" state. For the current frame target set, based on each target's 3D coordinates in the real world, original sub-tiles, and the intrinsic and extrinsic parameters of the corresponding edge camera, a cross-view identity recognition algorithm is used to determine the identity of multiple targets. If multiple targets from different edge cameras are determined to be the same target, a target merging transaction request is initiated to the tracking chain module to obtain the target's cross-view fusion appearance embedding and 3D stereo detection box. After processing the current frame target set, the module obtains the historical target set from the tracking chain module, and based on the corresponding target's cross-view fusion appearance embedding and 3D stereo detection box... The body detection frame uses a global multi-target association algorithm to associate all current frame target sets with historical target sets, resulting in successfully associated target sets and unassociated target sets. The successfully associated target sets are sent to the tracking chain module, and a target update transaction request is initiated. Each target in the unassociated target set is marked as lost. If the number of times a target in the "to be tracked" state is marked as lost exceeds a set first threshold, or the number of times a target in the "left" state is marked as lost exceeds a set second threshold, a target deletion transaction request is initiated to the tracking chain module. If the number of times a target in the "tracking" state is marked as lost exceeds a set third threshold, a target departure transaction request is initiated to the tracking chain module. The tracking chain module, comprising smart contracts and a consensus algorithm, is built on each edge node. The smart contract is responsible for receiving target storage transaction requests initiated by the edge visual perception module, as well as target update, target departure, and target deletion transaction requests initiated by the cross-view fusion multi-target tracking module. Based on the received target set, the smart contract initiates transactions according to different types of transaction requests. The consensus algorithm then packages and verifies the corresponding transactions. After verification, the data is stored on the blockchain so that the cross-view fusion multi-target tracking module can obtain the current frame target set and the historical target set.

2. The cross-camera multi-target tracking system based on a tracking chain according to claim 1, characterized in that: The coordinate transformer in the edge vision perception module is implemented as follows: (1) Based on the position information of the target in the original video image, calculate the pixel center coordinates of the target; (2) Based on the depth information of the current frame and the position information of the target in the original video image, obtain the depth sub-map of the target; the depth sub-map of the target contains both the depth information of the target and the depth information of the background clutter, and the background clutter needs to be processed; according to the pixel center coordinates of the target and the depth sub-map of the target, the depth of the target relative to the edge camera is obtained by weighted summation, and the depth information of the target after removing the background clutter is obtained. (3) Using the pixel center coordinates of the target and the intrinsic parameters of the edge camera, the position coordinates of the target in the edge camera are obtained. Based on the position coordinates of the target in the edge camera and the intrinsic parameters of the edge camera, the depth information of the target after removing background clutter is obtained, and the axial distance between the target and the edge camera in the world coordinate system is calculated. (4) Based on the axial distance between the target and the edge camera in the world coordinate system, the intrinsic and extrinsic parameters of the edge camera, and the pixel center coordinates of the target, the three-dimensional coordinates of each target in the real world are obtained through matrix transformation.

3. A cross-camera multi-target tracking system based on a tracking chain according to claim 1, characterized in that: The cross-view fusion multi-target tracking module implements the cross-view identity recognition algorithm as follows: (1) For each target in the target set of the current frame, the original sub-map of the target is input into a convolutional neural network with attention mechanism to extract the appearance mask of the target. Then the original sub-map of the target and the appearance mask of the target are merged to obtain the feature-enhanced sub-map of the target. (2) Based on the feature enhancement sub-map of the target, the intrinsic and extrinsic parameters of the corresponding edge camera are combined and transformed by matrix multiplication to obtain the transformed target feature enhancement sub-map; (3) Based on all targets in the current frame target set and the corresponding transformed target feature enhanced sub-maps, use a feature extraction network to extract the appearance embedding of each target, and use a clustering algorithm to classify the appearance embedding of the target to obtain the appearance classification of each target in the current frame target set; (4) If multiple targets belong to the same appearance category and the Euclidean distance of the three-dimensional coordinates of each target in the real world is less than or equal to the set fourth threshold, then the multiple targets are determined to be the same target.

4. A cross-camera multi-target tracking system based on a tracking chain according to claim 1, characterized in that: The implementation of the global multi-target association algorithm in the cross-view fusion multi-target tracking module is as follows: (1) For each target in the historical target set, the Kalman filter is used to predict the three-dimensional detection box of the target to obtain the predicted three-dimensional detection box; (2) For all current frame target sets, sort them according to the detection confidence of each target. If the detection confidence of the target exceeds the set fifth threshold, add the target to the high-scoring target set. If the detection confidence of the target is between the fifth threshold and the sixth threshold, add the target to the low-scoring target set. (3) Based on the high-scoring target set and the historical target set, a first cost matrix is ​​constructed, where each element in the first cost matrix represents the weighted sum of the intersection-union ratio of the three-dimensional detection boxes between the two targets and the similarity of the fused appearance embedding. (4) Based on the first cost matrix, the Hungarian matching algorithm is used to associate the high-scoring target set with the historical target set to obtain the target set that was successfully associated for the first time, the high-scoring target set that failed to be associated for the first time, and the historical target set that failed to be associated for the first time. (5) Based on the set of low-scoring targets and the set of historical targets that failed the first association, a second cost matrix is ​​constructed. Each element in the second cost matrix represents the weighted sum of the intersection-union ratio of the three-dimensional detection boxes between the two targets and the similarity of the fused appearance embedding. (6) Based on the second cost matrix, the Hungarian matching algorithm is used to associate the set of low-scoring targets with the set of historical targets that failed the first association, to obtain the set of targets that succeeded in the second association, the set of low-scoring targets that failed in the second association, and the set of historical targets that failed in the second association, respectively. The set of targets that were successfully associated for the first time and the set of targets that were successfully associated for the second time are collectively referred to as the set of targets that were successfully associated. The set of high-scoring targets that failed to be associated for the first time, the set of low-scoring targets that failed to be associated for the second time, and the set of historical targets that failed to be associated for the second time are collectively referred to as the set of targets that failed to be associated.

5. A cross-camera multi-target tracking system based on a tracking chain according to claim 1, characterized in that: In the tracking chain module, the smart contract initiates a transaction as follows: (1) If it is a target storage transaction, the target's identity identifier is used as the key, and the target's three-dimensional coordinates in the real world, detection confidence, original sub-plot, tracking status, and the intrinsic and extrinsic parameters of the corresponding edge camera and the current frame number are written into the block and committed. (2) If it is a target merging transaction, the three-dimensional coordinates of multiple identical targets in the real world are projected and transformed to obtain the three-dimensional stereo detection box of the target; in addition, the appearance embeddings of multiple identical targets are merged to obtain cross-view fusion appearance embedding. Generate a new identity identifier as the key, and write the target's 3D stereo detection box, cross-view fusion appearance embedding, detection confidence, tracking status, and the intrinsic and extrinsic parameters of the corresponding edge camera, as well as the current frame number, as values ​​into the new block and submit it; (3) If it is a target update transaction, for each association pair <current frame target, corresponding historical target> in the successfully associated target set, use the 3D stereo detection box of the current frame target and the Kalman filter to update the 3D stereo detection box of the corresponding historical target; based on the cross-view fusion appearance embedding of the current frame target, use feature merging to update the cross-view fusion appearance embedding of the corresponding historical target, and modify the tracking status of the corresponding historical target to tracking; finally, use the identity identifier of the corresponding historical target as the key, and write the updated 3D stereo detection box, the updated cross-view fusion appearance embedding, the detection confidence, the tracking status, the intrinsic and extrinsic parameters of the corresponding edge camera, and the current frame number as the values ​​into the new block and submit; (4) If the target leaves the transaction, the tracking status of the corresponding target is changed to "leaving". Then, the identity identifier of the corresponding target is used as the key, and the three-dimensional detection box, cross-view fusion appearance embedding, detection confidence, updated tracking status, intrinsic and extrinsic parameters of the corresponding edge camera, and current frame number are written into the new block and submitted. (5) If it is a target deletion transaction, the value with the corresponding target's identity identifier as the key will be deleted, this process will be written into a new block and committed.

6. A cross-camera multi-target tracking system based on a tracking chain according to claim 1, characterized in that: The consensus algorithm in the tracking chain module is implemented as follows: (1) Each edge node uses its own private key, random seed, reputation value and workload as input to generate a hash value and proof according to a verifiable random function. The random seed is the hash value of the latest block on the blockchain. The random seed is the same for all edge nodes. The reputation value is used to measure the reputation of the edge node. It is the number of times each edge node has successfully packaged data. The workload is the amount of transaction data submitted by each node. (2) Each edge node broadcasts the generated hash value, proof, reputation value and workload to all other edge nodes in the edge network. Other edge nodes receive the hash value, proof, reputation value and workload sent from the edge node and use a verifiable random function to authenticate the edge node. If the edge node is successfully authenticated, the edge node is added to the candidate list. (3) From the candidate list, select the edge node with the smallest hash value as the packaging edge node of the current frame. The packaging edge node is responsible for packaging and submitting the corresponding transactions submitted by other edge nodes. (4) The packaging edge node broadcasts the new block to the edge network. If an edge node is not a packaging edge node, it receives the new block sent by the packaging edge node, performs transaction verification, and if the transaction is valid, broadcasts the verification message to the edge network. (5) If the number of verification messages received by the packaging edge node exceeds the set seventh threshold, it proves that the transactions in the new block have been verified by at least the same number of edge nodes as the seventh threshold. The packaging edge node adds the new block to the blockchain and broadcasts the synchronization message to other nodes in the edge network.

7. A cross-camera multi-target tracking method based on a tracking chain, characterized in that... The implementation is as follows: (1) The original video images were acquired frame by frame using edge cameras. The intrinsic and extrinsic parameters of each edge camera were known, and each edge camera was calibrated at the pixel level. Each edge camera is considered an edge node, and all edge nodes together form an edge network; (2) Extract the target set of the current frame from the original video images collected by each edge camera through the target detector. The target set of the current frame contains the position information of each target in the original video image and the detection confidence. Extract the original sub-plot of each target based on the position information of each target in the original video image. (3) Use a depth estimation network to extract the depth information of the original video image of the current frame; (4) Combining the position information of each target in the original video image, the depth information of the original video image of the current frame, and the intrinsic and extrinsic parameters of the edge camera, the three-dimensional coordinates of each target in the real world are obtained through a coordinate transformer; (5) Send the identity identifier, three-dimensional coordinates in the real world, detection confidence, original sub-plot, tracking status, intrinsic and extrinsic parameters of the corresponding edge camera, and current frame number of each target in the current frame target set to the tracking chain, and initiate a target storage transaction request; (6) Obtain the target set of the current frame from the tracking chain. The tracking status of all targets in the target set of the current frame is pending tracking. For the target set of the current frame, based on the three-dimensional coordinates of each target in the real world, the original sub-plot, the intrinsic and extrinsic parameters of the corresponding edge camera, a cross-view identity recognition algorithm is used to judge multiple targets. If multiple targets collected from different edge cameras are judged to be the same target, a target merging transaction request is initiated to the tracking chain to obtain the cross-view fusion appearance embedding and three-dimensional detection box of the target. (7) After processing the current frame target set, obtain the historical target set from the tracking chain. Based on the cross-view fusion appearance embedding and three-dimensional detection box of the corresponding target, use the global multi-target association algorithm to associate all current frame target sets with the historical target sets to obtain the target set with successful association and the target set with unsuccessful association. Send the target set with successful association to the tracking chain and initiate a target update transaction request. Mark each target in the set of failed targets as lost. If the number of times a target in the target status of pending tracking is marked as lost exceeds a set first threshold, or the number of times a target in the target status of having left is marked as lost exceeds a set second threshold, then initiate a target deletion transaction request to the tracking chain. If the number of times a target in the tracking state has been marked as lost exceeds the set third threshold, a target departure transaction request will be sent to the tracking chain. (8) The tracking chain built on each edge node includes smart contracts and consensus algorithms. The smart contracts are responsible for receiving the target storage transaction requests, target update transaction requests, target leave transaction requests and target deletion transaction requests. The smart contracts initiate transactions according to different types of transaction requests based on the received target set. Then the consensus algorithm packages and verifies the corresponding transactions. After verification, the data is stored on the blockchain for obtaining the target set.

8. An electronic device, characterized in that, It includes a processor and a memory, wherein: the memory is used to store computer programs; the processor is used to execute the computer programs stored in the memory, wherein when the computer programs are executed, they implement the system described in any one of claims 1-6, or the method described in claim 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-6, or the system described in claim 7.

Images