A cross-camera trajectory association method, device and electronic equipment

Abstract

This application provides a method, apparatus, and electronic device for cross-camera trajectory association, relating to the field of computer vision technology. The method involves acquiring the trajectories of targets captured by multiple cameras, as target trajectories; acquiring the similarity between image features of every two trajectories within each target trajectory, as an association similarity; associating the two trajectories with the highest association similarity to obtain a trajectory set containing these two trajectories as a new target trajectory; calculating the association similarity between the image features of the new target trajectory and other target trajectories, and then returning to execute the step of associating the two trajectories with the highest association similarity to obtain a trajectory set containing these two trajectories as a new target trajectory, until a first number of target trajectories are obtained. Based on this, it is possible to effectively associate various target trajectories.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method, apparatus and electronic device for cross-camera trajectory association. Background Technology

[0002] To facilitate the management of targets (such as animals, vehicles, etc.), cameras can be used to capture the target's activity scene to determine its trajectory. However, the field of view of a single camera is limited, while the target's activity scene may be large, making it difficult to obtain the target's complete trajectory with a single camera.

[0003] Therefore, multiple cameras can be deployed to capture images of the target in each sub-region of the activity scene, obtain the target's trajectory in each camera, and obtain the target's complete trajectory in the entire activity scene based on the trajectory in each camera.

[0004] However, multiple targets may exist simultaneously in the sub-region of each camera. How to effectively correlate the trajectories of the same target in different cameras has become an urgent problem to be solved. Summary of the Invention

[0005] The purpose of this application is to provide a method, apparatus, and electronic device for cross-camera trajectory association, so as to effectively associate various target trajectories. The specific technical solution is as follows:

[0006] Firstly, in order to achieve the above objectives, embodiments of this application disclose a cross-camera trajectory association method, the method comprising:

[0007] Acquire the trajectory of the target captured by multiple cameras, and use it as the target trajectory;

[0008] The similarity between the image features of every two trajectories in each target trajectory is obtained as the correlation similarity.

[0009] The two trajectories with the highest correlation similarity are associated to obtain a new set of trajectories containing the two trajectories.

[0010] Calculate the correlation similarity between the image features of the new target trajectory and other target trajectories, and return to perform the step of associating the two trajectories with the highest correlation similarity to obtain a trajectory set containing the two trajectories as the new target trajectory, until a first number of target trajectories are obtained.

[0011] Optionally, obtaining the similarity between the image features of every two trajectories in each target trajectory as the association similarity includes:

[0012] The image features corresponding to each target trajectory in each video frame are obtained respectively, and used as the image features of the target trajectory;

[0013] Calculate the similarity between the image features of every two target trajectories, and use it as the original similarity.

[0014] Based on the relationship between the field of view regions of the multiple cameras, the original similarity is optimized to obtain each associated similarity.

[0015] Optionally, the optimization of the original similarity based on the relationship between the field of view regions of the multiple cameras to obtain each associated similarity includes:

[0016] For each pair of target trajectories, if there is an overlap between the time periods corresponding to the two target trajectories and there is no overlap between the field of view areas of the cameras to which the two target trajectories belong, then the original similarity between the two target trajectories is set to the first value, and each associated similarity is obtained.

[0017] Optionally, the optimization of the original similarity based on the relationship between the field of view regions of the multiple cameras to obtain each associated similarity includes:

[0018] For each target trajectory, the target trajectory is input into the pre-trained spatiotemporal transfer network corresponding to the first camera to which the target trajectory belongs, to obtain the transfer probability and prediction duration of each second camera among the multiple cameras other than the first camera;

[0019] The spatiotemporal transfer network is trained based on training samples corresponding to the first camera; the training samples are obtained based on historical sample trajectories in the first camera and the historical association results of the multiple cameras; the transfer probability of each second camera represents the probability that the target to which the target trajectory belongs enters the second camera after leaving the first camera; the prediction duration of each second camera represents the duration from when the target to which the target trajectory belongs leaves the first camera to when it enters the second camera.

[0020] The original similarity between the target trajectory and the target trajectory in the third camera is set to a first value; wherein the transfer probability of the third camera is less than a first threshold; and / or, the original similarity between the target trajectory and the trajectory to be processed in the fourth camera is set to a first value to obtain each associated similarity; wherein the transfer probability of the fourth camera is greater than the first threshold, and the absolute value of the difference between the duration of the time period corresponding to the target trajectory and the time period of the trajectory to be processed and the predicted duration of the fourth camera is greater than a second threshold.

[0021] Optionally, the method further includes:

[0022] After obtaining the first number of target trajectories, determine the second number of associated trajectory sets as the trajectory set to be processed;

[0023] For each set of trajectories to be processed, one trajectory in the set is used as input data, and the duration between the other trajectories in the set and the time period corresponding to the trajectory is used as output data. The spatiotemporal transfer network corresponding to the camera to which the trajectory belongs is then trained.

[0024] Optionally, after obtaining the similarity between the image features of every two trajectories in each target trajectory as the association similarity, the method further includes:

[0025] Calculate the feature values ​​of the similarity matrix containing the aforementioned association similarity;

[0026] The number of feature values ​​greater than the third threshold is determined as the first number.

[0027] Optionally, acquiring the target trajectories captured by each of the multiple cameras as the target trajectory includes:

[0028] Obtain the original trajectories of the target captured by multiple cameras;

[0029] For each original trajectory, according to a first detection method and / or a second detection method, the video frame in the video image corresponding to the original trajectory where the target exchange occurs is determined as the target video frame; wherein, the first detection method determines the target video frame based on the orientation of the target in the video image; the second detection method determines the target video frame based on the average level of similarity of image features of each video frame contained in the video image;

[0030] The original trajectory is divided according to the position of the target video frame in the video image to obtain multiple target trajectories.

[0031] Optionally, the first detection method includes the following steps:

[0032] For each video frame in the video image, the image region in the video frame corresponding to the original trajectory is input into a pre-trained orientation classification network to obtain the orientation of the target to which the original trajectory belongs in the video frame;

[0033] If the change between the orientation of the target to which the original trajectory belongs in the video frame and the orientation of the target to which the original trajectory belongs in the next video frame of the video image is greater than the fourth threshold, then the video frame is determined as the target video frame in the video image where the target exchange has occurred.

[0034] Optionally, the second detection method includes the following steps:

[0035] For each video frame in the video image, determine the video frame and the video frames before the video frame in the video image as the first video frame, and determine the video frames after the video frame in the video image as the second video frame;

[0036] The average level of similarity between the original trajectory and the corresponding image features in the first video frame is calculated as the first average similarity; and the average level of similarity between the original trajectory and the corresponding image features in the second video frame is calculated as the second average similarity.

[0037] The video frame with the largest sum of the first average similarity and the second average similarity is identified as the target video frame in which the target exchange occurred in the video image.

[0038] Secondly, in order to achieve the above objectives, embodiments of this application disclose a cross-camera trajectory association device, the device comprising:

[0039] The target trajectory acquisition module is used to acquire the trajectory of the target captured by multiple cameras, and use it as the target trajectory.

[0040] The association similarity acquisition module is used to obtain the similarity between the image features of every two trajectories in each target trajectory, which is used as the association similarity.

[0041] The association module is used to associate the two trajectories with the highest association similarity to obtain a set of trajectories containing the two trajectories as the new target trajectory;

[0042] The calculation module is used to calculate the correlation similarity between the image features of the new target trajectory and other target trajectories, and to trigger the correlation module until a first number of target trajectories are obtained.

[0043] Optionally, the association similarity acquisition module includes:

[0044] The image feature acquisition submodule is used to acquire the image features corresponding to each target trajectory in each video frame, and use them as the image features of the target trajectory.

[0045] The original similarity acquisition submodule is used to calculate the similarity between the image features of every two target trajectories, which is then used as the original similarity.

[0046] The association similarity acquisition submodule is used to optimize the original similarity based on the relationship between the field of view regions of the multiple cameras to obtain each association similarity.

[0047] Optionally, the association similarity acquisition submodule is specifically used to, for every two target trajectories, if there is an overlap between the time periods corresponding to the two target trajectories and there is no overlap between the field of view areas of the cameras to which the two target trajectories belong, then set the original similarity corresponding to the two target trajectories to the first value to obtain each association similarity.

[0048] Optionally, the association similarity acquisition submodule is specifically used to input the target trajectory into the pre-trained spatiotemporal transfer network corresponding to the first camera to which the target trajectory belongs for each target trajectory, so as to obtain the transfer probability and prediction duration of each second camera among the multiple cameras other than the first camera;

[0049] The spatiotemporal transfer network is trained based on training samples corresponding to the first camera; the training samples are obtained based on historical sample trajectories in the first camera and the historical association results of the multiple cameras; the transfer probability of each second camera represents the probability that the target to which the target trajectory belongs enters the second camera after leaving the first camera; the prediction duration of each second camera represents the duration from when the target to which the target trajectory belongs leaves the first camera to when it enters the second camera.

[0050] The original similarity between the target trajectory and the target trajectory in the third camera is set to a first value; wherein the transfer probability of the third camera is less than a first threshold; and / or, the original similarity between the target trajectory and the trajectory to be processed in the fourth camera is set to a first value to obtain each associated similarity; wherein the transfer probability of the fourth camera is greater than the first threshold, and the absolute value of the difference between the duration of the time period corresponding to the target trajectory and the time period of the trajectory to be processed and the predicted duration of the fourth camera is greater than a second threshold.

[0051] Optionally, the device further includes:

[0052] The module for determining the set of trajectories to be processed is used to determine the second set of trajectories obtained by association after obtaining the first number of target trajectories, as the set of trajectories to be processed;

[0053] The training module is used to train the spatiotemporal transfer network corresponding to the camera to which each trajectory belongs for each set of trajectories to be processed. The module takes one trajectory in the set as input data and the duration between the other trajectories in the set and the time period corresponding to the trajectory as output data.

[0054] Optionally, the device further includes:

[0055] The feature value calculation module is used to calculate the feature values ​​of the similarity matrix containing the association similarity after obtaining the similarity between the image features of every two trajectories in each target trajectory as the association similarity.

[0056] The first number determination module is used to determine the number of feature values ​​greater than the third threshold, which is taken as the first number.

[0057] Optionally, the target trajectory acquisition module includes:

[0058] The raw trajectory acquisition submodule is used to acquire the raw trajectories of the target captured by multiple cameras.

[0059] The target video frame determination submodule is used to determine, for each original trajectory, the video frame in the video image corresponding to which the target exchange occurs, as the target video frame, according to a first detection method and / or a second detection method; wherein, the first detection method determines the target video frame based on the orientation of the target in the video image; the second detection method determines the target video frame based on the average level of similarity of image features of each video frame contained in the video image;

[0060] The segmentation submodule is used to segment the original trajectory according to the position of the target video frame in the video image to obtain multiple target trajectories.

[0061] Optionally, the target video frame determination submodule is specifically used to input the image region corresponding to the original trajectory in each video frame of the video image into a pre-trained orientation classification network to obtain the orientation of the target to which the original trajectory belongs in the video frame.

[0062] If the change between the orientation of the target to which the original trajectory belongs in the video frame and the orientation of the target to which the original trajectory belongs in the next video frame of the video image is greater than the fourth threshold, then the video frame is determined as the target video frame in the video image where the target exchange has occurred.

[0063] Optionally, the target video frame determination submodule is specifically used to determine, for each video frame in the video image, the video frame and the video frame before the video frame in the video image as the first video frame, and to determine the video frame after the video frame in the video image as the second video frame.

[0064] The average level of similarity between the original trajectory and the corresponding image features in the first video frame is calculated as the first average similarity; and the average level of similarity between the original trajectory and the corresponding image features in the second video frame is calculated as the second average similarity.

[0065] The video frame with the largest sum of the first average similarity and the second average similarity is identified as the target video frame in which the target exchange occurred in the video image.

[0066] In another aspect of this application, in order to achieve the above objectives, an embodiment of this application also discloses an electronic device, which includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0067] The memory is used to store computer programs;

[0068] When the processor executes the program stored in the memory, it implements the cross-camera trajectory association method as described in the first aspect above.

[0069] In another aspect of this application, a computer-readable storage medium is provided, wherein instructions are stored therein, which, when executed on a computer, implement the cross-camera trajectory association method as described in the first aspect above.

[0070] In another aspect of this application, embodiments of this application also provide a computer program product containing instructions that, when run on a computer, cause the computer to execute the cross-camera trajectory association method described in the first aspect above.

[0071] This application provides a cross-camera trajectory association method, which can acquire the trajectory of a target collected by multiple cameras as the target trajectory; acquire the similarity between the image features of every two trajectories in each target trajectory as the association similarity; associate the two trajectories with the highest association similarity to obtain a trajectory set containing the two trajectories as a new target trajectory; calculate the association similarity between the image features of the new target trajectory and other target trajectories, and return to execute the step of associating the two trajectories with the highest association similarity to obtain a trajectory set containing the two trajectories as a new target trajectory, until a first number of target trajectories are obtained.

[0072] Since the image features of the same target are highly similar in different cameras, the cross-camera trajectory association method provided in this application can effectively determine the trajectories of the same target in each camera based on the global clustering algorithm and the similarity of the image features of the trajectory, that is, effectively associate the trajectories of each target.

[0073] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0074] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0075] Figure 1 A flowchart illustrating a cross-camera trajectory association method provided in this application embodiment;

[0076] Figure 2 A flowchart illustrating another cross-camera trajectory association method provided in this application embodiment;

[0077] Figure 3 A flowchart illustrating another cross-camera trajectory association method provided in this application embodiment;

[0078] Figure 4 A flowchart illustrating another cross-camera trajectory association method provided in this application embodiment;

[0079] Figure 5 A schematic diagram illustrating the relationship between the field of view regions of a camera, provided as an embodiment of this application;

[0080] Figure 6 A flowchart illustrating another cross-camera trajectory association method provided in this application embodiment;

[0081] Figure 7 A schematic diagram illustrating a preset direction provided in an embodiment of this application;

[0082] Figure 8 A structural diagram of a cross-camera trajectory association device provided in an embodiment of this application;

[0083] Figure 9 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0084] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0085] To effectively correlate the trajectories of the same target across different cameras, embodiments of this application provide a cross-camera trajectory correlation method, which can be applied to an electronic device. For example, the electronic device can communicate with each camera to obtain the target trajectory captured by each camera, and correlate the target trajectories based on the cross-camera trajectory correlation method provided in this application embodiment; alternatively, the electronic device can also acquire video images captured by each camera, obtain the target trajectory based on the video images, and then correlate the target trajectories based on the cross-camera trajectory correlation method provided in the basic application embodiment.

[0086] See Figure 1 , Figure 1 A flowchart of a cross-camera trajectory association method provided in this application embodiment, the method may include the following steps:

[0087] S101: Acquire the trajectory of the target captured by multiple cameras, and use it as the target trajectory.

[0088] S102: Obtain the similarity between the image features of every two trajectories in each target trajectory, and use it as the association similarity.

[0089] S103: Associate the two trajectories with the highest correlation similarity to obtain a set of trajectories containing the two trajectories as the new target trajectory.

[0090] S104: Calculate the correlation similarity between the image features of the new target trajectory and other target trajectories, and return to execute step S103 until a first number of target trajectories are obtained.

[0091] Since the image features of the same target in different cameras have a high degree of similarity, the cross-camera trajectory association method provided in this application can effectively determine the trajectories of the same target in each camera based on the global clustering algorithm and the similarity of the image features of the trajectory, that is, effectively associate the trajectories of each target.

[0092] Regarding step S101, the target in this embodiment can be an animal, or it can be a vehicle, but it is not limited to these. Multiple cameras can be deployed in different sub-regions to acquire images of each sub-region. There may or may not overlap between the sub-regions (i.e., the field of view) corresponding to any two cameras.

[0093] The trajectory of a target captured by a camera can be represented by the target's coordinates within the video image captured by that camera. That is, one trajectory can correspond to multiple video frames, and the corresponding time period is the time period of those multiple video frames. For example, the target's trajectory includes the target's coordinates within each video frame contained in the video image.

[0094] For step S102, the image features of a trajectory are the image features of the image region corresponding to the trajectory in the video image.

[0095] In one implementation, a video frame can be selected from the video images, and the image features of the image region corresponding to the trajectory in that video frame can be used as the image features of the trajectory.

[0096] Alternatively, multiple video frames can be selected from the video image, and the average value of the image features of the image regions corresponding to the trajectory in each video frame can be used as the image features of the trajectory.

[0097] In this embodiment of the application, image features can be represented by feature vectors. Correspondingly, the similarity between image features can be represented based on the cosine similarity of feature vectors, or it can be represented based on the Euclidean distance of feature vectors, but is not limited to these.

[0098] For steps S103 and S104, the highest correlation similarity indicates that the image features of the two target trajectories are highly similar, meaning that the two target trajectories belong to the same target. Therefore, the two target trajectories can be correlated. These two target trajectories can be referred to as the old trajectories. The set of these two target trajectories is taken as the new target trajectory, which correspondingly reduces the number of target trajectories by 1.

[0099] In one implementation, for each other target trajectory (i.e., target trajectories other than the two old trajectories mentioned above), the average value of the association similarity between the other target trajectory and the two old trajectories can be calculated as the association similarity between the image features of the other target trajectory and the new target trajectory.

[0100] In another implementation, the two old trajectories can be merged, and the image features of the merged trajectory can be obtained. Then, for each other target trajectory (i.e., target trajectories other than the two old trajectories mentioned above), the correlation similarity between the image features of the other target trajectory and the merged trajectory can be calculated, which is used as the correlation similarity between the image features of the other target trajectory and the trajectory set corresponding to the two old trajectories (i.e., the new target trajectory).

[0101] Based on steps S103 and S104 above, the number of target trajectories decreases by 1 after one iteration. Therefore, after multiple iterations, if the number of target trajectories decreases to the first number, the iteration stops. At this point, each target trajectory is the trajectory of the same target. It can be understood that a target trajectory at this point may be a set of trajectories containing multiple initial target trajectories, or it may be the initial target trajectory itself.

[0102] Following the above processing procedure, it is possible to effectively determine the trajectories belonging to the same target in each camera based on a global clustering algorithm and the similarity of image features of the trajectories, that is, to effectively associate the trajectories of each target. Compared with the method of associating target trajectories by comparing the similarity with a preset threshold, this method can avoid the subjective experience of threshold setting and improve the accuracy of target trajectory association.

[0103] In one embodiment, see Figure 2 ,exist Figure 1 Based on this, step S102 above may include the following steps:

[0104] S1021: Obtain the average value of the image features corresponding to each target trajectory in each video frame, and use it as the image feature of the target trajectory.

[0105] S1022: Calculate the similarity between the image features of every two target trajectories as the original similarity.

[0106] S1023: Based on the relationship between the field of view regions of multiple cameras, the original similarity is optimized to obtain the associated similarity.

[0107] Regarding step S1021, in one implementation, the image features corresponding to the target trajectory in each video frame can be extracted based on a feature extraction network, and the average value of each image feature can be calculated as the image feature of the target trajectory.

[0108] In another implementation, a target re-identification network trained on video images can be obtained, and the image features of the target trajectory in the video images can be acquired based on this network. Based on this, the acquired image features can also reflect the temporal information of the target trajectory, thereby improving the accuracy of the calculated association similarity and thus the accuracy of associating the target trajectory.

[0109] For step S1023, different methods can be used to optimize the original similarity.

[0110] In one embodiment, see Figure 3 ,exist Figure 2 Based on this, step S1023 above may include the following steps:

[0111] S10231: For each pair of target trajectories, if there is an overlap between the time periods corresponding to the two target trajectories and there is no overlap between the field of view areas of the cameras to which the two target trajectories belong, then the original similarity between the two target trajectories is set to the first value, and each associated similarity is obtained.

[0112] The first value can be 0. Alternatively, the first value can be set to a small value that is close to 0, for example, the first value can be set to 0.0001, but it is not limited to this.

[0113] In this embodiment, if there is no overlap between the field of view areas of the cameras to which the two target trajectories belong, it indicates that the same target cannot appear in the field of view areas of the two cameras simultaneously. Therefore, if there is an overlap between the time periods corresponding to the two target trajectories, it indicates that the two target trajectories are trajectories of different targets, and the original similarity between the two target trajectories can be set to the first value.

[0114] Based on the above processing, abnormal original similarity can be eliminated, the accuracy of association similarity can be improved, and thus the accuracy of associating target trajectories can be improved.

[0115] In one embodiment, see Figure 4 ,exist Figure 2 Based on this, step S1023 above may include the following steps:

[0116] S10232: For each target trajectory, input the target trajectory into the pre-trained spatiotemporal transfer network corresponding to the first camera to which the target trajectory belongs, and obtain the transfer probability and prediction duration of each second camera among multiple cameras other than the first camera.

[0117] S10233: Set the original similarity corresponding to the target trajectory and the target trajectory in the third camera to the first value; and / or, set the original similarity corresponding to the target trajectory and the trajectory to be processed in the fourth camera to the first value to obtain each associated similarity.

[0118] The spatiotemporal transfer network is trained using training samples from the first camera. These training samples are derived from historical sample trajectories in the first camera and historical association results from multiple cameras. For example, based on the relationship between the field of view areas of multiple cameras and historical sample trajectories in the first camera, historical association results from multiple cameras can be obtained; that is, historical trajectories from other cameras associated with historical sample trajectories in the first camera can be obtained. The transfer probability of each second camera represents the probability that the target trajectory belongs to will enter the second camera after leaving the first camera. The prediction duration of each second camera represents the duration from when the target trajectory belongs to the first camera to when it enters the second camera. The transfer probability of the third camera is less than a first threshold. The transfer probability of the fourth camera is greater than the first threshold, and the absolute value of the difference between the duration of the time interval between the trajectory to be processed and the time interval corresponding to the target trajectory and the prediction duration of the fourth camera is greater than a second threshold.

[0119] The spatiotemporal transfer network can include a classification subnetwork and a regression subnetwork. The classification subnetwork is used to determine the transfer probability, and the regression subnetwork is used to determine the prediction duration. In this embodiment, each camera can correspond to one spatiotemporal transfer network. The input data of the spatiotemporal transfer network is a trajectory in that camera. For example, the coordinates of the trajectory in the video image can be specified in a number of video frames (e.g., 100). The output of the spatiotemporal transfer network includes: the probability that the target to which the trajectory belongs enters each other camera after leaving the camera (i.e., the transfer probability of each other camera), and the duration from when the target to which the trajectory belongs leaves the camera to when it enters each other camera (i.e., the prediction duration of each other camera).

[0120] The first threshold can be 0.05, or 0.1, but is not limited to these values. If the transfer probability of the third camera is less than the first threshold, it indicates that the probability of the target trajectory in the first camera passing through the field of view of the third camera is very small. Therefore, the original similarity between the target trajectory and the target trajectory in the third camera can be set to the first value.

[0121] If the transfer probability of the fourth camera is greater than the first threshold, and the target trajectory and a trajectory in the fourth camera (which can be called the comparison trajectory) belong to the same target, then the difference between the duration of the time interval between the target trajectory and the comparison trajectory and the predicted duration of the fourth camera will not be too large. However, if the absolute value of the difference between the duration of the time interval between the processing trajectory and the target trajectory (which can be called the target duration) and the predicted duration of the fourth camera is greater than the second threshold, that is, the target duration is much greater than the predicted duration of the fourth camera, or the target duration is much less than the predicted duration of the fourth camera. Therefore, it can be determined that the processing trajectory and the target trajectory belong to different targets. The original similarity between the target trajectory and the processing trajectory is set to the first value.

[0122] For example, when the target duration is greater than max(1.5 × predicted duration t, predicted duration t + threshold), or when the target duration is less than min(0.5 × predicted duration t, predicted duration t - threshold), the original similarity between the target trajectory and the trajectory to be processed is set to the first value. The threshold threshold can be 10 seconds or 12 seconds, but is not limited to these values.

[0123] For example, see Figure 5 , Figure 5 This is a schematic diagram illustrating the relationship between the field of view areas of a camera, as provided in an embodiment of this application.

[0124] Figure 5The system contains 5 cameras, cameras a through e, with a first threshold of 5%, and the target is a vehicle. If the predicted trajectory of the target in camera a includes the following: an 80% probability that the vehicle will appear at camera c after a predicted time t1, a 20% probability that the vehicle will appear at camera e after a predicted time t2, and the probabilities of the vehicle appearing at cameras b and d are both less than 5%, then the original similarity between the target trajectory and all trajectories in cameras b and d can be set to 0. The original similarity between the target trajectory and the trajectory to be processed in cameras c and e can also be set to 0. The time interval between the trajectory to be processed and the target trajectory must be greater than max(1.5 × predicted time, predicted time + threshold) or less than min(0.5 × predicted time, predicted time - threshold).

[0125] Based on the above processing, abnormal original similarity can be eliminated, the accuracy of association similarity can be improved, and thus the accuracy of associating target trajectories can be improved.

[0126] In one implementation, the spatiotemporal transfer network corresponding to each camera can be pre-trained based on the sample trajectories collected by each camera and the correlation between these trajectories. The correlation between the sample trajectories can be obtained by technicians by labeling the relationships between the field of view regions of each camera.

[0127] In one embodiment, to save labeling costs, the correlation between the trajectories used to train the spatiotemporal transfer network of each camera can also be determined based on steps S101-S104 during the previous trajectory association. That is, based on the correlation determined in steps S101-S104, the corresponding pseudo-labels are obtained, and the spatiotemporal transfer network of each camera is trained.

[0128] In one implementation, when initially associating trajectories, a spatiotemporal transfer network may not be used to optimize the original similarity. After multiple trajectory associations, resulting in a certain number of associations, the spatiotemporal transfer network can be trained according to these established associations. The trained spatiotemporal transfer network can then be used to optimize the original similarity for the next association, improving the accuracy of the established associations. More accurate associations can further improve the accuracy of the training data, allowing for continued optimization of the spatiotemporal transfer network training.

[0129] Based on the above processing, the original similarity can be optimized based on the spatiotemporal transfer network, thereby improving the accuracy of the association similarity and thus the accuracy of the determined association. In turn, the accuracy of the spatiotemporal transfer network can be further improved based on the determined association. This process can be repeated cyclically, achieving mutual promotion and iterative optimization between the spatiotemporal transfer network and the original similarity optimization, while also saving the cost of manual labeling.

[0130] Accordingly, in one embodiment, the method may further include the following steps:

[0131] Step 1: After obtaining the first number of target trajectories, determine the second-to-last set of associated trajectories as the set of trajectories to be processed.

[0132] Step 2: For each set of trajectories to be processed, take one trajectory in the set as input data, and take the duration between the other trajectories in the set and the time period corresponding to the trajectory as output data, and train the spatiotemporal transfer network corresponding to the camera to which the trajectory belongs.

[0133] The second number can be half of the first number, or two-thirds of the first number, but is not limited to these. Since this embodiment prioritizes associating the two target trajectories with the highest correlation similarity to obtain a trajectory set, the accuracy of the first second number of trajectory sets determined according to steps S101-S104 is relatively high, and can be used to train the spatiotemporal transfer network to improve its accuracy.

[0134] Since trajectories belonging to the same set of trajectories to be processed are trajectories of the same target, when training the spatiotemporal transfer network, the probability of the output corresponding to the camera of each other trajectory in the set of trajectories to be processed and the input trajectory can be set to 1.

[0135] Based on the above processing, the accuracy of the spatiotemporal transfer network can be further improved based on the determined association relationship. By repeating this process, the spatiotemporal transfer network and the original similarity optimization can be mutually promoted and iteratively optimized, and the cost of manual labeling can be saved.

[0136] In one embodiment, after step S102 described above, the method may further include the following steps:

[0137] Calculate the eigenvalues ​​of the similarity matrix containing association similarity; determine the number of eigenvalues ​​greater than the third threshold, and use this as the first number.

[0138] The third threshold can be 0.9, or 0.95, but is not limited to these values. In this embodiment, processing can be based on a similarity matrix containing association similarities. That is, after determining each association similarity, a matrix containing each association similarity, i.e., a similarity matrix, can be generated. The number of rows in this similarity matrix is ​​the number of target trajectories (e.g., M), and the number of columns is the number of target trajectories. That is, the similarity matrix is ​​a symmetric matrix, where the diagonal elements can be 1, and the remaining elements represent the association similarity between the image features of each pair of target trajectories. Furthermore, the eigenvalues ​​of this similarity matrix can be calculated, i.e., M eigenvalues ​​are obtained. Then, the number of eigenvalues ​​greater than the third threshold among the M eigenvalues ​​can be determined, which is the first number.

[0139] In one embodiment, see Figure 6 ,exist Figure 1 Based on this, step S101 above may include the following steps:

[0140] S1011: Obtain the original trajectory of the target captured by each of the multiple cameras.

[0141] S1012: For each original trajectory, determine the video frame in the video image corresponding to the original trajectory where the target exchange occurs, according to the first detection method and / or the second detection method, and use it as the target video frame.

[0142] The first detection method determines the target video frame based on the orientation of the target in the video image; the second detection method determines the target video frame based on the average level of similarity of image features of each video frame contained in the video image.

[0143] S1013: Divide the original trajectory according to the position of the target video frame in the video image to obtain multiple target trajectories.

[0144] In the embodiments of this application, an original trajectory determined by processing video images based on a target tracking algorithm may contain multiple targets. That is, the image features of the original trajectory contain the image features of multiple targets. If the image features of the original trajectory are directly compared for trajectory association, the accuracy of the association will be reduced.

[0145] To avoid the above situation, the original trajectory where the target exchange occurred (i.e., the original trajectory containing multiple targets) can be determined, and the original trajectory can be divided into multiple target trajectories, so that each target trajectory contains only one target.

[0146] The first detection method determines the target video frame based on the orientation of the target in the video image. In one implementation, for an original trajectory, the orientation of the target corresponding to the original trajectory in each video frame contained in the video image can be detected. For example, if the target is a vehicle, it is possible to detect whether the vehicle's orientation (i.e., the direction the vehicle is moving) has changed, in order to determine whether a target exchange has occurred in the original trajectory, and to determine the video frame where the change occurred, i.e., the target video frame.

[0147] The second detection method determines the target video frame based on the average similarity of image features across all video frames contained in the video image. For example, for an original trajectory, image features corresponding to the original trajectory can be extracted from each video frame contained in the video image. The similarity between corresponding image features in every two video frames is then calculated, and the average of these similarities is calculated. If this average is low, it can be determined that a target swap has occurred in the original trajectory.

[0148] In one embodiment, the first detection method includes the following steps:

[0149] For each video frame in the video image, the image region in the video frame corresponding to the original trajectory is input into a pre-trained orientation classification network to obtain the orientation of the target to which the original trajectory belongs in the video frame; if the change between the orientation of the target to which the original trajectory belongs in the video frame and the orientation of the target to which the original trajectory belongs in the next video frame in the video image is greater than a fourth threshold, then the video frame is determined as the target video frame in the video image where the target exchange has occurred.

[0150] In this embodiment of the application, an orientation classification network can be pre-trained. For example, the input data of the orientation classification network is the image region occupied by the target in a video frame, and the output data includes the confidence score of the target's orientation for each preset direction.

[0151] In one implementation, the preset directions can include 8 directions, for example, see [link to relevant documentation]. Figure 7 , Figure 7 This is a schematic diagram of a preset direction provided in an embodiment of this application.

[0152] In this context, direction 0 indicates that the target is facing the camera, direction 4 indicates that the target is facing away from the camera, direction 2 indicates that the target is facing 90 degrees clockwise from direction 0, direction 6 indicates that the target is facing 90 degrees counterclockwise from direction 0, direction 3 indicates the direction of the center line between direction 4 and direction 2, direction 5 indicates the direction of the center line between direction 4 and direction 6, direction 7 indicates the direction of the center line between direction 0 and direction 6, and direction 1 indicates the direction of the center line between direction 0 and direction 2.

[0153] In the example above, if the change in the target's orientation between two adjacent predicted video frames is greater than 2, then a target swap can be determined to have occurred in the original trajectory. For instance, if the target's orientation is direction 1 in one predicted video frame and direction 3 in the next video frame, then a target swap can be determined to have occurred at that video frame, and that video frame is the target video frame.

[0154] In one embodiment, the second detection method includes the following steps:

[0155] Step 1: For each video frame in the video image, determine the video frame and the video frames before it in the video image as the first video frame, and determine the video frames after it in the video image as the second video frame.

[0156] Step 2: Calculate the average level of similarity between the corresponding image features of the original trajectory in the first video frame as the first average similarity; and calculate the average level of similarity between the corresponding image features of the original trajectory in the second video frame as the second average similarity.

[0157] Step 3: Determine the video frame with the largest sum of the first average similarity and the second average similarity as the target video frame in which the target swap occurred in the video image.

[0158] In the embodiments of this application, the average level of similarity between the image features corresponding to the original trajectory in N video frames can be calculated based on formula (1).

[0159] (1)

[0160] Where L represents the average level of similarity between the image features corresponding to the original trajectory in N video frames, S represents the similarity matrix formed by the similarity between the image features corresponding to the original trajectory in N video frames, the similarity matrix S is an N×N symmetric matrix with 1 on the diagonal, and sum(S) represents the sum of all elements in the similarity matrix S.

[0161] Furthermore, for each video frame, the sum of the first average similarity and the second average similarity can be calculated according to formula (2).

[0162] (2)

[0163] D represents the sum of the first and second average similarities corresponding to the nth video frame. S1 represents the similarity matrix formed by the similarities between the image features corresponding to the original trajectory in the first n video frames. The similarity matrix S1 is an n×n symmetric matrix with diagonal elements of 1. sum(S1) represents the sum of all elements in the similarity matrix S1. S2 represents the similarity matrix formed by the similarities between the image features corresponding to the original trajectory in other video frames besides the first n video frames. The similarity matrix S2 is a (Nn)×(Nn) symmetric matrix with diagonal elements of 1. sum(S2) represents the sum of all elements in the similarity matrix S2.

[0164] In addition, based on formula (1), if the average level of similarity between the corresponding image features in the video frames contained in the video image is less than the fifth threshold (for example, it can be 0.6), then it can be determined that a target exchange has occurred in the original trajectory.

[0165] In another implementation, the average similarity between the corresponding image features of the original trajectory in the first video frame can be calculated as the first average similarity; and the average similarity between the corresponding image features of the original trajectory in the second video frame can be calculated as the second average similarity.

[0166] Based on the same inventive concept, this application also provides a cross-camera trajectory association device, see [link to relevant documentation]. Figure 8 , Figure 8 This application provides a structural diagram of a cross-camera trajectory association device, which may include:

[0167] The target trajectory acquisition module 801 is used to acquire the trajectory of the target collected by multiple cameras, as the target trajectory;

[0168] The association similarity acquisition module 802 is used to acquire the similarity between the image features of every two trajectories in each target trajectory, as the association similarity;

[0169] The association module 803 is used to associate the two trajectories with the highest association similarity to obtain a set of trajectories containing the two trajectories as the new target trajectory;

[0170] The calculation module 804 is used to calculate the correlation similarity between the image features of the new target trajectory and other target trajectories, and to trigger the correlation module until a first number of target trajectories are obtained.

[0171] Optionally, the association similarity acquisition module 802 includes:

[0172] The image feature acquisition submodule is used to acquire the image features corresponding to each target trajectory in each video frame, and use them as the image features of the target trajectory.

[0173] The original similarity acquisition submodule is used to calculate the similarity between the image features of every two target trajectories, which is then used as the original similarity.

[0174] The association similarity acquisition submodule is used to optimize the original similarity based on the relationship between the field of view regions of the multiple cameras to obtain each association similarity.

[0175] Optionally, the association similarity acquisition submodule is specifically used to, for every two target trajectories, if there is an overlap between the time periods corresponding to the two target trajectories and there is no overlap between the field of view areas of the cameras to which the two target trajectories belong, then set the original similarity corresponding to the two target trajectories to the first value to obtain each association similarity.

[0176] Optionally, the association similarity acquisition submodule is specifically used to input the target trajectory into the pre-trained spatiotemporal transfer network corresponding to the first camera to which the target trajectory belongs for each target trajectory, so as to obtain the transfer probability and prediction duration of each second camera among the multiple cameras other than the first camera;

[0177] The spatiotemporal transfer network is trained based on training samples corresponding to the first camera; the training samples are obtained based on historical sample trajectories in the first camera and the historical association results of the multiple cameras; the transfer probability of each second camera represents the probability that the target to which the target trajectory belongs enters the second camera after leaving the first camera; the prediction duration of each second camera represents the duration from when the target to which the target trajectory belongs leaves the first camera to when it enters the second camera.

[0178] The original similarity between the target trajectory and the target trajectory in the third camera is set to a first value; wherein the transfer probability of the third camera is less than a first threshold; and / or, the original similarity between the target trajectory and the trajectory to be processed in the fourth camera is set to a first value to obtain each associated similarity; wherein the transfer probability of the fourth camera is greater than the first threshold, and the absolute value of the difference between the duration of the time period corresponding to the target trajectory and the time period of the trajectory to be processed and the predicted duration of the fourth camera is greater than a second threshold.

[0179] Optionally, the device further includes:

[0180] The module for determining the set of trajectories to be processed is used to determine the second set of trajectories obtained by association after obtaining the first number of target trajectories, as the set of trajectories to be processed;

[0181] The training module is used to train the spatiotemporal transfer network corresponding to the camera to which each trajectory belongs for each set of trajectories to be processed. The module takes one trajectory in the set as input data and the duration between the other trajectories in the set and the time period corresponding to the trajectory as output data.

[0182] Optionally, the device further includes:

[0183] The feature value calculation module is used to calculate the feature values ​​of the similarity matrix containing the association similarity after obtaining the similarity between the image features of every two trajectories in each target trajectory as the association similarity.

[0184] The first number determination module is used to determine the number of feature values ​​greater than the third threshold, which is taken as the first number.

[0185] Optionally, the target trajectory acquisition module 801 includes:

[0186] The raw trajectory acquisition submodule is used to acquire the raw trajectories of the target captured by multiple cameras.

[0187] The target video frame determination submodule is used to determine, for each original trajectory, the video frame in the video image corresponding to which the target exchange occurs, as the target video frame, according to a first detection method and / or a second detection method; wherein, the first detection method determines the target video frame based on the orientation of the target in the video image; the second detection method determines the target video frame based on the average level of similarity of image features of each video frame contained in the video image;

[0188] The segmentation submodule is used to segment the original trajectory according to the position of the target video frame in the video image to obtain multiple target trajectories.

[0189] Optionally, the target video frame determination submodule is specifically used to input the image region corresponding to the original trajectory in each video frame of the video image into a pre-trained orientation classification network to obtain the orientation of the target to which the original trajectory belongs in the video frame.

[0190] If the change between the orientation of the target to which the original trajectory belongs in the video frame and the orientation of the target to which the original trajectory belongs in the next video frame of the video image is greater than the fourth threshold, then the video frame is determined as the target video frame in the video image where the target exchange has occurred.

[0191] Optionally, the target video frame determination submodule is specifically used to determine, for each video frame in the video image, the video frame and the video frame before the video frame in the video image as the first video frame, and to determine the video frame after the video frame in the video image as the second video frame.

[0192] The average level of similarity between the original trajectory and the corresponding image features in the first video frame is calculated as the first average similarity; and the average level of similarity between the original trajectory and the corresponding image features in the second video frame is calculated as the second average similarity.

[0193] The video frame with the largest sum of the first average similarity and the second average similarity is identified as the target video frame in which the target exchange occurred in the video image.

[0194] This application also provides an electronic device, such as... Figure 9 As shown, it includes a processor 901, a communication interface 902, a memory 903, and a communication bus 904, wherein the processor 901, the communication interface 902, and the memory 903 communicate with each other through the communication bus 904.

[0195] Memory 903 is used to store computer programs;

[0196] When the processor 901 executes the program stored in the memory 903, it implements the cross-camera trajectory association method in the above embodiments.

[0197] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not indicate that there is only one bus or one type of bus.

[0198] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0199] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0200] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0201] This application also provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the cross-camera trajectory association method provided in this application.

[0202] This application also provides another computer program product containing instructions that, when run on a computer, cause the computer to execute the cross-camera trajectory association method provided in this application.

[0203] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state drive (SSD), etc.

[0204] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0205] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, electronic devices, computer-readable storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0206] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A method for cross-camera trajectory association, characterized in that, The method includes: Acquire the trajectory of the target captured by multiple cameras, and use it as the target trajectory; The similarity between the image features of every two trajectories in each target trajectory is obtained as the association similarity. Here, the image feature of a target trajectory represents the image feature of the corresponding image region in a video image containing the target. For each target trajectory, the association similarity between the target trajectory and the target trajectory in the third camera is obtained by setting the original similarity between the image features of the two target trajectories to a first value. And / or, the association similarity between the target trajectory and the trajectory to be processed in the fourth camera is obtained by setting the original similarity between the target trajectory and the trajectory to be processed to a first value. The association similarity between two target trajectories is represented by the first value, indicating that the two target trajectories... The trajectories are trajectories of different targets; the transition probability corresponding to the third camera is less than a first threshold; the camera to which the target trajectory belongs is the first camera, and the second camera is any camera other than the first camera among the plurality of cameras; the transition probability of each second camera represents the probability that the target to which the target trajectory belongs enters the second camera after leaving the first camera; the transition probability corresponding to the fourth camera is greater than the first threshold; the absolute value of the difference between the time interval between the time interval corresponding to the target trajectory and the time interval corresponding to the target trajectory and the predicted time interval of the fourth camera is greater than the second threshold; the predicted time interval of each second camera represents the time interval between the target to which the target trajectory belongs leaving the first camera and entering the second camera; The two trajectories with the highest correlation similarity are associated to obtain a new set of trajectories containing the two trajectories. Calculate the correlation similarity between the image features of the new target trajectory and other target trajectories, and return to perform the step of associating the two trajectories with the highest correlation similarity to obtain a trajectory set containing the two trajectories as the new target trajectory, until a first number of target trajectories are obtained.

2. The method according to claim 1, characterized in that, The step of obtaining the similarity between the image features of every two trajectories in each target trajectory, as the association similarity, includes: The image features corresponding to each target trajectory in each video frame are obtained respectively, and used as the image features of the target trajectory; Calculate the similarity between the image features of every two target trajectories, and use it as the original similarity. Based on the relationship between the field of view regions of the multiple cameras, the original similarity is optimized to obtain each associated similarity.

3. The method according to claim 2, characterized in that, The original similarity is optimized based on the relationship between the field of view regions of the multiple cameras to obtain various associated similarities, including: For each pair of target trajectories, if there is an overlap between the time periods corresponding to the two target trajectories and there is no overlap between the field of view areas of the cameras to which the two target trajectories belong, then the original similarity between the two target trajectories is set to the first value, and each associated similarity is obtained.

4. The method according to claim 2, characterized in that, The original similarity is optimized based on the relationship between the field of view regions of the multiple cameras to obtain various associated similarities, including: For each target trajectory, the target trajectory is input into the pre-trained spatiotemporal transfer network corresponding to the first camera to which the target trajectory belongs, to obtain the transfer probability and prediction duration of each second camera among the multiple cameras other than the first camera; The spatiotemporal transfer network is trained based on training samples corresponding to the first camera; the training samples are obtained based on historical sample trajectories in the first camera and historical association results of the multiple cameras. Set the original similarity between the target trajectory and the target trajectory in the third camera to the first value; and / or set the original similarity between the target trajectory and the trajectory to be processed in the fourth camera to the first value to obtain each associated similarity.

5. The method according to claim 4, characterized in that, The method further includes: After obtaining the first number of target trajectories, determine the second number of associated trajectory sets as the trajectory set to be processed; For each set of trajectories to be processed, one trajectory in the set is used as input data, and the duration between the other trajectories in the set and the time period corresponding to the trajectory is used as output data. The spatiotemporal transfer network corresponding to the camera to which the trajectory belongs is then trained.

6. The method according to claim 1, characterized in that, After obtaining the similarity between the image features of every two trajectories in each target trajectory as the correlation similarity, the method further includes: Calculate the feature values ​​of the similarity matrix containing the aforementioned association similarity; The number of feature values ​​greater than the third threshold is determined as the first number.

7. The method according to claim 1, characterized in that, The acquisition of the target trajectory captured by each of the multiple cameras, as the target trajectory, includes: Obtain the original trajectories of the target captured by multiple cameras; For each original trajectory, according to a first detection method and / or a second detection method, the video frame in the video image corresponding to the original trajectory where the target exchange occurs is determined as the target video frame; wherein, the first detection method determines the target video frame based on the orientation of the target in the video image; the second detection method determines the target video frame based on the average level of similarity of image features of each video frame contained in the video image; The original trajectory is divided according to the position of the target video frame in the video image to obtain multiple target trajectories.

8. The method according to claim 7, characterized in that, The first detection method includes the following steps: For each video frame in the video image, the image region in the video frame corresponding to the original trajectory is input into a pre-trained orientation classification network to obtain the orientation of the target to which the original trajectory belongs in the video frame; If the change between the orientation of the target to which the original trajectory belongs in the video frame and the orientation of the target to which the original trajectory belongs in the next video frame of the video image is greater than the fourth threshold, then the video frame is determined as the target video frame in the video image where the target exchange has occurred.

9. The method according to claim 7, characterized in that, The second detection method includes the following steps: For each video frame in the video image, determine the video frame and the video frames before the video frame in the video image as the first video frame, and determine the video frames after the video frame in the video image as the second video frame; The average level of similarity between the original trajectory and the corresponding image features in the first video frame is calculated as the first average similarity; and the average level of similarity between the original trajectory and the corresponding image features in the second video frame is calculated as the second average similarity. The video frame with the largest sum of the first average similarity and the second average similarity is identified as the target video frame in which the target exchange occurred in the video image.

10. A cross-camera trajectory correlation device, characterized in that, The device includes: The target trajectory acquisition module is used to acquire the trajectory of the target captured by multiple cameras, and use it as the target trajectory. The association similarity acquisition module is used to obtain the similarity between the image features of every two trajectories in each target trajectory based on a global clustering algorithm, as the association similarity. Here, the image feature of a target trajectory represents the image feature of the corresponding image region in a video image containing the target. For each target trajectory, the association similarity between the target trajectory and the target trajectory in the third camera is obtained by setting the original similarity between the image features of the two target trajectories to a first value; and / or, the association similarity between the target trajectory and the trajectory to be processed in the fourth camera is obtained by setting the original similarity between the target trajectory and the trajectory to be processed to a first value; the association similarity between two target trajectories is... The first value indicates that the two target trajectories are trajectories of different targets; the transfer probability corresponding to the third camera is less than the first threshold; the camera to which the target trajectory belongs is the first camera, and the second camera is the camera other than the first camera among the plurality of cameras; the transfer probability of each second camera represents the probability that the target to which the target trajectory belongs enters the second camera after leaving the first camera; the transfer probability corresponding to the fourth camera is greater than the first threshold; the absolute value of the difference between the time interval between the time interval corresponding to the target trajectory and the time interval corresponding to the target trajectory and the predicted time interval of the fourth camera is greater than the second threshold; the predicted time interval of each second camera represents the time interval between the target to which the target trajectory belongs leaving the first camera and entering the second camera; The association module is used to associate the two trajectories with the highest association similarity to obtain a set of trajectories containing the two trajectories as the new target trajectory; The calculation module is used to calculate the association similarity between the image features of the new target trajectory and other target trajectories other than the already associated trajectory based on the global clustering algorithm, and to trigger the association module until a first number of target trajectories are obtained.

11. The apparatus according to claim 10, characterized in that, The association similarity acquisition module includes: The image feature acquisition submodule is used to acquire the image features corresponding to each target trajectory in each video frame, and use them as the image features of the target trajectory. The original similarity acquisition submodule is used to calculate the similarity between the image features of every two target trajectories, which is then used as the original similarity. The association similarity acquisition submodule is used to optimize the original similarity based on the relationship between the field of view regions of the multiple cameras to obtain each association similarity.

12. The apparatus according to claim 11, characterized in that, The association similarity acquisition submodule is specifically used to, for each pair of target trajectories, if there is an overlap between the time periods corresponding to the two target trajectories and there is no overlap between the field of view areas of the cameras to which the two target trajectories belong, then set the original similarity of the two target trajectories to the first value to obtain each association similarity.

13. The apparatus according to claim 11, characterized in that, The association similarity acquisition submodule is specifically used to input the target trajectory into the pre-trained spatiotemporal transfer network corresponding to the first camera to which the target trajectory belongs for each target trajectory, so as to obtain the transfer probability and prediction duration of each second camera among the multiple cameras other than the first camera; The spatiotemporal transfer network is trained based on training samples corresponding to the first camera; the training samples are obtained based on historical sample trajectories in the first camera and historical association results of the multiple cameras. Set the original similarity between the target trajectory and the target trajectory in the third camera to the first value; and / or set the original similarity between the target trajectory and the trajectory to be processed in the fourth camera to the first value to obtain each associated similarity.

14. The apparatus according to claim 13, characterized in that, The device further includes: The module for determining the set of trajectories to be processed is used to determine the second set of trajectories obtained by association after obtaining the first number of target trajectories, as the set of trajectories to be processed; The training module is used to train the spatiotemporal transfer network corresponding to the camera to which each trajectory belongs for each set of trajectories to be processed. The module takes one trajectory in the set as input data and the duration between the other trajectories in the set and the time period corresponding to the trajectory as output data.

15. The apparatus according to claim 10, characterized in that, The device further includes: The feature value calculation module is used to calculate the feature values ​​of the similarity matrix containing the association similarity after obtaining the similarity between the image features of every two trajectories in each target trajectory as the association similarity. The first number determination module is used to determine the number of feature values ​​greater than the third threshold, which is taken as the first number.

16. The apparatus according to claim 10, characterized in that, The target trajectory acquisition module includes: The raw trajectory acquisition submodule is used to acquire the raw trajectories of the target captured by multiple cameras. The target video frame determination submodule is used to determine, for each original trajectory, the video frame in the video image corresponding to which the target exchange occurs, as the target video frame, according to a first detection method and / or a second detection method; wherein, the first detection method determines the target video frame based on the orientation of the target in the video image; the second detection method determines the target video frame based on the average level of similarity of image features of each video frame contained in the video image; The segmentation submodule is used to segment the original trajectory according to the position of the target video frame in the video image to obtain multiple target trajectories.

17. The apparatus according to claim 16, characterized in that, The target video frame determination submodule is specifically used to input the image region corresponding to the original trajectory in each video frame of the video image into a pre-trained orientation classification network to obtain the orientation of the target to which the original trajectory belongs in the video frame. If the change between the orientation of the target to which the original trajectory belongs in the video frame and the orientation of the target to which the original trajectory belongs in the next video frame of the video image is greater than the fourth threshold, then the video frame is determined as the target video frame in the video image where the target exchange has occurred.

18. The apparatus according to claim 16, characterized in that, The target video frame determination submodule is specifically used to determine, for each video frame in the video image, the video frame and the video frame before the video frame in the video image as the first video frame, and to determine the video frame after the video frame in the video image as the second video frame. The average level of similarity between the original trajectory and the corresponding image features in the first video frame is calculated as the first average similarity; and the average level of similarity between the original trajectory and the corresponding image features in the second video frame is calculated as the second average similarity. The video frame with the largest sum of the first average similarity and the second average similarity is identified as the target video frame in which the target exchange occurred in the video image.

19. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-9.

20. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-9.

Images