Space-time multi-source data target association method and device, electronic equipment and storage medium

Abstract

The application provides a kind of spatiotemporal multi-source data target association method, device, electronic equipment and storage medium, first, each single-source time series data is carried out single-source target association, then when the local similarity of target pair is less than the preset threshold, the topological structure similarity of target pair is calculated by the topological structure characteristics of each target, and the topological structure similarity of target pair is combined to carry out multi-source target association to the single-frame data of each data source at any time;Finally, according to the first association result obtained by single-source target association and the second association result obtained by multi-source target association, the target association result of each data source is determined.The method can solve the problem of insufficient accuracy caused by using hierarchical association method under the condition that the formation structure of multiple data sources is incomplete, and by introducing the topological relationship association method of topological structure characteristics, the target association result can be more accurate, the phenomenon of target association result failure can be avoided, and the association efficiency can be improved.

Description

Technical Field

[0001] This invention relates to the field of multi-modal fusion technology, and in particular to a method, apparatus, electronic device and storage medium for spatiotemporal multi-source data target association. Background Technology

[0002] Object detection is a fundamental and crucial technology in the field of computer vision. In recent years, it has received widespread attention and has been widely applied in various industries such as intelligent video surveillance, smart transportation, robot vision navigation, medical diagnosis, and autonomous driving.

[0003] Since the image information acquired by a single sensor has its own limitations and biases, such as radar sensors being unable to acquire detailed information about the target, and optical sensors having poor adaptability to the environment, neither can meet the target detection requirements under actual complex scene conditions. Therefore, multi-source data fusion is required, and the key to multi-source data fusion is to complete the target association of multi-source data.

[0004] Currently, target association in multi-source data can be achieved using a hierarchical association method based on target structural features. This method relies on the target's nearest neighbors to establish local structural features and uses a global benchmark to establish global structural features. While simple and easy to implement, this method lacks accuracy and is prone to failure when faced with significant interference.

[0005] Therefore, there is an urgent need to provide a spatiotemporal multi-source data target association method. Summary of the Invention

[0006] This invention provides a spatiotemporal multi-source data target association method, apparatus, electronic device, and storage medium to address the shortcomings of low target association accuracy in existing technologies.

[0007] This invention provides a spatiotemporal multi-source data target association method, comprising:

[0008] Obtain single-source time series data from each data source to be associated, extract the location information of each target in each single-source time series data, and perform single-source target association for each single-source time series data based on the location information of each target.

[0009] For any single frame of data from each data source at any given time, determine the local positional relationship features of each target in each single frame of data, and calculate the local similarity of each target pair in the single frame of data from each data source at any given time based on the local positional relationship features of each target.

[0010] If the local similarity of the target pair is less than a preset threshold, the topological structure features of the target pair in each single frame of data are determined. Based on the topological structure features of each target in the target pair, the topological structure similarity of the target pair is calculated. Based on the topological structure similarity of the target pair, multi-source target association is performed on the single frame data of each data source at any time.

[0011] Based on the first association result obtained from single-source target association and the second association result obtained from multi-source target association, the target association result of each data source is determined.

[0012] According to the spatiotemporal multi-source data target association method provided by the present invention, the step of determining the target association result of each data source based on the first association result obtained from single-source target association and the second association result obtained from multi-source target association specifically includes:

[0013] For any given moment, determine the corresponding target in the single-frame data of each data source at any given moment in the first association result corresponding to the same target in the first association result of each data source at any given moment, and determine whether the second association result corresponding to the same target in the single-frame data of each data source at any given moment is consistent with the corresponding target;

[0014] If the number of times when the judgment result is consistent is greater than a preset value, the Hungarian algorithm is used to determine the target association result of each data source corresponding to the same target.

[0015] According to a spatiotemporal multi-source data target association method provided by the present invention, the method further includes calculating the local similarity of each target pair in a single frame of data from each data source at any given time based on the local positional relationship features of each target, and then further comprising:

[0016] If the local similarity of the target pair is greater than or equal to the preset threshold, then based on the two reference target pairs with high local similarity in the single frame data of each data source, the global positional relationship features of the target pair in each single frame data are determined.

[0017] Based on the global positional relationship features of each target in the target pair, the global similarity of the target pair is calculated, and multi-source target association is performed on the single-source time series data of each data source according to the global similarity of the target pair.

[0018] According to the spatiotemporal multi-source data target association method provided by the present invention, the step of determining the global positional relationship features of the targets in each single frame of data based on two reference target pairs with high local similarity in single frame data from each data source includes:

[0019] Using each reference target in the two reference target pairs as a global structural reference point in the corresponding single frame data, the global positional relationship features of the targets in each single frame data are determined based on the distance between the reference target in each single frame data and the corresponding global structural reference point, and the angle between the line direction connecting the reference target in each single frame data and the line direction connecting the two global structural reference points.

[0020] According to the spatiotemporal multi-source data target association method provided by the present invention, the method further includes determining the global positional relationship features of the target in each single frame of data based on the distance between the reference target and the corresponding global structural reference point in each single frame of data, and the angle between the line direction connecting the reference target and the corresponding global structural reference point in each single frame of data and the line direction connecting the two global structural reference points.

[0021] Calculate the category attribute features of the target pair in each single frame of data;

[0022] Based on the category attribute features of the target in each single frame of data, the distance between the reference target and the corresponding global structural reference point in each single frame of data, and the angle between the line direction connecting the reference target and the corresponding global structural reference point in each single frame of data and the line direction connecting the two global structural reference points, the global positional relationship features of the target in each single frame of data are determined.

[0023] According to the spatiotemporal multi-source data target association method provided by the present invention, the step of performing single-source target association on each single-source time series data based on the location information of each target specifically includes:

[0024] Based on the location information of each target, a Kalman filter algorithm is used to perform single-source target association on each single-source time series data to determine the first association result in each single-source time series data.

[0025] According to the spatiotemporal multi-source data target association method provided by the present invention, the step of performing multi-source target association on single-frame data of each data source at any given time based on the topological similarity of the target pairs specifically includes:

[0026] Based on the topological similarity of the target pairs, the Hungarian algorithm is used to perform multi-source target association on single-frame data from each data source at any given time, and the second association result is determined.

[0027] The present invention also provides a spatiotemporal multi-source data target association device, comprising:

[0028] The single-source target association module is used to acquire single-source time series data from each data source to be associated, extract the location information of each target in each single-source time series data, and perform single-source target association on each single-source time series data based on the location information of each target.

[0029] The local similarity calculation module is used to determine the local positional relationship features of each target in each single frame of data from each data source at any given time, and to calculate the local similarity of each target pair in the single frame of data from each data source at any given time based on the local positional relationship features of each target.

[0030] The multi-source target association module is used to determine the topological structure features of each target in the target pair in each single frame of data if the local similarity of the target pair is less than a preset threshold, calculate the topological structure similarity of the target pair based on the topological structure features of each target in the target pair, and perform multi-source target association on the single-source time series data of each data source according to the topological structure similarity of the target pair.

[0031] The target association result determination module is used to determine the target association result of each data source based on the first association result obtained from single-source target association and the second association result obtained from multi-source target association.

[0032] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the spatiotemporal multi-source data target association method as described above.

[0033] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the spatiotemporal multi-source data target association method as described above.

[0034] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the spatiotemporal multi-source data target association method as described above.

[0035] The spatiotemporal multi-source data target association method, apparatus, electronic device, and storage medium provided by this invention first perform single-source target association on each single-source time series data. Then, when the local similarity of a target pair is less than a preset threshold, the topological similarity of the target pair is calculated based on the topological features of each target. Combining the topological similarity of the target pair, multi-source target association is performed on single-frame data from each data source at any given time. Finally, based on the first association result obtained from the single-source target association and the second association result obtained from the multi-source target association, the target association result for each data source is determined. This method can solve the problem of insufficient accuracy caused by using hierarchical association methods when the formation structure of multiple data sources is incomplete. By introducing a topological relationship association method based on topological features, the target association results can be made more accurate, avoiding the phenomenon of target association result failure, and also improving association efficiency. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on the drawings described below without creative effort.

[0037] Figure 1 This is one of the flowcharts illustrating the spatiotemporal multi-source data target association method provided by the present invention;

[0038] Figure 2 This is a schematic diagram illustrating the relationship between the target and the corresponding first key target and second key target in a single frame of data from any data source at any time provided by the present invention.

[0039] Figure 3 This is a schematic diagram of the relative positional relationships between targets in a single frame of data collected at any given time from a data source with a complete formation configuration provided by the present invention.

[0040] Figure 4 This is a schematic diagram of the relative positional relationships between targets in a single frame of data collected at any given moment from a data source with an incomplete formation configuration, as provided by this invention.

[0041] Figure 5 This is a schematic diagram of the polar coordinate system constructed with the target Pi as the center, provided by the present invention;

[0042] Figure 6 This is a schematic diagram illustrating the positional relationship between a target in a single frame of data and two global structural reference points 1 and 2, provided by the present invention.

[0043] Figure 7This is the second flowchart of the spatiotemporal multi-source data target association method provided by the present invention;

[0044] Figure 8 This is a schematic diagram of the spatiotemporal multi-source data target correlation device provided by the present invention;

[0045] Figure 9 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0047] Object detection is a fundamental and crucial technology in the field of computer vision. In recent years, it has received widespread attention and has been widely applied in various industries such as intelligent video surveillance, smart transportation, robot vision navigation, medical diagnosis, and autonomous driving.

[0048] Taking target detection in the security industry as an example, radar sensors or optical sensors are commonly used. These sensors move at a certain speed and collect information about the target area at predetermined speeds and time intervals. Because radar sensors cannot acquire detailed information about the target, and optical sensors have poor environmental adaptability, they cannot be used alone to detect and identify targets. Multi-source data fusion is necessary, and the key to multi-source data fusion is to establish target correlation among the multiple data sources.

[0049] Target association in multi-source data can be achieved using a hierarchical association method based on target structural features. This method relies on the target's nearest neighbors to establish local structural features and uses a global benchmark to establish global structural features. While simple and easy to implement, this method lacks accuracy and is prone to failure when faced with significant interference.

[0050] Another approach is to use a topological association method based on the target's topology. First, a topological coordinate system centered on the target is established based on polar coordinates. Then, the topological features of the target are generated by using the positions of other targets in the topological coordinate system. Finally, the similarity of the topological features of the heterogeneous data is calculated. This method can effectively combat interference, but it cannot distinguish between targets that are close to each other.

[0051] Alternatively, target association can be performed directly using single-source sequence data. This involves first obtaining the association trajectory of single-source sequence data using Kalman filtering, and then calculating the similarity of heterogeneous data by summing sequence structure features. However, this method is highly susceptible to the influence of a single erroneous data link in the sequence.

[0052] Therefore, improving the accuracy of target association in multi-source data fusion and realizing the rapid transformation of perceived data into information has become a new challenge. To address this, this invention provides a spatiotemporal multi-source data target association method.

[0053] Figure 1 This is a flowchart illustrating a spatiotemporal multi-source data target association method provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes:

[0054] S1, obtain single-source time series data of each data source to be associated, extract the location information of each target in each single-source time series data, and perform single-source target association for each single-source time series data based on the location information of each target;

[0055] S2, for any single frame of data from each data source at any given time, determine the local positional relationship features of each target in each single frame of data, and calculate the local similarity of each target pair in the single frame of data from each data source at any given time based on the local positional relationship features of each target.

[0056] S3, if the local similarity of the target pair is less than a preset threshold, then determine the topological structure features of each target in the target pair in each single frame of data, calculate the topological structure similarity of the target pair based on the topological structure features of each target in the target pair, and perform multi-source target association on the single frame data of each data source at any time according to the topological structure similarity of the target pair.

[0057] S4. Based on the first association result obtained from single-source target association and the second association result obtained from multi-source target association, determine the target association result of each data source.

[0058] Specifically, the spatiotemporal multi-source data target association method provided in this embodiment of the invention is executed by a spatiotemporal multi-source data target association device. This device can be configured in a server, which can be a local server or a cloud server. The local server can be a computer, etc., and this embodiment of the invention does not make specific limitations on this.

[0059] Here, spatiotemporal multi-source data target association refers to associating targets from multiple data sources in time and space, that is, determining which targets in the data from multiple data sources belong to the same target. Data sources are typically data acquisition devices, which can be different types of sensors; no specific limitation is made here. The number of data sources can be set as needed, and there can be multiple data sources, including both data sources of the same type and data sources of different types.

[0060] In this embodiment of the invention, the data source can be an image data acquisition device, such as an optical sensor, radar sensor, or other sensors. Furthermore, the target can be an object to be identified in the image data, and there can be multiple targets. Taking the security industry as an example, the targets can be at least two types of equipment involving land, sea, and air, such as ocean-going ships, airport aircraft, and mobile vehicles, etc., without specific limitations here.

[0061] First, execute step S1 to obtain the single-source time series data of each data source to be associated. The data collected from each data source is a single-source time series data, that is, the data is obtained by arranging the data in chronological order. The data sources to be associated refer to multiple different data sources whose single-source time series data need to be associated with the target.

[0062] Next, the location information of each target in each single-source time series data is extracted. Each single-source time series data includes single-frame data from multiple time points, and each frame contains multiple targets. Therefore, the location information of each target in each frame can be determined, i.e., the location information of each target in each single-source time series data. The location information of each target can be represented by the coordinates of each target in each frame of data.

[0063] Subsequently, based on the location information of each target in each frame of data, single-source target association can be performed on each single-source time series data. This involves locating and tracking each target in different frames of the single-source time series data, thereby achieving target association. Conventional localization and tracking algorithms can be used here, and no specific limitations are specified.

[0064] Then, step S2 is executed. For any given moment, each single frame of data from different data sources (i.e., single frames of data from different single-source time series data at the same moment) determines the local positional relationship features of each target in each single frame. In each single frame, each target corresponds to a local positional relationship feature. This feature can be used to characterize the positional relationship between the target and its corresponding key target in the single frame. This positional relationship can be represented by the distance between the target and its corresponding key target, as well as the angle between the connecting line and the horizontal baseline.

[0065] For any single frame of data, the key targets corresponding to a certain target in that single frame may include a first key target that is closest to the target and a second key target that is farthest from the target. Accordingly, the local positional relationship features of the target can be represented by a first distance between the target and the corresponding first key target, a second distance between the target and the corresponding second key target, a first angle between the direction of the line connecting the target and the corresponding first key target and the direction of the horizontal baseline, and a second angle between the direction of the line connecting the target and the corresponding second key target and the direction of the horizontal baseline. That is:

[0066]

[0067] in, For any given moment, target C is a single frame of data from data source O1. i The local positional relationship characteristics, For any given moment, target C is a single frame of data from data source O1. i To the corresponding first key objective distance, For any given moment, target C is a single frame of data from data source O1. i and the corresponding first key objective The first angle between the direction of the line connecting the two lines and the direction of the horizontal baseline; For any given moment, target C is a single frame of data from data source O1. i To the corresponding second key objective distance, For any given moment, target C is a single frame of data from data source O1. i and the corresponding second key objective The second angle between the direction of the line connecting the two points and the direction of the horizontal baseline.

[0068] In a single frame of data from data source O1 at any given time, the target C i With the corresponding first key objective Second key objective The relationship between them is as follows Figure 2 As shown.

[0069] Furthermore, by analyzing the local positional relationship features of each target in each single frame of data at any given time, the local similarity of each target pair in single frames of data from different data sources at any given time can be calculated. A target pair can consist of one target from a single frame of data from one data source and one target from a single frame of data from another data source at the same time. The local similarity of a target pair can be calculated by the distance between the local positional relationship features of the two targets in the target pair. The distance between the local positional relationship features of the two targets can be Mahalanobis distance, Euclidean distance, etc.

[0070] In a single frame of data from data source O1 at any given time, the target C i The target E in a single frame of data from data source O2 at any given time j The local similarity S of the target pair ij That is, in a single frame of data from data source O1 at any given time, the target C i The target E in a single frame of data from data source O2 at any given time j The distance between the local positional relationship features can be expressed as:

[0071]

[0072] in, For any given moment, the target E in a single frame of data from data source O2 j The local positional relationship characteristics are different between data source O1 and data source O2.

[0073] Understandably, when the number of targets in each frame of data collected by different data sources at any given time is the same, and matches the actual number of targets, it indicates that the formation configuration of the different data sources is complete. In this case, the relative positional relationship of the targets in each frame of data collected by different data sources at any given time is accurate and consistent with reality. However, when the number of targets in each frame of data collected by different data sources at any given time is less than the actual number of targets, it indicates that the formation configuration of the different data sources is incomplete. In this case, the relative positional relationship of the targets in each frame of data collected by different data sources at any given time deviates from reality.

[0074] Figure 3 This is a schematic diagram showing the relative positional relationships between targets in a single frame of data collected at any given time from a data source with a complete formation configuration. Figure 4 This is a schematic diagram showing the relative positional relationships between targets in a single frame of data collected at any given time from a data source with an incomplete formation configuration.

[0075] Figure 3 The local positional relationship features of a target object can be represented as (d1, α1, d2, α2), where d1 is the distance from the target object to the corresponding first key target, α1 is the first angle between the direction of the line connecting the target object and the corresponding first key target and the direction of the horizontal baseline; d2 is the distance from the target object to the corresponding second key target, and α2 is the second angle between the direction of the line connecting the target object and the corresponding second key target and the direction of the horizontal baseline.

[0076] Figure 4The target in the dashed box is the target that appears in a single frame of data collected by a certain data source at any time. This is only a part of the actual target, and the relative positional relationship of each target in this single frame of data cannot accurately determine the local positional relationship characteristics of the target.

[0077] Therefore, in this embodiment of the invention, it is necessary to determine whether the data source has a complete formation configuration by comparing the local similarity of each target pair with a preset threshold. This preset threshold can be set as needed and is not specifically limited here. If the local similarity of a target pair is less than the preset threshold, the formation configuration of each data source is considered incomplete, and step S3 is executed to achieve target association by calculating the topological similarity of the topological features of each target pair. Otherwise, the formation configuration of each data source is considered complete, and target association can be achieved by continuing to calculate the global similarity of the global positional relationship features of each target pair using a hierarchical association method, which is not specifically limited here.

[0078] In step S3, if the local similarity of the target pair is less than a preset threshold, the topological features of the targets in each single frame of data are determined. Each target pair includes two targets. The topological features of each target can be used to characterize the positional relationship between the target and the remaining targets in the single frame of data. This positional relationship can be represented by the spatial position between the target and the remaining targets in polar coordinates. The topological features of each target can be represented in matrix form.

[0079] Here, a polar coordinate system can be established with the target as the center, and the surrounding space can be divided into M×N cells, where N is the number of sectors and M is the number of annular regions. Each cell is assigned the value u. ij , where i is the radial index and j is the circumferential index. Through u ij The value of indicates whether the target exists in the cell, that is:

[0080]

[0081] Based on this, the topological features of the target can be quantified, thereby obtaining the topological features.

[0082] It is understandable that the topological features of the target can be represented by a topological matrix. A topological matrix can be constructed first, and its elements initialized to 0. The elements in the topological matrix correspond one-to-one with the cells obtained from the surrounding spatial division. Therefore, the values ​​assigned to these cells can be used as the values ​​of the corresponding elements in the topological matrix to update the topological matrix.

[0083] Figure 5In this embodiment of the invention, a polar coordinate system is constructed with target Pi as the center, and targets A, B, C, D, and E are included around it. The topological features of target Pi can be represented as follows:

[0084]

[0085] Subsequently, the topological similarity of the target pair can be calculated based on the topological features of each target. The topological similarity of the target pair can be calculated by the distance between the topological features of the two targets in the target pair. The distance between the topological features of the two targets can be Mahalanobis distance, Euclidean distance, etc.

[0086] The topological similarity between a target pair formed by a target in a single frame of data source O1 at any given time and a target in a single frame of data source O2 at any given time. The distance between the topological features of a target in a single frame of data from data source O1 at any given time and a target in a single frame of data from data source O2 at any given time can be expressed as:

[0087]

[0088] in, For any given moment, target C is a single frame of data from data source O1. i Topological features, For any given moment, the target E in a single frame of data from data source O2 j The topological structure features are given, where m is the number of targets observed by data source O1 and n is the number of targets observed by data source O2.

[0089] Furthermore, multi-source target association can be performed on single-source time series data from various data sources based on the topological similarity of target pairs. Here, target pairs with a topological similarity greater than a first threshold can be directly considered as the same target, thereby achieving target association in single-source time series data from various data sources.

[0090] Finally, step S4 is executed to determine the final target association result of the multi-source data based on the first association result obtained from the single-source target association and the second association result obtained from the multi-source target association.

[0091] Taking a data source with two data sources, namely the first data source and the second data source, as an example, the first association result of the first data source can be represented as: Represents the target in the k-th single frame of data from the first data source. All of these represent the same target, namely, the i1th target in the first single frame of the single-source time series data from the first data source, the i2th target in the second single frame, ..., the i1th target in the Kth single frame. Kassociated with a target, where 0 < i1 < m, and K is the number of single-frame data in each single-source time series data. The first association result of the second data source can be expressed as: represents the target in the k-th single-frame data of the second data source both represent the same target, that is, the j1-th target in the 1st single-frame data, the j2-th target in the 2nd single-frame data,..., the j K associated with a target, where 0 < j1 < n.

[0092] The second association result of the first data source and the second data source at the k-th moment can be expressed as: represents the target in the k-th single-frame data of the first data source and the target in the k-th single-frame data of the second data source represent the same target.

[0093] By fusing the first association result obtained from single-source target association and the second association result obtained from multi-source target association, the target association result of the final multi-source data can be determined. The fusion method can be set as needed. For example, according to the number of times the second association result at any moment is consistent with the first association results of each data source, the same target in the multi-source data can be comprehensively determined to obtain the target association result of the multi-source data.

[0094] In the spatio-temporal multi-source data target association method provided in the embodiments of the present invention, first, single-source target association is performed on each single-source time series data. Then, when the local similarity of the target pair is less than the preset threshold, the topological structure similarity of the target pair is calculated through the topological structure features of each target. Combining the topological structure similarity of the target pair, multi-source target association is performed on the single-frame data of each data source at any moment; finally, according to the first association result obtained from single-source target association and the second association result obtained from multi-source target association, the target association result of each data source is determined. This method can solve the problem of insufficient accuracy caused by using the hierarchical association method when the formation structures of multiple data sources are incomplete. By introducing the topological relationship association method of topological structure features, not only can the target association result be more accurate, avoiding the phenomenon of invalid target association results, but also the association efficiency can be improved.

[0095] Based on the above embodiments, in the spatio-temporal multi-source data target association method provided in the embodiments of the present invention, determining the target association result of each data source according to the first association result obtained from single-source target association and the second association result obtained from multi-source target association specifically includes:

[0096] For any given moment, determine the corresponding target in the single-frame data of each data source at any given moment in the first association result corresponding to the same target in the first association result of each data source at any given moment, and determine whether the second association result corresponding to the same target in the single-frame data of each data source at any given moment is consistent with the corresponding target;

[0097] If the number of times when the judgment result is consistent is greater than a preset value, the Hungarian algorithm is used to determine the target association result of each data source corresponding to the same target.

[0098] Specifically, in this embodiment of the invention, when determining the target association result of multi-source data, for any time k (1≤k≤K) in the single-source time series data of each data source, the corresponding target in the single frame data of any time k in the first association result of each data source corresponding to the same target is determined.

[0099] For example, the first association result corresponding to the same target Z from the first data source is: In this first association result, the corresponding target in the single frame data at any time k is The first association result of the second data source corresponding to the same target Z is: In this first association result, the corresponding target in the single frame data at any time k is

[0100] Then, determine the second association result of the first data source and the second data source corresponding to the same target Z at any time k. as well as Whether they are consistent. For example, if at any time k the second association result of the first data source and the second data source corresponds to the same target Z. If they match, then they are consistent; otherwise, they are inconsistent.

[0101] Finally, all moments with consistent results are identified, and the number of such moments is counted. If the total number of moments exceeds a preset value, the association with the same target Z is considered successful. In this case, the Hungarian algorithm can be used to determine the target association results for each data source corresponding to the same target Z. The preset value can be set as needed and is not specifically limited here.

[0102] It is understood that, in this embodiment of the invention, to simplify the above process, a fusion matrix can be constructed first based on the number of targets observed from each data source. This fusion matrix can be represented as:

[0103] SM=0 m×n

[0104] Where m is the number of targets observed by the first data source, and n is the number of targets observed by the second data source. All elements in this fusion matrix have an initial value of 0.

[0105] Subsequently, for any time k, if the first data source and the second data source correspond to the same target Z, and the corresponding target of this same target Z in a single frame of data from the first data source is the i-th target, and the corresponding target of this same target Z in a single frame of data from the second data source is the j-th target, then if the second association result of the first data source and the second data source corresponding to the same target Z at any time k is... Then we have:

[0106] SM ij =SM ij +1

[0107] The fusion matrix is ​​continuously updated through the above process.

[0108] Subsequently, for any element SM in the fusion matrix ij Determine SM ij The relationship between SM and the preset value, if SM ij If the value is greater than the preset value, then the association with the same target Z is considered successful.

[0109] In this embodiment of the invention, a judgment condition is introduced. When the judgment condition is met, the Hungarian algorithm is used to determine the target association results corresponding to the same target from each data source, which can further ensure the accuracy of the target association results.

[0110] Based on the above embodiments, the spatiotemporal multi-source data target association method provided in this embodiment of the invention calculates the local similarity of each target pair in a single frame of data from each data source at any given time based on the local positional relationship features of each target, and then further includes:

[0111] If the local similarity of the target pair is greater than or equal to the preset threshold, then based on the two reference target pairs with high local similarity in the single frame data of each data source, the global positional relationship features of the target pair in each single frame data are determined.

[0112] Based on the global positional relationship features of each target in the target pair, the global similarity of the target pair is calculated, and multi-source target association is performed on the single-source time series data of each data source according to the global similarity of the target pair.

[0113] Specifically, in this embodiment of the invention, after calculating the local similarity of each target pair in a single frame of data from each data source at any given time, if the local similarity of the target pair is greater than or equal to a preset threshold, the hierarchical association method can be used to calculate the global similarity of the global positional relationship features of each target pair to achieve target association.

[0114] If the local similarity of a target pair is greater than or equal to a preset threshold, first determine the two reference target pairs with high local similarity in a single frame of data from each data source at any given time. The target pairs can be sorted in descending order of local similarity, and then the top two from the sorted results are selected as reference target pairs.

[0115] Then, using two reference target pairs, the global positional relationship characteristics of the targets in each single frame of data are determined. In each single frame of data at any given time, there are two reference targets, and the global positional relationship characteristics of the targets in each single frame of data at any given time can be characterized by the orientation and distance information between the two reference targets and the targets.

[0116] Furthermore, based on the global positional relationship features of each target in the target pair, the global similarity of the target pair is calculated. The global similarity of the target pair can be calculated by the distance between the global positional relationship features of the two targets in the target pair. The distance between the global positional relationship features of the two targets can be Mahalanobis distance, Euclidean distance, etc.

[0117] The global similarity between a target pair formed by a target in a single frame of data source O1 at any given time and a target in a single frame of data source O2 at any given time. The distance between the global positional relationship features of a target in a single frame of data source O1 at any given time and a target in a single frame of data source O2 at any given time can be expressed as:

[0118]

[0119] in, For any given moment, target C is a single frame of data from data source O1. i Global positional relationship features, For any given moment, the target E in a single frame of data from data source O2 j The global positional relationship characteristics.

[0120] Finally, based on the global similarity of the target pairs, multi-source target association is performed on the single-source time series data from each data source. The Hungarian algorithm can be used here; specific implementation details are not provided.

[0121] In this embodiment of the invention, after calculating the local similarity of each target pair, when the local similarity of the target pair is greater than or equal to a preset threshold, the global positional relationship features of the targets in each single frame of data are further determined using a hierarchical association method. The global similarity of the target pair, calculated using the global positional relationship features of each target in the pair, is then used to achieve multi-source target association for single-source time-series data from various data sources. This method is simple and easy to implement, and it also ensures the accuracy of multi-source target association even when the local similarity of the target pair is greater than or equal to the preset threshold.

[0122] Based on the above embodiments, the spatiotemporal multi-source data target association method provided in this embodiment of the invention, wherein the step of determining the global positional relationship features of the targets in each single frame of data based on two reference target pairs with high local similarity in single frame data from each data source includes:

[0123] Using each reference target in the two reference target pairs as a global structural reference point in the corresponding single frame data, the global positional relationship features of the targets in each single frame data are determined based on the distance between the reference target in each single frame data and the corresponding global structural reference point, and the angle between the line direction connecting the reference target in each single frame data and the line direction connecting the two global structural reference points.

[0124] Specifically, in this embodiment of the invention, when determining the global positional relationship features of the targets in each single frame of data, each reference target in the two reference target pairs can be used as a global structural reference point in the corresponding single frame of data. Distance information is represented by the distance value between the reference target and the corresponding global structural reference point in each single frame of data, and azimuth information is represented by the angle between the line direction connecting the reference target and the corresponding global structural reference point in each single frame of data and the line direction connecting the two global structural reference points. The global positional relationship features of the targets in each single frame of data are determined using the distance value and the angle.

[0125] Figure 6 This is a schematic diagram illustrating the positional relationship between a target in a single frame of data and two global structural reference points 1 and 2. (Example:) Figure 6 As shown, the global positional relationship features of a target in a single frame of data can be represented as:

[0126]

[0127] in, Let i be the distance from the target point i to the reference point 1. Let be the angle between the direction of the line connecting target i and reference point 2 and the direction of the line connecting reference point 1 and reference point 2; Let i be the distance from the target point i to the reference point 2. Let be the angle between the direction of the line connecting target i and reference point 2 and the direction of the line connecting reference point 1 and reference point 2. Figure 6 middle,

[0128] In this embodiment of the invention, the global positional relationship features of the target are determined by using two reference targets in a single frame of data, which makes the obtained global positional relationship features of the target simpler and easier to express.

[0129] Based on the above embodiments, the spatiotemporal multi-source data target association method provided in this embodiment of the invention further includes determining the global positional relationship features of the target in each single frame of data based on the distance value between the reference target and the corresponding global structural reference point in each single frame of data, and the angle between the line direction connecting the reference target and the corresponding global structural reference point in each single frame of data and the line direction connecting the two global structural reference points.

[0130] Calculate the category attribute features of the target pair in each single frame of data;

[0131] Based on the category attribute features of the target in each single frame of data, the distance between the reference target and the corresponding global structural reference point in each single frame of data, and the angle between the line direction connecting the reference target and the corresponding global structural reference point in each single frame of data and the line direction connecting the two global structural reference points, the global positional relationship features of the target in each single frame of data are determined.

[0132] Specifically, in this embodiment of the invention, when determining the global positional relationship features of the targets in each single frame of data, the category attribute features of the targets can also be introduced into the global positional relationship features. That is, the category attribute features of the targets in each single frame of data are first calculated. These category attribute features can be used to characterize the category attributes of the targets. For example, the category attribute features can be represented as:

[0133]

[0134] Among them, l i Let i be the category attribute features of target i. This represents the probability that target i belongs to class S.

[0135] Furthermore, the global position relationship feature of the target pair in each single-frame data can be determined by the category attribute feature of the target in the target pair in each single-frame data, the distance value between the reference target in each single-frame data and the corresponding global structure reference point, and the angle between the connection direction of the reference target in each single-frame data and the corresponding global structure reference point and the connection direction of the two global structure reference points. At this time, the global position relationship feature of target i can be expressed as:

[0136]

[0137] In the embodiments of the present invention, introducing the category attribute feature of the target into the global position relationship feature can make the subsequent calculation of the global similarity more accurate and can grasp the category attribute of the target in real time.

[0138] Based on the above embodiments, in the spatio-temporal multi-source data target association method provided in the embodiments of the present invention, for each single-source time series data, single-source target association is performed based on the position information of each target, specifically including:

[0139] Based on the position information of each target, the Kalman filter algorithm is used to perform single-source target association on each single-source time series data to determine the first association result in each single-source time series data.

[0140] Specifically, in the embodiments of the present invention, when performing single-source target association on each single-source time series data, it can be realized by combining the position information of each target with the Kalman filter algorithm.

[0141] Assume that each single-source time series data includes a total of K single-frame data, and the kth single-frame data numbered in time sequence is denoted as C k , 1 < k < K, and n k targets are detected from this single-frame data, denoted as First, perform association matching on these K single-frame data. Assume that the position information of each target obtained from each single-frame data has corrected the system error and has been transformed and aligned in time and space coordinates and unified into the same spatio-temporal coordinate system. After being unified into the same spatio-temporal coordinate system, the following method is used to implement single-source target association:

[0142] Step 1: Since the initial motion state of the target cannot be obtained from each single-frame data, therefore, first perform target-to-target association matching on the first two single-frame data C 1 and C 2 . Perform association matching on the targets and <^ detected in the first two single-frame data.A target formation structure-based association matching method is used for target association matching. The position information of each target in the first two frames of data is obtained from the association matching results. The position information of each target in the first two frames of data is substituted into the Kalman filter for model filtering initialization to obtain the motion prediction value, motion estimation error and estimated covariance matrix for the next time step.

[0143] The prediction formula for the Kalman filter is:

[0144]

[0145]

[0146] The update formula for the Kalman filter is:

[0147]

[0148]

[0149]

[0150] Step 2: Perform target association matching by obtaining the predicted motion value of the target at the next moment and the observed value of the target at the next moment. The association matching method based on the target formation structure features is used for association matching. The association matching results are used to obtain the observed values ​​of the corresponding targets at the next moment and the previous moment.

[0151] Step 3: Substitute the observed values, predicted values, and estimated covariance matrix corresponding to the target updated in Step 2 into the Kalman filter for update prediction filtering to obtain the motion prediction value, motion estimation error, and estimated covariance matrix at the next time step.

[0152] Step 4: If all data has been processed, end the sequence filtering association;

[0153] If new observation data is available, substitute the new observation data, motion prediction values, motion estimation errors, and estimated covariance matrices obtained in step 3 into step 2 and repeat the process.

[0154] In this embodiment of the invention, the Kalman filter algorithm is used to achieve single-source target association, which can ensure the accuracy of the first association result.

[0155] Based on the above embodiments, the spatiotemporal multi-source data target association method provided in this embodiment of the invention, wherein the multi-source target association is performed on single-frame data of each data source at any given time based on the topological similarity of the target pairs, specifically includes:

[0156] Based on the topological similarity of the target pairs, the Hungarian algorithm is used to perform multi-source target association on single-frame data from each data source at any given time, and the second association result is determined.

[0157] Specifically, in this embodiment of the invention, the Hungarian algorithm can be used to perform multi-source target association, which can reduce the difficulty of multi-source target association.

[0158] Figure 7 This is a schematic diagram illustrating the complete process of the spatiotemporal multi-source data target association method provided in this embodiment of the invention, such as... Figure 7 As shown, the method includes:

[0159] S10, obtain the single-source time series data of each data source to be associated, and extract the location information of each target in each single-source time series data;

[0160] S20, based on the location information of each target, perform single-source target association for each single-source time series data;

[0161] S30, for single-frame data from each data source at any given time, determine the local positional relationship features of each target in each single-frame data, and calculate the local similarity of each target pair in the single-frame data from each data source at any given time based on the local positional relationship features of each target.

[0162] S40, if the local similarity of the target pair is less than a preset threshold, then determine the topological structure features of the target pair in each single frame of data, calculate the topological structure similarity of the target pair based on the topological structure features of each target in the target pair, and perform multi-source target association on the single frame data of each data source at any time according to the topological structure similarity of the target pair.

[0163] S50, if the local similarity of the target pair is greater than or equal to the preset threshold, then based on the two reference target pairs with high local similarity in the single frame data of each data source, determine the global positional relationship features of the target pair in each single frame data.

[0164] Based on the global positional relationship features of each target in a target pair, the global similarity of the target pair is calculated, and multi-source target association is performed on the single-source time series data of each data source according to the global similarity of the target pair.

[0165] S60, based on the first association result obtained from single-source target association and the second association result obtained from multi-source target association, determine the target association result of each data source.

[0166] like Figure 8 As shown, based on the above embodiments, this embodiment of the invention provides a spatiotemporal multi-source data target association device, including:

[0167] The single-source target association module 81 is used to acquire single-source time series data of each data source to be associated, extract the location information of each target in each single-source time series data, and perform single-source target association on each single-source time series data based on the location information of each target.

[0168] The local similarity calculation module 82 is used to determine the local positional relationship features of each target in each single frame of data from each data source at any given time, and to calculate the local similarity of each target pair in the single frame of data from each data source at any given time based on the local positional relationship features of each target.

[0169] The multi-source target association module 83 is used to determine the topological structure features of each target in the target pair in each single frame of data if the local similarity of the target pair is less than a preset threshold, calculate the topological structure similarity of the target pair based on the topological structure features of each target in the target pair, and perform multi-source target association on the single-source time series data of each data source according to the topological structure similarity of the target pair.

[0170] The target association result determination module 84 is used to determine the target association result of each data source based on the first association result obtained from single-source target association and the second association result obtained from multi-source target association.

[0171] Based on the above embodiments, the spatiotemporal multi-source data target association device provided in this embodiment of the invention, wherein the target association result determination module is specifically used for:

[0172] For any given moment, determine the corresponding target in the single-frame data of each data source at any given moment in the first association result corresponding to the same target in the first association result of each data source at any given moment, and determine whether the second association result corresponding to the same target in the single-frame data of each data source at any given moment is consistent with the corresponding target;

[0173] If the number of times when the judgment result is consistent is greater than a preset value, the Hungarian algorithm is used to determine the target association result of each data source corresponding to the same target.

[0174] Based on the above embodiments, the spatiotemporal multi-source data target association device provided in this embodiment of the invention, wherein the multi-source target association module is further used for:

[0175] If the local similarity of the target pair is greater than or equal to the preset threshold, then based on the two reference target pairs with high local similarity in the single frame data of each data source, the global positional relationship features of the target pair in each single frame data are determined.

[0176] Based on the global positional relationship features of each target in the target pair, the global similarity of the target pair is calculated, and multi-source target association is performed on the single-source time series data of each data source according to the global similarity of the target pair.

[0177] Based on the above embodiments, the spatiotemporal multi-source data target association device provided in this embodiment of the invention, wherein the multi-source target association module is further specifically used for:

[0178] Using each reference target in the two reference target pairs as a global structural reference point in the corresponding single frame data, the global positional relationship features of the targets in each single frame data are determined based on the distance between the reference target in each single frame data and the corresponding global structural reference point, and the angle between the line direction connecting the reference target in each single frame data and the line direction connecting the two global structural reference points.

[0179] Based on the above embodiments, the spatiotemporal multi-source data target association device provided in this embodiment of the invention, wherein the multi-source target association module is further specifically used for:

[0180] Calculate the category attribute features of the target pair in each single frame of data;

[0181] Based on the category attribute features of the target in each single frame of data, the distance between the reference target and the corresponding global structural reference point in each single frame of data, and the angle between the line direction connecting the reference target and the corresponding global structural reference point in each single frame of data and the line direction connecting the two global structural reference points, the global positional relationship features of the target in each single frame of data are determined.

[0182] Based on the above embodiments, the spatiotemporal multi-source data target association device provided in this embodiment of the invention, wherein the single-source target association module is specifically used for:

[0183] Based on the location information of each target, a Kalman filter algorithm is used to perform single-source target association on each single-source time series data to determine the first association result in each single-source time series data.

[0184] Based on the above embodiments, the spatiotemporal multi-source data target association device provided in this embodiment of the invention, wherein the multi-source target association is specifically used for:

[0185] Based on the topological similarity of the target pairs, the Hungarian algorithm is used to perform multi-source target association on single-frame data from each data source at any given time, and the second association result is determined.

[0186] Specifically, the functions of each module in the spatiotemporal multi-source data target association device provided in this embodiment of the invention correspond one-to-one with the operation flow of each step in the above method-like embodiments, and the achieved effects are also the same. For details, please refer to the above embodiments, and this will not be repeated in this embodiment of the invention.

[0187] Figure 9 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 9 As shown, the electronic device may include: a processor 910, a communication interface 920, a memory 930, and a communication bus 940, wherein the processor 910, the communication interface 920, and the memory 930 communicate with each other through the communication bus 940. The processor 910 can call logical instructions in the memory 930 to execute the spatiotemporal multi-source data target association method provided in the above embodiments. The method includes: acquiring single-source time series data of each data source to be associated, extracting the position information of each target in each single-source time series data, and performing single-source target association on each single-source time series data based on the position information of each target; for single-frame data of each data source at any time, determining the local positional relationship features of each target in each single-frame data, and calculating the local similarity of each target pair in the single-frame data of each data source at any time based on the local positional relationship features of each target; if the local similarity of the target pair is less than a preset threshold, determining the topological structure features of the target pair in each single-frame data, calculating the topological structure similarity of the target pair based on the topological structure features of each target in the target pair, and performing multi-source target association on the single-frame data of each data source at any time based on the topological structure similarity of the target pair; and determining the target association result of each data source based on the first association result obtained by single-source target association and the second association result obtained by multi-source target association.

[0188] Furthermore, the logical instructions in the aforementioned memory 930 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0189] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute a spatiotemporal multi-source data target association method. The method includes: acquiring single-source time series data of each data source to be associated, extracting the location information of each target in each single-source time series data, and performing single-source target association on each single-source time series data based on the location information of each target; for a single frame of data from each data source at any given time, determining the local positional relationship features of each target in each single frame of data, and based on... Based on the local positional relationship features of each target, calculate the local similarity of each target pair in the single frame data of each data source at any given time; if the local similarity of the target pair is less than a preset threshold, determine the topological structure features of the target pair in each single frame data, calculate the topological structure similarity of the target pair based on the topological structure features of each target in the target pair, and perform multi-source target association on the single frame data of each data source at any given time according to the topological structure similarity of the target pair; determine the target association result of each data source according to the first association result obtained from the single-source target association and the second association result obtained from the multi-source target association.

[0190] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a spatiotemporal multi-source data target association method. This method includes: acquiring single-source time-series data from each data source to be associated, extracting the location information of each target in each single-source time-series data, and performing single-source target association on each single-source time-series data based on the location information of each target; for a single frame of data from each data source at any given time, determining the local positional relationship features of each target in each single frame of data, and based on the local positional relationship features of each target… The method involves calculating the local similarity of each target pair in a single frame of data from each data source at any given time. If the local similarity of the target pair is less than a preset threshold, the topological structure features of the target pair in each single frame of data are determined. Based on the topological structure features of each target in the target pair, the topological structure similarity of the target pair is calculated. Based on the topological structure similarity of the target pair, multi-source target association is performed on the single frame of data from each data source at any given time. Based on the first association result obtained from the single-source target association and the second association result obtained from the multi-source target association, the target association result of each data source is determined.

[0191] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0192] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0193] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A spatiotemporal multi-source data target association method, characterized in that, include: Obtain single-source time series data from each data source to be associated, extract the location information of each target in each single-source time series data, and perform single-source target association for each single-source time series data based on the location information of each target. For any single frame of data from each data source at any given time, determine the local positional relationship features of each target in each single frame of data, and calculate the local similarity of each target pair in the single frame of data from each data source at any given time based on the local positional relationship features of each target. The formation of each data source is determined by comparing the local similarity of each target pair with a preset threshold. If the local similarity of the target pair is less than the preset threshold, the formation of each data source is considered incomplete. The topological structure features of the targets in each single frame of data for each target pair are determined. Based on the topological structure features of each target in the target pair, the topological structure similarity of the target pair is calculated. Based on the topological structure similarity of the target pair, multi-source target association is performed on the single frame data of each data source at any given time. The incomplete formation of each data source means that the relative positional relationship of each target in the single frame data collected by different data sources at any given time deviates from the actual positional relationship. Based on the first association result obtained from single-source target association and the second association result obtained from multi-source target association, the target association result of each data source is determined.

2. The spatiotemporal multi-source data target association method according to claim 1, characterized in that, The step of determining the target association results of each data source based on the first association result obtained from single-source target association and the second association result obtained from multi-source target association specifically includes: For any given moment, determine the corresponding target in the single-frame data of each data source at any given moment in the first association result corresponding to the same target in the first association result of each data source at any given moment, and determine whether the second association result corresponding to the same target in the single-frame data of each data source at any given moment is consistent with the corresponding target; If the number of times when the judgment result is consistent is greater than a preset value, the Hungarian algorithm is used to determine the target association result of each data source corresponding to the same target.

3. The spatiotemporal multi-source data target association method according to claim 1, characterized in that, Based on the local positional relationship features of each target, the local similarity of each target pair in the single frame data of each data source at any given time is calculated, and then the method further includes: If the local similarity of the target pair is greater than or equal to the preset threshold, then based on the two reference target pairs with high local similarity in the single frame data of each data source, the global positional relationship features of the target pair in each single frame data are determined. Based on the global positional relationship features of each target in the target pair, the global similarity of the target pair is calculated, and multi-source target association is performed on the single-source time series data of each data source according to the global similarity of the target pair.

4. The spatiotemporal multi-source data target association method according to claim 3, characterized in that, The determination of the global positional relationship features of the targets in each single frame of data based on two reference target pairs with high local similarity from each data source includes: Using each reference target in the two reference target pairs as a global structural reference point in the corresponding single frame data, the global positional relationship features of the targets in each single frame data are determined based on the distance between the reference target in each single frame data and the corresponding global structural reference point, and the angle between the line direction connecting the reference target in each single frame data and the line direction connecting the two global structural reference points.

5. The spatiotemporal multi-source data target association method according to claim 4, characterized in that, The method of determining the global positional relationship features of the target in each single frame of data based on the distance between the reference target and the corresponding global structural reference point in each single frame of data, and the angle between the line direction connecting the reference target and the corresponding global structural reference point in each single frame of data and the line direction connecting the two global structural reference points, further includes: Calculate the category attribute features of the target pair in each single frame of data; Based on the category attribute features of the target in each single frame of data, the distance between the reference target and the corresponding global structural reference point in each single frame of data, and the angle between the line direction connecting the reference target and the corresponding global structural reference point in each single frame of data and the line direction connecting the two global structural reference points, the global positional relationship features of the target in each single frame of data are determined.

6. The spatiotemporal multi-source data target association method according to any one of claims 1-5, characterized in that, The step of associating each single-source time series data with a single-source target based on the location information of each target specifically includes: Based on the location information of each target, a Kalman filter algorithm is used to perform single-source target association on each single-source time series data to determine the first association result in each single-source time series data.

7. The spatiotemporal multi-source data target association method according to any one of claims 1-5, characterized in that, The step of performing multi-source target association on single-frame data from each data source at any given time based on the topological similarity of the target pairs specifically includes: Based on the topological similarity of the target pairs, the Hungarian algorithm is used to perform multi-source target association on single-frame data from each data source at any given time, and the second association result is determined.

8. A spatiotemporal multi-source data target association device, characterized in that, include: The single-source target association module is used to acquire single-source time series data from each data source to be associated, extract the location information of each target in each single-source time series data, and perform single-source target association on each single-source time series data based on the location information of each target. The local similarity calculation module is used to determine the local positional relationship features of each target in each single frame of data from each data source at any given time, and to calculate the local similarity of each target pair in the single frame of data from each data source at any given time based on the local positional relationship features of each target. The multi-source target association module is used to determine whether the array configuration of each data source is complete by comparing the local similarity of each target pair with a preset threshold. If the local similarity of the target pair is less than the preset threshold, the array configuration of each data source is considered incomplete. The module determines the topological structure features of each target in the target pair in each single frame of data, calculates the topological structure similarity of the target pair based on the topological structure features, and performs multi-source target association on the single-source time series data of each data source according to the topological structure similarity of the target pair. The incomplete array configuration of each data source means that the relative positional relationship of each target in each single frame of data collected by different data sources at any time deviates from the actual positional relationship. The target association result determination module is used to determine the target association result of each data source based on the first association result obtained from single-source target association and the second association result obtained from multi-source target association.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the spatiotemporal multi-source data target association method as described in any one of claims 1-7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the spatiotemporal multi-source data target association method as described in any one of claims 1-7.

Images