Track mining analysis method based on cross-modal data
By constructing a correlation analysis model of cross-modal data and performing confidence-weighted fusion, the problem of fusion of different modes in ship monitoring is solved, and the precise mining and analysis of ship tracks is realized, and the accuracy of data mining and analysis is improved.
Patent Information
- Application Number
- CN202510108180.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-23
- Publication Date
- 2025-05-27
AI Technical Summary
In ship monitoring scenarios, how to effectively integrate different mode data (such as remote sensing images, detection signals, AIS data, text information) to achieve accurate mining and analysis of ship tracks, especially when data obtained by different means have different confidence and lack effective fusion methods.
A track mining analysis method based on cross-modal data is proposed. By obtaining track data of four different modes, data spatiotemporal correlation and screening, a correlation analysis model is constructed, the correlation relationship between tracks is determined, and the fusion track is obtained through confidence-weighted fusion.
It effectively improves the accuracy of cross-modal data mining analysis, solves the confidence problem of data from different sources, builds a correlation model with minimum error, and achieves a high track fusion accuracy.
Smart Images
Figure CN120046101A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of remote sensing image processing, and particularly relates to a method for mining and analyzing ship tracks based on cross-modal data. Background Art
[0002] With the rapid development of sensor technology, it has become a widely used detection mode to detect the same target by multiple means to obtain information in different dimensions. The means obtained by different sensors are often of different modalities, such as images, signals, texts, etc. In the ship monitoring scenario, remote sensing images, radar signal detection, AIS positioning data, external text information, etc. are various modalities of information widely used. How to utilize the deep features of different modality data to mine and analyze information such as ship tracks is one of the hotspots and difficulties in the current remote sensing field. The main difficulty is that the data obtained by different means are of different modalities and there is a lack of effective means for fusion analysis.
[0003] In addition, the data obtained by different means have different confidence levels, and how to construct an association and fusion model is also a difficult problem. Summary of the Invention
[0004] In view of this, the present invention proposes a method for mining and analyzing ship tracks based on cross-modal data, which can effectively associate four types of multi-modal data, namely remote sensing images, detection signals, AIS data, and text information. While making full use of the characteristics of different data, factors such as data volume and confidence level are considered, and an association analysis model is constructed, so as to effectively fuse cross-modal data information and achieve accurate mining and analysis of ship tracks.
[0005] In order to achieve the above object, the technical solution adopted by the present invention is as follows:[[]]END]]
[0006] A method for mining and analyzing ship tracks based on cross-modal data, comprising the following steps:[[]]END]]
[0007] Step 1, obtain four different modalities of ship track data. The ship track data is composed of ship track point data, and the ship track point data represents the longitude and latitude coordinates of an object at a certain moment. Perform data spatio-temporal association among the ship track data of the four modalities, and screen the ship track data of the four modalities; wherein, the four different modalities are remote sensing images, detection signals, AIS data, and text information.
[0008] Step 2, perform ship track analysis on the ship track data of each modality after screening to obtain ship tracks in the four modalities, which are respectively denoted as S 1 、S 2 、S 3 、S 4 ;
[0009] Step 3, combine S 1 、S 2, S 3 , S 4 Perform correlation analysis and modeling to determine the correlation relationships among the four tracks;
[0010] Step 4: Based on the correlated track points among the four tracks, obtain the fused track through the method of confidence weighted fusion;
[0011] Step 5: For the track points without correlation relationships among the four tracks, construct the track points at the corresponding moments in the fused track;
[0012] Step 6: According to all the track points in the fused track, calculate the track points corresponding to all the moments in S 1 , S 2 , S 3 , S 4 by linear interpolation, so as to obtain the final track after cross-modal data mining and analysis.
[0013] Furthermore, the specific method of Step 1 is as follows:
[0014] Step 1a: Respectively obtain the time and longitude and latitude coordinate information of all the track point data in the track data of the four modalities of remote sensing images, detection signals, AIS data, and text information, and obtain the time interval, longitude interval, and latitude interval in each modality, expressed as where the superscript m = 1, 2, 3, 4 respectively represents the four data modalities of remote sensing images, detection signals, AIS data, and text information, T start represents the earliest moment in the track data, T end represents the latest moment in the track data, X min represents the minimum latitude in the track data, X max represents the maximum latitude in the track data, Y min represents the minimum longitude in the track data, Y max represents the maximum longitude in the track data;
[0015] Step 1b: Respectively screen the track data of the four modalities. If the time, longitude, and latitude corresponding to a certain track point data in a certain modality do not appear in the time interval, longitude interval, and latitude interval of any other modality, then delete the track point data.
[0016] Furthermore, the specific method of Step 2 is as follows:
[0017] Express the screened track data of each modality in the following form to obtain the track in this modality:
[0018]
[0019] Among them, the superscript m = 1, 2, 3, 4, respectively representing four data modalities of remote sensing images, detection signals, AIS data, and text information. is the nth track point under the filtered modality m. is the latitude of is the longitude of is the corresponding moment of the track point N m is the total number of track points under the filtered modality m.
[0020] Furthermore, the specific method of step 3 is as follows:
[0021] Step 3a, construct an association analysis model:
[0022]
[0023] Among them, N 1 , N 2 , N 3 , N 4 respectively represent the total number of track points in S 1 , S 2 , S 3 , S 4 , i, j, k, l respectively represent the track point numbers in S 1 , S 2 , S 3 , S 4 . f represents that there is an association relationship between the track points corresponding to the subscripts. f = 0 means non - existence, and f = 1 means existence. represents the Euclidean distance between the i - th and j - th track points. represents the sum of the Euclidean distances between every two of the i - th, j - th, and k - th track points. represents the sum of the Euclidean distances between every two of the i - th, j - th, k - th, and l - th track points.
[0024] Step 3b, solve the above - mentioned association analysis model to obtain the values of each f, that is, the association relationships between the four tracks of S 1 , S 2 , S 3 , S 4 .
[0025] Furthermore, the specific method of step 4 is as follows:
[0026] Step 4a, for each group of track points with an association relationship, select the moment of any track point in the group as the reference moment t, and move the other track points in the group to the reference moment t by linear interpolation. At this time, the track points in this group are unified to the same moment t in time.
[0027] Step 4b: For each reference time t that appears in Step 4a, generate a track point at time t in the fused track through confidence fusion:
[0028]
[0029] where the superscript m = 1, 2, 3, 4 represents four data modalities of remote sensing images, detection signals, AIS data, and text information, respectively, and N m is the total number of track points in S m λ m is the empirical confidence of the four data modalities and satisfies represents the track point corresponding to time t in S m If does not exist, then the corresponding N m is taken as 1, and λ m is taken as 0.
[0030] Furthermore, the specific method of Step 5 is as follows:
[0031] For the track point P 1 in S 2 、S 3 、S 4 that is not associated with other track points, obtain its corresponding time T n , calculate the track point P' n at time T n in the fused track through linear interpolation, and then calculate the final track point P n at time T n in the fused track through confidence improvement: Tn :
[0032] P Tn =(1 - λ)P' n +λP n
[0033] where λ is the empirical confidence of the data modality to which P n belongs.
[0034] Furthermore, the specific method of Step 6 is as follows:
[0035] Step 6a: Arrange the times corresponding to all track points in S 1 、S 2 、S 3 、S 4 in chronological order, and each time is denoted as T num , num = 1, 2, 3...;
[0036] Step 6b. For the fused track, calculate the track P corresponding to the moment T by means of linear interpolation respectively num , which is the final track obtained by cross-modal mining and analysis. Tnum
[0037] The present invention has the following beneficial effects:
[0038] 1. While considering the deep features of the data, the present invention also takes into account the confidence level and the amount of data, can correlate and mine and analyze the track data from different sources, solve the confidence problem of data from different sources, and can effectively improve the accuracy of cross-modal data mining and analysis;
[0039] 2. The present invention constructs an association model with the minimum error, and simultaneously takes into account the intrinsic confidence level of different modal data and the amount of data during modeling, and can obtain a relatively high track fusion accuracy.
[0040] 3. The method of the present invention is applicable to four types of multi-modal data, namely remote sensing images, detection signals, AIS data, and text information, and has high calculation efficiency. Description of the Drawings
[0041] Figure 1 is the overall flowchart of the present invention.
[0042] Figure 2 is the schematic diagram of the track obtained from four types of multi-modal data.
[0043] Figure 3 is the schematic diagram of the cross-modal mining and analysis result of the ship track. Detailed Embodiment
[0044] The following further describes the present invention in detail with reference to the drawings and embodiments.
[0045] A track mining and analysis method based on cross-modal data. First, the method conducts a preliminary spatio-temporal coincidence analysis on multi-modal data to narrow the data range, then analyzes the single-modal data to obtain preliminary tracks, achieving data cleaning through this step. On this basis, an association analysis model that comprehensively considers data features, confidence level, and data volume is constructed to complete the association analysis of the data, and then the data is fused and optimized and the accuracy is improved, finally obtaining a high-precision track mining and analysis result.
[0046] As Figure 1 shown, the method specifically includes the following steps:
[0047] Step 1: Obtain track data in four different modalities. The track data consists of track point data, and the track point data represents the longitude and latitude coordinates of an object at a certain moment. Perform data spatio-temporal association among the track data in the four modalities and screen the track data in the four modalities. Among them, the four different modalities are remote sensing images, detection signals, AIS data, and text information. The specific method is as follows:
[0048] Step 1a: Respectively obtain the moment and longitude and latitude coordinate information of all track point data in the track data of the four modalities of remote sensing images, detection signals, AIS data, and text information, and obtain the time interval, longitude interval, and latitude interval in each modality, expressed as where the superscript m = 1, 2, 3, 4 represents the four data modalities of remote sensing images, detection signals, AIS data, and text information respectively, T start represents the earliest moment in the track data, T end represents the latest moment in the track data, X min represents the minimum latitude in the track data, X max represents the maximum latitude in the track data, Y min represents the minimum longitude in the track data, Y max represents the maximum longitude in the track data;
[0049] Step 1b: Respectively screen the track data in the four modalities. If the moment, longitude, and latitude corresponding to a certain track point data in a certain modality do not appear in the time interval, longitude interval, and latitude interval of any other modality, then delete the track point data.
[0050] Step 2: Perform track analysis on the track data in each modality after screening to obtain the tracks in the four modalities, denoted as S 1 、S 2 、S 3 、S 4 ; Represent the track data in each modality after screening in the following form to obtain the track in this modality:
[0051]
[0052] where the superscript m = 1, 2, 3, 4 represents the four data modalities of remote sensing images, detection signals, AIS data, and text information respectively, is the nth track point in modality m after screening, is 's latitude, is 's longitude, is the corresponding moment of the track point , N m is the total number of track points in modality m after screening.
[0053] Step 3: Perform correlation analysis modeling on S 1 , S 2 , S 3 , S 4 to determine the correlation relationship among the four tracks; the specific method is as follows:
[0054] Step 3a: Construct a correlation analysis model:
[0055]
[0056] where N 1 , N 2 , N 3 , N 4 represent the total number of track points in S 1 , S 2 , S 3 , S 4 respectively, i, j, k, l represent the track point numbers in S 1 , S 2 , S 3 , S 4 respectively, f represents the existence of a correlation relationship between the track points corresponding to the subscripts, f = 0 means non-existence, and f = 1 means existence, represents the Euclidean distance between the i-th and j-th track points, represents the sum of the Euclidean distances between every two of the i-th, j-th, and k-th track points, represents the sum of the Euclidean distances between every two of the i-th, j-th, k-th, and l-th track points;
[0057] Step 3b: Solve the above correlation analysis model to obtain the values of each f, that is, the correlation relationship among the four tracks of S 1 , S 2 , S 3 , S 4 .
[0058] Step 4: Obtain the fused track through confidence-weighted fusion based on the track points with correlation relationships among the four tracks; the specific method is as follows:
[0059] Step 4a: For each group of track points with correlation relationships, select the moment of any track point in the group as the reference moment t, and move the other track points in the group to the reference moment t through linear interpolation. At this time, the track points in this group are unified to the same moment t in time;
[0060] Step 4b: For each reference moment t that appears in Step 4a, generate the track point at moment t in the fused track through confidence fusion:
[0061]
[0062] Among them, the superscript m = 1, 2, 3, 4, respectively representing four data modalities of remote sensing images, detection signals, AIS data, and text information, N m is the total number of waypoints in S m , λ m is the empirical confidence of the four data modalities and satisfies represents the waypoint corresponding to the time t in S m . If does not exist, then the corresponding N m is taken as 1, and λ m is taken as 0.
[0063] Step 5, for the waypoints in the four tracks that have no association relationship, construct the waypoints corresponding to the corresponding time in the fused track; the specific method is:
[0064] For the waypoint P 1 in S 2 , S 3 , S 4 , S n that has not been associated with other waypoints, obtain its corresponding time T n , calculate the waypoint P' n at time T n in the fused track by linear interpolation, and then calculate the final waypoint P n at time T Tn in the fused track by confidence improvement:
[0065] P Tn = (1 - λ)P' n + λP n
[0066] Among them, λ is the empirical confidence of the data modality to which P n belongs.
[0067] Step 6, according to all the waypoints in the fused track, calculate the waypoints corresponding to all times in S 1 , S 2 , S 3 , S 4 by linear interpolation, so as to obtain the final track after cross-modal data mining and analysis; the specific method is:
[0068] Step 6a, combine S 1 , S 2 , S 3 , S 4The moments corresponding to all the track points in [the data] are arranged in chronological order, and each moment is denoted as T num , where num = 1, 2, 3...;
[0069] Step 6b. For the fused track, by means of linear interpolation, calculate the track P corresponding to the moment T num respectively, which is the final track obtained by cross-modal mining and analysis. Tnum That is the final track obtained by cross-modal mining and analysis.
[0070] The effect of this method can be further illustrated by the following experiments:
[0071] 1. Experimental method.
[0072] Through simulation, the real track of a ship was generated, and on this basis, remote sensing image, signal data, AIS data and text data information were simulated according to the confidence model. Then, the method of the present invention was used for mining and analysis, and the error between the mining and analysis result and the real track was calculated. A traditional weighted analysis and fusion method was selected for comparison with the method of the present invention.
[0073] 2. Parameter description.
[0074] In step 4, the confidence weight coefficients selected in this experiment are 0.3, 0.15, 0.3 and 0.25 respectively.
[0075] 3. Experimental results
[0076] The track results of remote sensing image, signal data, AIS data and text data information are as Figure 2 shown, and the final cross-modal mining and analysis result is as Figure 3 shown.
[0077] After calculation, the accuracy rate of the track obtained by mining and analysis using the method of the present invention is 95.3%, and the accuracy of the traditional method is 81.4%. The experimental results show that the method of the present invention can effectively fuse information of different modalities and improve the accuracy of ship track mining and analysis.
Claims
1. A track mining and analysis method based on cross-modal data, characterized in that: The following steps are involved: Step 1, obtaining track data of four different modes, the track data consists of track point data, the track point data represents the latitude and longitude coordinates of the object at a certain moment, performing data spatiotemporal correlation between the track data of the four modes, and screening the track data of the four modes; wherein the four different modes are remote sensing images, detection signals, AIS data, and text information; Step 2: Perform track analysis on the track data of each mode after screening to obtain the tracks under four modes, which are recorded as S 1 , S 2 , S 3 , S 4 ; Step 3: S 1 , S 2 , S 3 , S 4 Conduct correlation analysis modeling to determine the correlation between the four tracks; Step 4, according to the track points with correlation in the four tracks, a fused track is obtained by means of confidence weighted fusion; Step 5, for the track points that have no correlation in the four tracks, construct the track points at the corresponding time in the fused track; Step 6: Calculate S by linear interpolation based on all track points in the fused track. 1 , S 2 , S 3 , S 4 The track points corresponding to all moments in the , thus obtaining the final track after cross-modal data mining and analysis.
2. The method for track mining and analysis based on cross-modal data according to claim 1, characterized in that: The specific method of step 1 is: Step 1a, respectively obtain the time and longitude and latitude coordinate information of all track point data in the track data of four modes: remote sensing image, detection signal, AIS data, and text information, and obtain the time interval, longitude interval, and latitude interval in each mode, expressed as Wherein, the superscript m=1, 2, 3, 4, respectively represents the four data modes of remote sensing image, detection signal, AIS data, and text information, T start Indicates the earliest time in the track data, T end Indicates the latest time in the track data, X min Indicates the minimum latitude in the track data, X max Indicates the maximum latitude and Y in the track data min Indicates the minimum longitude, Y max Indicates the maximum longitude in the track data; Step 1b, respectively filter the track data of the four modes. If the time, longitude, and latitude corresponding to a track point data of a certain mode do not appear in the time interval, longitude interval, and latitude interval of any other mode, delete the track point data.
3. A track mining and analysis method based on cross-modal data according to claim 2, characterized in that: The specific method of step 2 is: The track data of each mode after screening is expressed as follows to obtain the track under this mode: Among them, the superscript m=1,2,3,4, respectively represents the four data modes of remote sensing image, detection signal, AIS data, and text information. is the nth track point under the filtered mode m, for The latitude, for The longitude, Track point The corresponding moment, N m is the total number of track points under mode m after screening.
4. The method for track mining and analysis based on cross-modal data according to claim 3, characterized in that: The specific method of step 3 is: Step 3a, build an association analysis model: Among them, N 1 、N 2 、N 3 、N 4 Respectively represent S 1 , S 2 , S 3 , S 4 The total number of track points in the, i, j, k, l represent S 1 , S 2 , S 3 , S 4 The track point number in the subscript, f indicates that there is an association relationship between the track points corresponding to the subscript, f = 0 means that there is no association relationship, and f = 1 means that there is an association relationship. Represents the Euclidean distance between two track points i and j, Represents the sum of the Euclidean distances between every two track points among the three track points i, j, and k. Represents the sum of the Euclidean distances between every two track points among the four track points i, j, k, and l; Step 3b, solve the above correlation analysis model to obtain the value of each f, that is, S 1 , S 2 , S 3 , S 4 The relationship between the four tracks.
5. The method for track mining and analysis based on cross-modal data according to claim 4, characterized in that: The specific method of step 4 is: Step 4a, for each group of associated track points, select the time of any track point in the group as the reference time t, and move the other track points in the group to the reference time t by linear interpolation. At this time, the track points in the group are unified to the same time t in time; Step 4b, for each reference time t appearing in step 4a, generate the track point at time t in the fused track by means of confidence fusion: Wherein, the superscript m = 1, 2, 3, 4, respectively representing the four data modes of remote sensing image, detection signal, AIS data, and text information, N m For S m The total number of track points, λ m is the empirical confidence of the four data modes, and satisfies Indicates S m The track point corresponding to time t, if does not exist, then the corresponding N m The value is 1, λ m The value is 0.
6. The method for track mining and analysis based on cross-modal data according to claim 1, characterized in that: The specific method of step 5 is: For S 1 , S 2 , S 3 , S 4 Track point P that is not associated with other track points n , get the corresponding time T n , calculate the time T in the fused track by linear interpolation n The track point P' N Then, the time T in the fusion track is calculated by confidence enhancement n The final track point P Tn : P Tn =(1-λ)P‘ n +λP n Where λ is P n The empirical confidence level of the data modality.
7. The method for track mining and analysis based on cross-modal data according to claim 1, characterized in that: The specific method of step 6 is: Step 6a, S 1 , S 2 , S 3 , S 4 The times corresponding to all track points in the sequence are arranged in order, and each time is recorded as T num , num=1,2,3...; Step 6b: For the fused track, calculate T num The track P corresponding to the time Tnum , which is the final track obtained by cross-modal mining analysis.