A target collaborative perception method and system based on intelligent fleet navigation

Abstract

The present invention discloses a target collaborative perception method and system based on intelligent fleet navigation. Target spatial information is acquired through AIS and laser radar fusion positioning, and multi-perspective video streams are collected in real time based on multiple intelligent ships. After the system extracts the video key frames, it uses Sobel edge detection and PCA analysis technology to construct an object feature set, and combines the network status dynamic evaluation to determine the data transmission volume. The feature set is analyzed for correlation using a pre-trained self-attention model to screen out features with high attention scores, thereby achieving efficient target recognition and shared perception. The present invention can significantly improve target recognition accuracy and system response speed, rationally utilize multi-intelligent ship data for target collaborative perception, and effectively improve the comprehensive environmental perception capabilities of multiple ships.

Description

Technical Field

[0001] The present invention relates to the field of intelligent fleets, and more specifically, to a target collaborative perception method and system based on intelligent fleet navigation. Background Art

[0002] With the continuous development of the fishing industry and navigation, intelligent fleets are playing an increasingly important role in marine environmental monitoring, resource exploration, search and rescue missions, and fishing. However, in complex aquatic environments, the perception capabilities of individual vessels are limited, making it difficult to fully and accurately obtain information about surrounding targets and achieve efficient collaborative operations. In traditional intelligent ship navigation and mission execution, there is a lack of comprehensive target recognition methods for multi-source data, multi-sensor data, and multi-video source data. Traditional methods are often limited to object recognition on a single vessel, and suffer from problems such as small perception range, insufficient information fusion, and low target recognition accuracy. Their comprehensive recognition capabilities are low, and data silos are easily formed. This is not conducive to the collaborative environmental perception and analysis of multiple intelligent ships, and it is difficult to improve the overall operational efficiency of multiple intelligent ships.

[0003] Therefore, there is an urgent need for a target collaborative perception method based on intelligent fleet navigation to solve the above problems. Summary of the Invention

[0004] The present invention overcomes the defects of the prior art and proposes a target collaborative perception method and system based on intelligent fleet navigation.

[0005] A first aspect of the present invention provides a target collaborative perception method based on intelligent fleet navigation, comprising:

[0006] S102: Among multiple intelligent ships, positioning and sensing target spatial information through AIS information and lidar detection data;

[0007] S104: Based on multiple intelligent ships, multiple video data based on target perception are acquired in real time;

[0008] S106: extracting key frames from the multiple video data, segmenting the key frames into object regions based on the Sobel operator, and extracting principal components of object features through a PCA analysis method to form multiple first feature sets;

[0009] S108: Real-time shared data volume through intelligent ship network status assessment;

[0010] S110: Using the feature map data of the existing identified objects, a self-attention mechanism model is trained, and a query matrix, a key matrix, and a value matrix are determined. Each two first feature sets are combined, and based on the self-attention mechanism model, an attention score of the combined feature set is calculated. High-scoring features are screened to obtain features that meet the real-time shared data volume and are marked as the second feature set.

[0011] S112: Target detection and recognition are performed based on the second feature set through the image recognition model to generate a recognition result. Target collaborative perception is performed based on the recognition result and target spatial information, and the perception result is sent to multiple intelligent ships.

[0012] In this solution, the S102 is specifically as follows:

[0013] The AIS information includes the position, heading and speed information of each smart ship;

[0014] Perform spatial analysis on the LiDAR detection data to generate point cloud data, filter the point cloud data, segment different target objects, and obtain the target object bounding box based on the point cloud information;

[0015] The spatial deviation of the target object is averaged based on the target object bounding box obtained by analyzing multiple intelligent ships to obtain the initial spatial information based on the point cloud information;

[0016] Combined with AIS information, the position information of each smart ship is imported into the initial spatial information to generate target spatial information.

[0017] In this solution, the S104 is specifically:

[0018] Through the video monitoring device, multiple intelligent ships can sense surrounding target objects and take real-time video of the target range to obtain corresponding video data;

[0019] The video data is stored in the smart ship database and shared through the Internet of Things.

[0020] In this solution, the S106 is specifically:

[0021] Extract key frames from video data to obtain key image frames;

[0022] Perform image enhancement, image noise reduction, and standardization preprocessing on key image frames;

[0023] Based on the Sobel operator, the gradient of each pixel in the key image frame is calculated based on the gray value, and the edge is divided by the gradient value. The object area is segmented by edge division to obtain the object area;

[0024] Based on the object area, color and edge feature extraction is performed in the key image frame;

[0025] Edge features are extracted during the process of dividing edges by gradient values, and color features are obtained through color histogram statistics;

[0026] Based on the PCA analysis method, color and edge features are taken as input and two input feature vectors are generated;

[0027] Matrix the input eigenvector to form a feature matrix, and calculate the covariance matrix C of the feature matrix;

[0028] Perform eigenvalue decomposition on the covariance matrix C to obtain decomposition eigenvalues and decomposition eigenvectors;

[0029] Sort the decomposition eigenvalues by size and extract the first 70% of the decomposition eigenvectors as the principal components;

[0030] Principal component extraction is performed based on the two input feature vectors to obtain a first feature set.

[0031] In this solution, the S108 is specifically:

[0032] In a real-time cycle, the packet loss rate, network delay and network jitter of each smart ship are counted;

[0033] Based on the packet loss rate, network delay and network jitter, the average value is calculated to obtain the averaged network parameters;

[0034] The maximum data transmission volume in a real-time cycle is evaluated by averaging network parameters to obtain the real-time shared data volume.

[0035] In this solution, the S110 includes:

[0036] Obtain feature map data and feature recognition rate of identified objects from the system database;

[0037] Setting attention scores based on feature recognition rates;

[0038] Correlate feature graph data with attention scores;

[0039] Construct a self-attention mechanism model, map the feature map data through linear transformation, and generate the initial query matrix, initial key matrix, and initial value matrix;

[0040] Import the feature map data into the self-attention mechanism model and calculate the attention score corresponding to the feature map. Use the absolute difference between the attention score and the attention score as the loss function to train the self-attention mechanism model, and cyclically adjust the query matrix, initial key matrix, and initial value matrix until the loss function converges to the preset difference.

[0041] After training is completed, the query matrix, key matrix and value matrix are determined.

[0042] In this solution, the S110 further includes:

[0043] Combining every two first feature sets to obtain a combined feature set;

[0044] Import the combined feature set into the self-attention mechanism model, calculate the attention score of the feature vectors, and filter out the feature sets with high scores and mark them as high-scoring feature data;

[0045] The high-scoring feature data of all combined feature sets are analyzed and screened, and based on the real-time shared data volume, feature data that meets the data volume is selected from the high-scoring feature data and marked as the second feature set.

[0046] In this solution, the S112 is specifically:

[0047] Importing the second feature set into the image recognition model to perform object classification and object image positioning, and generate recognition results;

[0048] Based on the recognition results and target spatial information, target collaborative perception is performed, and the perception results are sent to multiple smart ships through the central platform.

[0049] A second aspect of the present invention further provides a target collaborative perception system based on intelligent fleet navigation, the system comprising: a memory and a processor, wherein the memory includes a target collaborative perception program based on intelligent fleet navigation, and when the target collaborative perception program based on intelligent fleet navigation is executed by the processor, the following steps are implemented:

[0050] S102: Among multiple intelligent ships, positioning and sensing target spatial information through AIS information and lidar detection data;

[0051] S104: Based on multiple intelligent ships, multiple video data based on target perception are acquired in real time;

[0052] S106: extracting key frames from the multiple video data, segmenting the key frames into object regions based on the Sobel operator, and extracting principal components of object features through a PCA analysis method to form multiple first feature sets;

[0053] S108: Real-time shared data volume through intelligent ship network status assessment;

[0054] S110: Using the feature map data of the existing identified objects, a self-attention mechanism model is trained, and a query matrix, a key matrix, and a value matrix are determined. Each two first feature sets are combined, and based on the self-attention mechanism model, an attention score of the combined feature set is calculated. High-scoring features are screened to obtain features that meet the real-time shared data volume and are marked as the second feature set.

[0055] S112: Target detection and recognition are performed based on the second feature set through the image recognition model to generate a recognition result. Target collaborative perception is performed based on the recognition result and target spatial information, and the perception result is sent to multiple intelligent ships.

[0056] The third aspect of the present invention also provides a computer-readable storage medium, which includes a target collaborative perception program based on intelligent fleet navigation. When the target collaborative perception program based on intelligent fleet navigation is executed by a processor, the steps of the target collaborative perception method based on intelligent fleet navigation as described in any one of the above items are implemented.

[0057] The present invention discloses a target collaborative perception method and system based on intelligent fleet navigation. Target spatial information is acquired through AIS and laser radar fusion positioning, and multi-perspective video streams are collected in real time based on multiple intelligent ships. After the system extracts the video key frames, it uses Sobel edge detection and PCA analysis technology to construct an object feature set, and combines the network status dynamic evaluation to determine the data transmission volume. The feature set is analyzed for correlation using a pre-trained self-attention model to screen out features with high attention scores, thereby achieving efficient target recognition and shared perception. The present invention can significantly improve target recognition accuracy and system response speed, rationally utilize multi-intelligent ship data for target collaborative perception, and effectively improve the comprehensive environmental perception capabilities of multiple ships. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A flow chart of a target collaborative perception method based on intelligent fleet navigation according to the present invention is shown;

[0059] Figure 2 A block diagram of a target collaborative perception system based on intelligent fleet navigation according to the present invention is shown. DETAILED DESCRIPTION

[0060] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0061] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0062] Figure 1 The flowchart of the target collaborative perception method based on intelligent fleet navigation of the present invention is shown.

[0063] like Figure 1As shown, the first aspect of the present invention provides a target collaborative perception method based on intelligent fleet navigation, comprising:

[0064] S102: Among multiple intelligent ships, positioning and sensing target spatial information through AIS information and lidar detection data;

[0065] S104: Based on multiple intelligent ships, multiple video data based on target perception are acquired in real time;

[0066] S106: extracting key frames from the multiple video data, segmenting the key frames into object regions based on the Sobel operator, and extracting principal components of object features through a PCA analysis method to form multiple first feature sets;

[0067] S108: Real-time shared data volume through intelligent ship network status assessment;

[0068] S110: Using the feature map data of the existing identified objects, a self-attention mechanism model is trained, and a query matrix, a key matrix, and a value matrix are determined. Each two first feature sets are combined, and based on the self-attention mechanism model, an attention score of the combined feature set is calculated. High-scoring features are screened to obtain features that meet the real-time shared data volume and are marked as the second feature set.

[0069] S112: Target detection and recognition are performed based on the second feature set through the image recognition model to generate a recognition result. Target collaborative perception is performed based on the recognition result and target spatial information, and the perception result is sent to multiple intelligent ships.

[0070] According to an embodiment of the present invention, the S102 is specifically:

[0071] The AIS information includes the position, heading and speed information of each smart ship;

[0072] Perform spatial analysis on the LiDAR detection data to generate point cloud data, filter the point cloud data, segment different target objects, and obtain the target object bounding box based on the point cloud information;

[0073] The spatial deviation of the target object is averaged based on the target object bounding box obtained by analyzing multiple intelligent ships to obtain the initial spatial information based on the point cloud information;

[0074] Combined with AIS information, the position information of each smart ship is imported into the initial spatial information to generate target spatial information.

[0075] It should be noted that the target perception generally involves intelligent identification and perception of other ships, shore buildings, buoys, docks and other objects, so that smart ships can better complete tasks such as water area monitoring, surveillance, search and rescue, and transportation.

[0076] According to an embodiment of the present invention, the S104 is specifically as follows:

[0077] Through the video monitoring device, multiple intelligent ships can sense surrounding target objects and take real-time video of the target range to obtain corresponding video data;

[0078] The video data is stored in the smart ship database and shared through the Internet of Things.

[0079] It should be noted that the data sharing is performed through the Internet of Things, and multiple smart ships and the central platform can share data through the Internet of Things to achieve collaborative perception of target objects.

[0080] According to an embodiment of the present invention, the S106 is specifically as follows:

[0081] Extract key frames from video data to obtain key image frames;

[0082] Perform image enhancement, image noise reduction, and standardization preprocessing on key image frames;

[0083] Based on the Sobel operator, the gradient of each pixel in the key image frame is calculated based on the gray value, and the edge is divided by the gradient value. The object area is segmented by edge division to obtain the object area;

[0084] Based on the object area, color and edge feature extraction is performed in the key image frame;

[0085] Edge features are extracted during the process of dividing edges by gradient values, and color features are obtained through color histogram statistics;

[0086] Based on the PCA analysis method, color and edge features are taken as input and two input feature vectors are generated;

[0087] Matrix the input eigenvector to form a feature matrix, and calculate the covariance matrix C of the feature matrix;

[0088] Perform eigenvalue decomposition on the covariance matrix C to obtain decomposition eigenvalues and decomposition eigenvectors;

[0089] Sort the decomposition eigenvalues by size and extract the first 70% of the decomposition eigenvectors as the principal components;

[0090] Principal component extraction is performed based on the two input feature vectors to obtain a first feature set.

[0091] It should be noted that the sobel operator filters edges by calculating the grayscale gradient of each pixel to adapt to the effective separation of object images in aquatic environments. In addition, based on analysis requirements, object area segmentation can be performed through other edge detection algorithms. Before calculating the gradient of the grayscale value, the image needs to be grayed. The two input feature vectors represent the corresponding feature vectors of color features and edge features, respectively, and both vectors include multiple. The top 70%, that is, the top 70% of the highest-scoring data. Multiple video data generate corresponding multiple first feature sets. The first feature set includes the principal component features of both color and edge data.

[0092] According to an embodiment of the present invention, the S108 is specifically as follows:

[0093] In a real-time cycle, the packet loss rate, network delay and network jitter of each smart ship are counted;

[0094] Based on the packet loss rate, network delay and network jitter, the average value is calculated to obtain the averaged network parameters;

[0095] The maximum data transmission volume in a real-time cycle is evaluated by averaging network parameters to obtain the real-time shared data volume.

[0096] It should be noted that the averaged network parameters include the average packet loss rate, average network delay, and average network jitter. Averaging is the process of averaging the network parameters of multiple smart ships.

[0097] According to an embodiment of the present invention, the S110 includes:

[0098] Obtain feature map data and feature recognition rate of identified objects from the system database;

[0099] Setting attention scores based on feature recognition rates;

[0100] Correlate feature graph data with attention scores;

[0101] Construct a self-attention mechanism model, map the feature map data through linear transformation, and generate the initial query matrix, initial key matrix, and initial value matrix;

[0102] Import the feature map data into the self-attention mechanism model and calculate the attention score corresponding to the feature map. Use the absolute difference between the attention score and the attention score as the loss function to train the self-attention mechanism model, and cyclically adjust the query matrix, initial key matrix, and initial value matrix until the loss function converges to the preset difference.

[0103] After training is completed, the query matrix, key matrix and value matrix are determined.

[0104] It should be noted that the feature map data is a feature map obtained during the recognition process in the historical database of recognized object images. Generally speaking, when performing image recognition based on a CNN network or other artificial intelligence recognition models for image analysis, the intermediate recognition process usually requires the extraction of relevant feature maps, and the feature maps can be collected and stored based on the platform database. The feature recognition rate is the successful recognition rate after the relevant feature map is used to identify the object, which can reflect to a certain extent the importance of the feature in the actual recognition process. In the feature map data, each feature map includes an attention score. The attention score is proportional to the feature recognition rate. For example, if the feature recognition rate is 50%, the attention score can be set to the same value, 50 points.

[0105] The loss function can be set as f = |attention score - attention score|. By minimizing the difference between the two, the self-attention model is trained to obtain the learning parameters (query matrix, key matrix, and value matrix). The default difference can be set to 10, that is, the score difference is within 10.

[0106] Import the feature map data into the self-attention mechanism model and calculate the attention score corresponding to the feature map. Specifically, the corresponding score is obtained by calculating the score matrix Z. The calculation is as follows:

[0107] ,

[0108] Among them, Z is the final calculated score matrix, through which the attention score is obtained, Q, K, V, are the dimensions of the query matrix, key matrix, value matrix, and vectors in the feature graph, respectively.

[0109] According to an embodiment of the present invention, the S110 further includes:

[0110] Combining every two first feature sets to obtain a combined feature set;

[0111] Import the combined feature set into the self-attention mechanism model, calculate the attention score of the feature vectors, and filter out the feature sets with high scores and mark them as high-scoring feature data;

[0112] The high-scoring feature data of all combined feature sets are analyzed and screened, and based on the real-time shared data volume, feature data that meets the data volume is selected from the high-scoring feature data and marked as the second feature set.

[0113] It should be noted that if the amount of high-scoring feature data exceeds the amount of real-time shared data, random filtering can be performed to reduce the feature data to meet the data volume. Alternatively, high-scoring features can be selected based on ranking by score. The purpose of the combination is to fuse the principal component feature data obtained from different intelligent ships and filter the fused feature data to obtain high-scoring features. If there are three first feature sets, the number of pairwise combinations is three, and the three corresponding combined feature sets can be obtained through these three combinations.

[0114] According to an embodiment of the present invention, the S112 is specifically as follows:

[0115] Importing the second feature set into the image recognition model to perform object classification and object image positioning, and generate recognition results;

[0116] Based on the recognition results and target spatial information, target collaborative perception is performed, and the perception results are sent to multiple smart ships through the central platform.

[0117] It should be noted that the image recognition model can utilize models for related image feature recognition, such as yolov5, CNN-based image recognition models, support vector machines, decision trees, and other models for image recognition and classification.

[0118] In this invention, the entire intelligent ship operating system includes multiple intelligent ships and a central platform. The central platform can transmit perception and analysis results to multiple intelligent ships. Furthermore, based on the terminal's needs, a specific intelligent ship can serve as the central platform for collaborative target perception. Both the first and second feature sets include color and edge features.

[0119] It is worth mentioning here that in the traditional navigation and mission execution of intelligent ships, there is a lack of comprehensive target recognition methods for multi-source data, multi-sensor data, and multi-video source data. Traditional methods are often limited to object recognition of a single ship, and their comprehensive recognition capabilities are low. They cannot effectively combine the multi-source data of other nearby intelligent ships for shared analysis or collaborative perception, which easily forms data islands, is not conducive to the collaborative environmental perception analysis of multiple intelligent ships, and it is difficult to improve the comprehensive operation efficiency of multiple intelligent ships. Based on this, the present invention obtains videos of intelligent ships and performs feature extraction on video key frames based on the PCA analysis method. The feature extraction is used to obtain the main component features for target recognition. In the second step, a self-attention mechanism model is trained. The self-attention mechanism model is used to calculate attention scores for different feature data. Scores are calculated in the main component features (i.e., the first feature set). Based on the scores, a second feature set with a high score is selected. Based on the shared network, the intelligent ships realize fusion analysis of the source video data. After obtaining the second feature set, the target can be collaboratively perceived and efficiently recognized. The recognition results are combined with the spatial information of target perception to effectively achieve precise target perception and spatial confirmation, effectively improving the comprehensive environmental perception capability of multiple ships.

[0120] Based on the shared network, multiple smart ships can share the corresponding target perception results.

[0121] According to an embodiment of the present invention, the further embodiment includes:

[0122] Obtaining an attention score for the feature data based on the second feature set;

[0123] generating a plurality of second feature maps according to the feature vectors in the second feature set, and associating the second feature maps with the attention scores;

[0124] According to the DCA feature fusion algorithm, the multiple second feature maps are multiplied by the first weight respectively to obtain multiple category activation maps;

[0125] Perform spatial weighted fusion on multiple category activation maps. During the weighted fusion process, each category activation map corresponds to a score weight, which is equal to the corresponding attention score, and finally obtains the fused feature data;

[0126] Perform target detection and recognition on the fused feature data.

[0127] It should be noted that, in the image recognition process, since the second feature set is the data for acquiring image features in multi-angle video data of multiple intelligent ships, the data is multi-dimensional and multi-source. When the general image recognition model performs feature analysis, there may be low feature processing efficiency for this feature set. Therefore, the present invention improves the DCA fusion method to fuse the feature data (i.e., feature vectors) in the second feature set, and applies the attention score as a weight for weighted fusion in the fusion process to improve the subsequent recognition efficiency and feature processing efficiency of the fused features, greatly improve the adaptability to a variety of image recognition models, and thereby improve the practicality and efficiency of the target perception system.

[0128] The first weight is a user-preset value. A second feature map corresponds to a class activation map and an attention score (also used for score weight).

[0129] According to an embodiment of the present invention, forming a plurality of first feature sets further includes:

[0130] Determine whether the quantity of the first feature set is greater than a preset multiple of the real-time shared data quantity, and if so, construct an autoencoder;

[0131] Based on the object area, color and edge features are extracted in the key image frame, and the color and edge features are imported into the autoencoder as the original input features. The encoder maps the input features to low-dimensional representation data;

[0132] Based on the decoder, the low-dimensional representation data is reconstructed and reconstructed feature data is generated;

[0133] Based on the mean square error as the loss function, the difference between the reconstructed feature data and the original input features is evaluated, and the autoencoder is cyclically optimized until the loss function converges to the expected value. The reconstructed feature data at this time is feature vectorized to obtain the first feature set.

[0134] It should be noted that the preset multiple can be set to 2 or above, which is used to determine whether the data volume of the first feature set is much larger than the real-time shared data volume. If so, it means that the feature data volume extracted from the current multiple video data is large, the dimension is high, and the redundancy is high. It is difficult to reduce the data volume through the PCA extraction method, and it is also necessary to ensure the retention of the main feature data. Therefore, when the data volume is much larger than the real-time shared data volume, the present invention performs corresponding dimensionality reduction operations on the feature data through the autoencoder to further reduce data redundancy, extract feature data of the main dimensions, and form a first feature set. The judgment of the first feature set is based on the feature set obtained in advance by the PCA method. If the data volume of the first feature set is much larger than the real-time shared data volume, it is re-analyzed based on the color and edge features to generate a new first feature set. The expected value is a smaller result value of the loss function and can be set by the user.

[0135] Figure 2 A block diagram of a target collaborative perception system based on intelligent fleet navigation according to the present invention is shown.

[0136] A second aspect of the present invention further provides a target collaborative perception system 2 based on intelligent fleet navigation, the system comprising: a memory 21 and a processor 22. The memory 21 includes a target collaborative perception program based on intelligent fleet navigation. When the target collaborative perception program based on intelligent fleet navigation is executed by the processor 22, the following steps are implemented:

[0137] S102: Among multiple intelligent ships, positioning and sensing target spatial information through AIS information and lidar detection data;

[0138] S104: Based on multiple intelligent ships, multiple video data based on target perception are acquired in real time;

[0139] S106: extracting key frames from the multiple video data, segmenting the key frames into object regions based on the Sobel operator, and extracting principal components of object features through a PCA analysis method to form multiple first feature sets;

[0140] S108: Real-time shared data volume through intelligent ship network status assessment;

[0141] S110: Using the feature map data of the existing identified objects, a self-attention mechanism model is trained, and a query matrix, a key matrix, and a value matrix are determined. Each two first feature sets are combined, and based on the self-attention mechanism model, an attention score of the combined feature set is calculated. High-scoring features are screened to obtain features that meet the real-time shared data volume and are marked as the second feature set.

[0142] S112: Target detection and recognition are performed based on the second feature set through the image recognition model to generate a recognition result. Target collaborative perception is performed based on the recognition result and target spatial information, and the perception result is sent to multiple intelligent ships.

[0143] According to an embodiment of the present invention, the S102 is specifically:

[0144] The AIS information includes the position, heading and speed information of each smart ship;

[0145] Perform spatial analysis on the LiDAR detection data to generate point cloud data, filter the point cloud data, segment different target objects, and obtain the target object bounding box based on the point cloud information;

[0146] The spatial deviation of the target object is averaged based on the target object bounding box obtained by analyzing multiple intelligent ships to obtain the initial spatial information based on the point cloud information;

[0147] Combined with AIS information, the position information of each smart ship is imported into the initial spatial information to generate target spatial information.

[0148] It should be noted that the target perception generally involves intelligent identification and perception of other ships, shore buildings, buoys, docks and other objects, so that smart ships can better complete tasks such as water area monitoring, surveillance, search and rescue, and transportation.

[0149] According to an embodiment of the present invention, the S104 is specifically as follows:

[0150] Through the video monitoring device, multiple intelligent ships can sense surrounding target objects and take real-time video of the target range to obtain corresponding video data;

[0151] The video data is stored in the smart ship database and shared through the Internet of Things.

[0152] It should be noted that the data sharing is performed through the Internet of Things, and multiple smart ships and the central platform can share data through the Internet of Things to achieve collaborative perception of target objects.

[0153] According to an embodiment of the present invention, the S106 is specifically as follows:

[0154] Extract key frames from video data to obtain key image frames;

[0155] Perform image enhancement, image noise reduction, and standardization preprocessing on key image frames;

[0156] Based on the Sobel operator, the gradient of each pixel in the key image frame is calculated based on the gray value, and the edge is divided by the gradient value. The object area is segmented by edge division to obtain the object area;

[0157] Based on the object area, color and edge feature extraction is performed in the key image frame;

[0158] Edge features are extracted during the process of dividing edges by gradient values, and color features are obtained through color histogram statistics;

[0159] Based on the PCA analysis method, color and edge features are taken as input and two input feature vectors are generated;

[0160] Matrix the input eigenvector to form a feature matrix, and calculate the covariance matrix C of the feature matrix;

[0161] Perform eigenvalue decomposition on the covariance matrix C to obtain decomposition eigenvalues and decomposition eigenvectors;

[0162] Sort the decomposition eigenvalues by size and extract the first 70% of the decomposition eigenvectors as the principal components;

[0163] Principal component extraction is performed based on the two input feature vectors to obtain a first feature set.

[0164] It should be noted that the sobel operator filters edges by calculating the grayscale gradient of each pixel to adapt to the effective separation of object images in aquatic environments. In addition, based on analysis requirements, object area segmentation can be performed through other edge detection algorithms. Before calculating the gradient of the grayscale value, the image needs to be grayed. The two input feature vectors represent the corresponding feature vectors of color features and edge features, respectively, and both vectors include multiple. The top 70%, that is, the top 70% of the highest-scoring data. Multiple video data generate corresponding multiple first feature sets. The first feature set includes the principal component features of both color and edge data.

[0165] According to an embodiment of the present invention, the S108 is specifically as follows:

[0166] In a real-time cycle, the packet loss rate, network delay and network jitter of each smart ship are counted;

[0167] Based on the packet loss rate, network delay and network jitter, the average value is calculated to obtain the averaged network parameters;

[0168] The maximum data transmission volume in a real-time cycle is evaluated by averaging network parameters to obtain the real-time shared data volume.

[0169] It should be noted that the averaged network parameters include the average packet loss rate, average network delay, and average network jitter. Averaging is the process of averaging the network parameters of multiple smart ships.

[0170] According to an embodiment of the present invention, the S110 includes:

[0171] Obtain feature map data and feature recognition rate of identified objects from the system database;

[0172] Setting attention scores based on feature recognition rates;

[0173] Correlate feature graph data with attention scores;

[0174] Construct a self-attention mechanism model, map the feature map data through linear transformation, and generate the initial query matrix, initial key matrix, and initial value matrix;

[0175] Import the feature map data into the self-attention mechanism model and calculate the attention score corresponding to the feature map. Use the absolute difference between the attention score and the attention score as the loss function to train the self-attention mechanism model, and cyclically adjust the query matrix, initial key matrix, and initial value matrix until the loss function converges to the preset difference.

[0176] After training is completed, the query matrix, key matrix and value matrix are determined.

[0177] It should be noted that the feature map data is a feature map obtained during the recognition process in the historical database of recognized object images. Generally speaking, when performing image recognition based on a CNN network or other artificial intelligence recognition models for image analysis, the intermediate recognition process usually requires the extraction of relevant feature maps, and the feature maps can be collected and stored based on the platform database. The feature recognition rate is the successful recognition rate after the relevant feature map is used to identify the object, which can reflect to a certain extent the importance of the feature in the actual recognition process. In the feature map data, each feature map includes an attention score. The attention score is proportional to the feature recognition rate. For example, if the feature recognition rate is 50%, the attention score can be set to the same value, 50 points.

[0178] The loss function can be set as f = |attention score - attention score|. By minimizing the difference between the two, the self-attention model is trained to obtain the learning parameters (query matrix, key matrix, and value matrix). The default difference can be set to 10, that is, the score difference is within 10.

[0179] Import the feature map data into the self-attention mechanism model and calculate the attention score corresponding to the feature map. Specifically, the corresponding score is obtained by calculating the score matrix Z. The calculation is as follows:

[0180] ,

[0181] Among them, Z is the final calculated score matrix, through which the attention score is obtained, Q, K, V, are the dimensions of the query matrix, key matrix, value matrix, and vectors in the feature graph, respectively.

[0182] According to an embodiment of the present invention, the S110 further includes:

[0183] Combining every two first feature sets to obtain a combined feature set;

[0184] Import the combined feature set into the self-attention mechanism model, calculate the attention score of the feature vectors, and filter out the feature sets with high scores and mark them as high-scoring feature data;

[0185] The high-scoring feature data of all combined feature sets are analyzed and screened, and based on the real-time shared data volume, feature data that meets the data volume is selected from the high-scoring feature data and marked as the second feature set.

[0186] It should be noted that if the amount of high-scoring feature data exceeds the amount of real-time shared data, random filtering can be performed to reduce the feature data to meet the data volume. Alternatively, high-scoring features can be selected based on ranking by score. The purpose of the combination is to fuse the principal component feature data obtained from different intelligent ships and filter the fused feature data to obtain high-scoring features. If there are three first feature sets, the number of pairwise combinations is three, and the three corresponding combined feature sets can be obtained through these three combinations.

[0187] According to an embodiment of the present invention, the S112 is specifically as follows:

[0188] Importing the second feature set into the image recognition model to perform object classification and object image positioning, and generate recognition results;

[0189] Based on the recognition results and target spatial information, target collaborative perception is performed, and the perception results are sent to multiple smart ships through the central platform.

[0190] It should be noted that the image recognition model can utilize models for related image feature recognition, such as yolov5, CNN-based image recognition models, support vector machines, decision trees, and other models for image recognition and classification.

[0191] In this invention, the entire intelligent ship operating system includes multiple intelligent ships and a central platform. The central platform can transmit perception and analysis results to multiple intelligent ships. Furthermore, based on the terminal's needs, a specific intelligent ship can serve as the central platform for collaborative target perception. Both the first and second feature sets include color and edge features.

[0192] It is worth mentioning here that in the traditional navigation and mission execution of intelligent ships, there is a lack of comprehensive target recognition methods for multi-source data, multi-sensor data, and multi-video source data. Traditional methods are often limited to object recognition of a single ship, and their comprehensive recognition capabilities are low. They cannot effectively combine the multi-source data of other nearby intelligent ships for shared analysis or collaborative perception, which easily forms data islands, is not conducive to the collaborative environmental perception analysis of multiple intelligent ships, and it is difficult to improve the comprehensive operation efficiency of multiple intelligent ships. Based on this, the present invention obtains videos of intelligent ships and performs feature extraction on video key frames based on the PCA analysis method. The feature extraction is used to obtain the main component features for target recognition. In the second step, a self-attention mechanism model is trained. The self-attention mechanism model is used to calculate attention scores for different feature data. Scores are calculated in the main component features (i.e., the first feature set). Based on the scores, a second feature set with a high score is selected. Based on the shared network, the intelligent ships realize fusion analysis of the source video data. After obtaining the second feature set, the target can be collaboratively perceived and efficiently recognized. The recognition results are combined with the spatial information of target perception to effectively achieve precise target perception and spatial confirmation, effectively improving the comprehensive environmental perception capability of multiple ships.

[0193] Based on the shared network, multiple smart ships can share the corresponding target perception results.

[0194] The third aspect of the present invention also provides a computer-readable storage medium, which includes a target collaborative perception program based on intelligent fleet navigation. When the target collaborative perception program based on intelligent fleet navigation is executed by a processor, the steps of the target collaborative perception method based on intelligent fleet navigation as described in any one of the above items are implemented.

[0195] The present invention discloses a target collaborative perception method and system based on intelligent fleet navigation. Target spatial information is acquired through AIS and laser radar fusion positioning, and multi-perspective video streams are collected in real time based on multiple intelligent ships. After the system extracts the video key frames, it uses Sobel edge detection and PCA analysis technology to construct an object feature set, and combines the network status dynamic evaluation to determine the data transmission volume. The feature set is analyzed for correlation using a pre-trained self-attention model to screen out features with high attention scores, thereby achieving efficient target recognition and shared perception. The present invention can significantly improve target recognition accuracy and system response speed, rationally utilize multi-intelligent ship data for target collaborative perception, and effectively improve the comprehensive environmental perception capabilities of multiple ships.

[0196] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0197] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0198] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0199] Those skilled in the art will appreciate that all or part of the steps of the above-mentioned method embodiments may be implemented by hardware associated with program instructions, and the aforementioned program may be stored in a computer-readable storage medium. When the program is executed, the program executes the steps of the above-mentioned method embodiments. The aforementioned storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0200] Alternatively, if the integrated units described above are implemented as software modules and sold or used as standalone products, they can also be stored on a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product, stored on a storage medium, includes instructions for enabling a computer device (such as a personal computer, server, or network device) to execute all or part of the methods described in various embodiments of the present invention. The aforementioned storage media include various media capable of storing program code, such as removable storage devices, ROM, RAM, magnetic disks, or optical disks.

[0201] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be covered by the scope of protection of the present invention.

Claims

1. A target collaborative perception method based on intelligent fleet navigation, characterized in that: include: S102: Among multiple intelligent ships, positioning and sensing target spatial information through AIS information and lidar detection data; S104: Based on multiple intelligent ships, multiple video data based on target perception are acquired in real time; S106: extracting key frames from the multiple video data, segmenting the key frames into object regions based on the Sobel operator, and extracting principal components of object features through a PCA analysis method to form multiple first feature sets; S108: Real-time shared data volume through intelligent ship network status assessment; S110: Using the feature map data of the existing identified objects, a self-attention mechanism model is trained, and a query matrix, a key matrix, and a value matrix are determined. Each two first feature sets are combined, and based on the self-attention mechanism model, an attention score of the combined feature set is calculated. High-scoring features are screened to obtain features that meet the real-time shared data volume and are marked as the second feature set. S112: Target detection and recognition are performed based on the second feature set using the image recognition model to generate a recognition result. Target collaborative perception is performed based on the recognition result and target spatial information, and the perception result is sent to multiple intelligent ships. The S110 includes: Obtain feature map data and feature recognition rate of identified objects from the system database; Setting attention scores based on feature recognition rates; Correlate feature graph data with attention scores; Construct a self-attention mechanism model, map the feature map data through linear transformation, and generate the initial query matrix, initial key matrix, and initial value matrix; Import the feature map data into the self-attention mechanism model and calculate the attention score corresponding to the feature map. Use the absolute difference between the attention score and the attention score as the loss function to train the self-attention mechanism model, and cyclically adjust the query matrix, initial key matrix, and initial value matrix until the loss function converges to the preset difference. After training is completed, the query matrix, key matrix and value matrix are determined; The S110 further includes: Combining every two first feature sets to obtain a combined feature set; Import the combined feature set into the self-attention mechanism model, calculate the attention score of the feature vectors, and filter out the feature sets with high scores and mark them as high-scoring feature data; The high-scoring feature data of all combined feature sets are analyzed and screened, and based on the real-time shared data volume, feature data that meets the data volume is selected from the high-scoring feature data and marked as the second feature set.

2. The target collaborative perception method based on intelligent fleet navigation according to claim 1 is characterized in that: The S102 is specifically as follows: The AIS information includes the position, heading and speed information of each smart ship; Perform spatial analysis on the LiDAR detection data to generate point cloud data, filter the point cloud data, segment different target objects, and obtain the target object bounding box based on the point cloud information; The spatial deviation of the target object is averaged based on the target object bounding box obtained by analyzing multiple intelligent ships to obtain the initial spatial information based on the point cloud information; Combined with AIS information, the position information of each smart ship is imported into the initial spatial information to generate target spatial information.

3. The target collaborative perception method based on intelligent fleet navigation according to claim 1 is characterized in that: The S104 is specifically as follows: Through the video monitoring device, multiple intelligent ships can sense surrounding target objects and take real-time video of the target range to obtain corresponding video data; The video data is stored in the smart ship database and shared through the Internet of Things.

4. The target collaborative perception method based on intelligent fleet navigation according to claim 1 is characterized in that: The S108 is specifically as follows: In a real-time cycle, the packet loss rate, network delay and network jitter of each smart ship are counted; Based on the packet loss rate, network delay and network jitter, the average value is calculated to obtain the averaged network parameters; The maximum data transmission volume in a real-time cycle is evaluated by averaging network parameters to obtain the real-time shared data volume.

5. The target collaborative perception method based on intelligent fleet navigation according to claim 1 is characterized in that: The S112 is specifically as follows: Importing the second feature set into the image recognition model to perform object classification and object image positioning, and generate recognition results; Based on the recognition results and target spatial information, target collaborative perception is performed, and the perception results are sent to multiple smart ships through the central platform.

6. A target collaborative perception system based on intelligent fleet navigation, characterized by: The system includes: a memory and a processor, wherein the memory includes a target collaborative perception program based on intelligent fleet navigation, and when the target collaborative perception program based on intelligent fleet navigation is executed by the processor, the steps of the target collaborative perception method based on intelligent fleet navigation according to any one of claims 1 to 5 are implemented.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a target collaborative perception program based on intelligent fleet navigation. When the target collaborative perception program based on intelligent fleet navigation is executed by a processor, the steps of the target collaborative perception method based on intelligent fleet navigation according to any one of claims 1 to 5 are implemented.

Images