Ship situation awareness system and method suitable for Beidou satellite high packet loss rate scene

Abstract

The invention discloses a ship situation awareness system and method suitable for a Beidou satellite high packet loss rate scene. The system comprises a ship sensor, a video monitoring device, a ship time sequence fusion module, a ship time sequence retrieval module, a ship image dimension reduction module, a Beidou satellite sending module, a Beidou satellite receiving module, a shore-end intelligent high-packet-loss interpolation module and a shore-end intelligent situation awareness module. The method deeply excavates multivariate time sequence potential relevance provided by a ship sensor, has the capability of simultaneously processing image data and sensor single-point data, provides a secondary decision basis for ship remote monitoring and management, and realizes all-day and all-weather normalized resource intercommunication and sharing of ship situation data. The method has very high economic value and scientific and technological development prospects for development of the technical level of exploration, emergency management and the like of polar region ocean equipment in China.

Description

Technical Field

[0001] The present invention relates to a ship situation awareness system, and in particular to a ship situation awareness system and method suitable for Beidou satellite high packet loss rate scenarios. Background Art

[0002] The ocean is rich in natural resources and also indirectly affects global climate and environmental changes. More and more ships are sailing to uninhabited areas to explore resources and events in the polar oceans. These ships are equipped with many types of sensors to complete engineering tasks such as marine resource development, ecological environment monitoring, and information sharing, but the attenuation of electromagnetic waves in the water makes it difficult for data to be communicated over long distances. At present, the operating land communication networks are mainly 5G, Internet, Ad hoc, etc. Their communication service coverage is too small compared to the vast ocean, and key information in deep-sea equipment cannot be transmitted using land communication networks. In contrast, Beidou satellite short message communication technology is not limited by distance in marine environment data transmission, and can provide timely and effective ship-shore information interaction and situational awareness.

[0003] Satellite communications, as the most relied-on information transmission method for data exchange between deep-sea ships and land, also play a key role in the military field. In this century, the U.S. Navy proposed the "sense-transmission-response" shore-sea integrated support system, which uses satellites to share ship data in real time to ensure the integrity of ship information, realize on-site shore-to-ship support and full-life information management, and is applied to the U.S. Nimitz aircraft carrier in service. Therefore, the situational awareness systems and methods currently studied for land networks cannot be applied to ship situational awareness in marine environments, and studying ship situational awareness under Beidou satellite channels with high packet loss rates is a huge challenge. Summary of the invention

[0004] The present invention is oriented to the strategic needs of ship data perception technology, and aims at the bottleneck problem of Beidou satellite information transmission under the background of ocean-going ship communication, and provides a ship situation awareness system and method suitable for Beidou satellite high packet loss rate scenarios. The present invention fully considers the particularity of Beidou satellite high packet loss rate communication scenarios, and designs and invents a ship image dimensionality reduction module based on the ship timing fusion module, which combines the three sub-parts of original image preprocessing, ship image dimensionality reduction, and energy judgment to reduce the impact of background pixels of ship gas turbine video monitoring on Beidou satellite transmission, increase the ratio of key equipment information to background noise information in a single frame image, and improve the effectiveness of information transmission of the ship situation awareness system; in view of the inevitable packet loss phenomenon of Beidou satellite, a shore-side intelligent high packet loss interpolation module is designed to reconstruct and recover lost data packets, and a distributed distance D is proposed. L(P,Q) Improved GAN method, which strengthens the network performance of GAN by adding a distributed distance term to the multivariate time series distance formula; in view of the particularity of ship situation awareness, on the basis of the fuzzy neural network, a sine-cosine optimization layer and its weight update method are proposed, and the real-time requirement of ship long-range situation awareness is ensured by introducing sine-cosine function terms.

[0005] The purpose of the present invention is achieved through the following technical solutions:

[0006] A ship situation awareness system applicable to the high packet loss rate scenario of Beidou satellites, including ship sensors, video monitoring equipment, ship time series fusion module, ship time series retrieval module, ship image dimensionality reduction module, Beidou satellite sending module, Beidou satellite receiving module, shore-end intelligent high packet loss interpolation module, and shore-end intelligent situation awareness module, wherein:

[0007] The ship sensors are responsible for collecting the ship's multivariate time series in real time and providing the state of the ship's power plant in the deep and far sea;

[0008] The video monitoring equipment is responsible for collecting the gas turbine video monitoring images in real time;

[0009] The ship time series fusion module is responsible for preprocessing the ship's multi-source time series collected by the ship sensors in real time;

[0010] The ship time series retrieval module is responsible for receiving the ship's multi-source time series preprocessed by the ship time series fusion module and sending the ship's multi-source time series to the Beidou satellite sending module in an optimized retrieval order;

[0011] The ship image dimensionality reduction module is responsible for performing dimensionality reduction processing on each video monitoring image collected by the video monitoring equipment in real time and sending the images that meet the Beidou satellite communication frequency after processing to the Beidou satellite sending module;

[0012] The Beidou satellite sending module is responsible for compiling the ship's multi-source time series collected by different sensors and the video monitoring images collected by the video monitoring equipment in real time into Beidou satellite data packets and arranging all the data packets in chronological order;

[0013] The Beidou satellite receiving module is responsible for receiving the data from the Beidou satellite sending module and sending the data to the shore-end intelligent high packet loss interpolation module;

[0014] The shore-end intelligent high packet loss interpolation module is responsible for further processing the received ship's multi-source time series to realize the joint reconstruction and recovery of sensor data and video images;

[0015] The shore - end intelligent perception module is responsible for receiving data from the shore - end intelligent high - packet - loss interpolation module, parsing the data of the ship gas turbine and displaying it in real time, and realizing ship situation awareness through the design and optimization of a fuzzy neural network.

[0016] A ship situation awareness method applicable to the high - packet - loss rate scenario of Beidou satellites implemented by using the above - mentioned system, the method includes the following steps:

[0017] Step 1: Taking the exhaust temperature of the power turbine of the ship gas turbine, nitrogen oxide emissions, power turbine power, and compressor inlet pressure sensors as the acquisition terminals, the video monitoring device collects the operating state of the ship gas turbine in real time to realize the simultaneous acquisition of multi - variable time series and video images;

[0018] Step 2: In the C / S local - area network communication structure, adopting a communication protocol that combines Modbus TCP and Zigbee, the real - time collected ship multi - variable time series and videos in units of image frames are packaged and sent to the ship time - series fusion module;

[0019] Step 3: In the ship time - series fusion module, using the Pearson correlation coefficient to calculate the correlation between different sensor data, and dividing it into four sub - parts: [0, 0.25] , (0.25, 0.5), [0.5, 0.75), [0.75, 1]. The data of each sub - part are respectively transmitted and processed, and the processed multi - variable time series are sent to the ship time - series retrieval module;

[0020] Step 4: The ship time - series retrieval module performs information retrieval for different sub - parts. First, all sub - nodes are completely randomly hashed to form new tree nodes, then the balance degrees of the left and right sub - trees are calculated, and only the nodes with unbalanced asymmetry are added to the tree. Finally, the retrieved data are stored in the sending queue of the Beidou satellite sending module;

[0021] Step 5: The ship image dimensionality reduction module processes the real - time monitored operating pictures of the ship gas turbine in the format of frames, and sends the dimensionality - reduced pictures to the Beidou satellite sending module at each effective communication frequency of the Beidou satellite;

[0022] Step 6: The Beidou satellite sending module arranges all data packets in chronological order, compiles the ship multi - source time series collected by different sensors and the video monitoring images collected by the video monitoring device in real time with ASCII codes. Each transmission no longer requires repeating the data types and units in the message segment of the Beidou 2.1 protocol, but is based on the keywords in the "ship situation awareness dictionary" defined by ASCII codes. The Beidou satellite receiving module also queries the keywords in the "ship situation awareness dictionary" to judge the types of sensors and video monitoring images;

[0023] Step 7: The shore - side intelligent high - packet - loss interpolation module processes the data from the Beidou satellite receiving module, and the inevitable packet loss of Beidou satellites is reconstructed and recovered in this module;

[0024] Step 8: The shore - side intelligent situation awareness module analyzes the data processed by the shore - side intelligent high - packet - loss interpolation module to achieve ship situation awareness.

[0025] Compared with the prior art, the present invention has the following advantages:

[0026] 1. The present invention designs a Beidou satellite transmission module for ship situation awareness that is more suitable for the high - packet - loss channel of Beidou satellites. Compared with the existing Beidou satellite transmission modules in ship navigation monitoring systems, the Beidou satellite transmission module designed in the present invention can simultaneously have the ability to transmit ship multivariate time series with different attributes and video surveillance images, while the existing inventions usually only have the ability to transmit ship multivariate time series or transmit video surveillance images, and the two are integrated to achieve a more comprehensive ship comprehensive situation awareness.

[0027] 2. The existing Beidou satellite image transmission system uses the method of transmitting the whole photo without considering the redundant / noise background image part in the on - site operation images of the equipment, which seriously increases the communication burden of Beidou satellites. Therefore, the present invention designs a ship image dimensionality reduction module to achieve the dimensionality reduction processing of video surveillance images, adds pixel transmission constraints based on energy judgment on the basis of the existing image transmission technology, solves the problem of the limited length of the Beidou satellite short message string, and avoids transmitting the complex background noise pixel frames of ship gas turbines.

[0028] 3. The Beidou satellite communication system on land can guarantee the installation location, but the antenna angle of the ship changes continuously during navigation, and the marine weather is much worse than that on land. The existing Beidou satellite communication technology cannot adapt to the design scenario of the present invention. Therefore, the present invention designs shore - side intelligent high - packet - loss interpolation to achieve data reconstruction and recovery in the case of Beidou satellite packet loss. Taking the data of ship gas turbines as an example, a GAN method based on the improved distributed distance D L (P,Q) is designed. By defining a new distributed distance and combining with a generative artificial intelligence model, a new interpolation model for lost packets of ship gas turbine multivariate time series is formed. Using a small amount of historical data as the training part of the model, the availability of the received data packets is guaranteed to a great extent, and the problem of high packet - loss rate in the Beidou satellite marine environment is solved.

[0029] 4. Some existing patents have designed various neural network structures, but their structures are often too complex, resulting in overfitting, and the model training time is too long to meet the real-time requirements of the system. However, in order to ensure the real-time requirements of Beidou satellite communication and ship situation awareness, the present invention designs a shore-end intelligent perception module, newly adds a sine-cosine optimization layer, and combines it with a fuzzy inference layer to form an optimized neural network structure with a trigonometric function combination to solve the real-time information transmission requirements of ship situation awareness, effectively ensuring the situation awareness of ocean-going ships under the high-packet-loss Beidou satellite channel and ensuring the reliability of technology implementation.

[0030] 5. The present invention deeply explores the potential correlation of the multivariate time series provided by ship sensors, has the ability to process image data and sensor single-point data simultaneously, provides a basis for secondary decision-making for ship remote monitoring and management, and realizes the all-day and all-weather normalized resource interconnection and sharing of ship situation data.

[0031] 6. The present invention has very high economic value and scientific and technological development prospects for the exploration of China's polar ocean equipment and the development of technical levels such as emergency management. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 Schematic diagram of the ship situation awareness system structure applicable to the high-packet-loss scenario of Beidou satellites;

[0033] Figure 2 Internal diagram of the ship image dimensionality reduction module;

[0034] Figure 3 Internal diagram of the shore-end intelligent high-packet-loss interpolation module;

[0035] Figure 4 Internal diagram of the Beidou satellite transmission module and the Beidou satellite reception module;

[0036] Figure 5 Results of the image perception part of the ship situation awareness system;

[0037] Figure 6 Results of the ship operation state perception of the ship situation awareness system. DETAILED DESCRIPTION OF THE INVENTION

[0038] The technical solutions of the present invention will be further described below in conjunction with the drawings, but are not limited thereto. Any modification or equivalent replacement of the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention shall be covered by the protection scope of the present invention.

[0039] The present invention provides a ship situation awareness system applicable to the high-packet-loss scenario of Beidou satellites, as Figure 1As shown in the figure, the system includes a ship sensor, a video monitoring device, a ship time series fusion module, a ship time series retrieval module, a ship image dimensionality reduction module, a Beidou satellite transmission module, a Beidou satellite reception module, a shore-end intelligent high packet loss interpolation module, and a shore-end intelligent situation awareness module, where:

[0040] The ship sensor is responsible for collecting the ship's multi-source time series in real time and providing the state of the deep-sea and far-sea ship power plant, including the exhaust temperature of the power turbine of the gas turbine, nitrogen oxide emissions, power turbine power, compressor inlet pressure, etc. Common sensors only collect single-type information, such as: longitude and latitude information, heading, speed, etc. However, in this invention, the multi-source time series generated by multiple sensors are fused to achieve richer ship data fusion and ensure a high level of ship situation awareness;

[0041] The video monitoring device is responsible for collecting the gas turbine video monitoring images in real time;

[0042] The ship sensor and the video monitoring device are established in a C / S local area network communication structure, and a communication protocol that combines Modbus TCP and Zigbee is used to package the ship's multi-source time series collected in real time and the video in units of image frames. The C / S local area network communication structure usually uses PLC, industrial control computers, and FPGA as the main hardware;

[0043] The ship time series fusion module is responsible for preprocessing the ship's multi-source time series collected by the ship sensor in real time. In this module, a host computer monitoring system combined with a database and real-time software is designed. Usually, the cloud platform and digital twin technology need to be introduced to optimize the display effect;

[0044] The ship time series retrieval module is responsible for receiving the ship's multi-source time series preprocessed by the ship time series fusion module and sending the ship's multi-source time series to the Beidou satellite transmission module in an optimized retrieval order;

[0045] The ship image dimensionality reduction module is responsible for reducing the dimensionality of each video monitoring image collected by the video monitoring device in real time and sending the processed images that meet the Beidou satellite communication frequency to the Beidou satellite transmission module,

[0046] The Beidou satellite transmission module is responsible for compiling the ship's multi-source time series collected by different sensors and the video monitoring images collected by the video monitoring device in real time into Beidou satellite data packets, and arranging all the data packets in chronological order. Each data packet ensures that it reaches the maximum single communication capacity of the Beidou satellite to achieve the highest efficiency of information transmission. The Beidou satellite transmission module usually has various forms, such as using a Beidou satellite handheld terminal, a Beidou satellite multi-card machine, and a Beidou satellite data transmission integrated machine as the hardware device;

[0047] The Beidou satellite receiving module is responsible for receiving the data sent by the Beidou satellite sending module and sending the data to the shore-side intelligent high-packet-loss interpolation module. The Beidou satellite receiving module also adopts a communication protocol that integrates Modbus TCP and Zigbee, and combines the display and control screen, the Beidou-3 command machine, and the cloud platform server as hardware devices;

[0048] The shore-side intelligent high-packet-loss interpolation module is responsible for further processing the received ship multi-source time series to realize the joint reconstruction and recovery of sensor data and video images;

[0049] The shore-side intelligent perception module is responsible for receiving the data from the shore-side intelligent high-packet-loss interpolation module, parsing the ship gas turbine data and displaying it in real time, and realizing ship situation perception through the design and optimization of a fuzzy neural network.

[0050] As Figure 2 shown, the ship image dimensionality reduction module includes an image preprocessing module, a background gray level calculation module, and an energy judgment module, where: the image preprocessing module is responsible for extracting the significant target part, the background gray level calculation module is responsible for optimizing the background of the useless ship gas turbine monitoring pictures, and the energy judgment module is responsible for judging whether to retain this part of the image.

[0051] As Figure 3 shown, the Beidou satellite sending module consists of two parts: a hardware structure and a Beidou communication protocol. The hardware structure includes a 24V main / backup power supply module, a Beidou / Tiantong satellite antenna, a SIM communication card parallel connection module, and a GPU deep learning module, where: the 24V main / backup power supply module is responsible for providing electrical energy, the Beidou / Tiantong satellite antenna is responsible for receiving the radio frequency signals of Beidou satellites or Tiantong satellites, the SIM communication card parallel connection module is responsible for integrating 5G communication to achieve mobile multimedia communication, and the GPU deep learning module is responsible for calculating deep learning algorithms. The satellite communication protocol adopts the Beidou 2.1 / 4.0 protocol, including a Beidou message compilation module and a data queuing and sending module, where: the Beidou message compilation module is responsible for compiling the information flow into the format specified by the Beidou protocol, and the data queuing and sending module is responsible for temporarily storing the information flow in the buffer to adapt to the communication frequency limit of Beidou short messages.

[0052] As Figure 4As shown in the figure, the Beidou satellite receiving module consists of two parts: a hardware structure and the Beidou 2.1 / 4.0 protocol. Among them: The hardware structure includes a 24V main / backup power supply module, a Beidou / Tiantong satellite antenna, a GPU deep learning module, and a shore-end intelligent information processing server. Among them: The 24V main / backup power supply module is responsible for providing electrical energy. The Beidou / Tiantong satellite antenna is responsible for receiving the radio frequency signals of Beidou satellites or Tiantong satellites. The GPU deep learning module is responsible for calculating deep learning algorithms. The shore-end intelligent information processing server is responsible for receiving Beidou short messages at the shore end. The satellite communication protocol adopts the Beidou 2.1 / 4.0 protocol, including a Beidou message parsing module and a situation awareness calculation module. Among them: The Beidou message parsing module is responsible for parsing ship situation information using the satellite protocol. The situation awareness calculation module is responsible for analyzing the real-time situation of remote ships.

[0053] The shore-end intelligent high packet loss interpolation module consists of an input layer, a sine-cosine optimization layer, a fuzzy inference layer, and an output layer. The input layer is responsible for inputting the multi-source time series information of the ship. The sine-cosine optimization layer is responsible for processing the time series. The fuzzy inference layer is responsible for improving the generalization ability of the module to different time series information by introducing fuzziness. The output layer is responsible for judging different states of ship situation awareness.

[0054] The present invention also provides a ship situation awareness method applicable to the scenario of high packet loss rate of Beidou satellites. The method includes the following steps:

[0055] Step 1: Using the exhaust gas temperature of the power turbine of the ship's gas turbine, nitrogen oxide emissions, power turbine power, and compressor inlet pressure sensors as the acquisition terminals, the video monitoring device collects the operating state of the ship's gas turbine in real time to achieve the simultaneous acquisition of multi-source time series and video images;

[0056] Step 2: In the C / S local area network communication structure, using a communication protocol that combines Modbus TCP and Zigbee, the real-time collected multi-source time series of the ship and the video in units of image frames are packaged and sent to the ship time series fusion module;

[0057] Step 3: In the ship time series fusion module, calculate the correlation between different sensor data using the Pearson correlation coefficient and divide it into four sub-parts: [0, 0.25], (0.25, 0.5), [0.5, 0.75), [0.75, 1]. The data of each sub-part are transmitted and processed respectively, and the processed multi-source time series are sent to the ship time series retrieval module;

[0058] Step 4: The ship time series retrieval module performs information retrieval for different sub - parts. Since Beidou satellite communication has requirements for real - time performance, but the existing KD - tree will continuously accumulate useless historical records when accessing the cache. Therefore, during execution, all sub - nodes are first completely randomly hashed to form new tree nodes, then the balance degrees of the left and right sub - trees are calculated, and only the nodes with unbalanced asymmetry are added to the tree. Finally, the retrieved data is stored in the sending queue of the Beidou satellite sending module;

[0059] Step 5: The ship image dimensionality reduction module processes the real - time monitored ship gas turbine operation pictures in frame format and sends the dimensionality - reduced pictures to the Beidou satellite sending module at each effective communication frequency of the Beidou satellite. The specific steps are as follows:

[0060] Step 5.1: Pre - process the original image in frame units and extract the significant target parts:

[0061]

[0062] Among them, I 1 , I 2 …, I i …, I k …, I N is each pixel, S(I k ) is the significant feature of pixel I k , D(I k , I N ) is the distance between two pixels I k , I N , S(c m ) is the significant feature of the pixel color size, c i is the pixel color size, m is the m - th pixel, n is the total number of pixels, and f i is the possibility of the pixel c i appearing.

[0063] Step 5.2: Design a ship image dimensionality reduction module more suitable for ship situation awareness, optimize the background of the useless ship gas turbine monitoring pictures, and the background gray - scale calculation module performs the following operations:

[0064]

[0065] Among them, (x, y) represents each pixel point, x and y are the features of the pixel point, A(x, y) is the image after gray - scale processing, G(x, y) is the Gaussian filter, * is the convolution operation, exp is the exponential calculation symbol, F(x, y) is the result of the background gray - scale calculation module, and β is the established adaptive coefficient.

[0066] Step 5.3: Further calculate the condition E for triggering Beidou satellite communication from the convolved imagemax and E min The energy judgment module is as follows:

[0067]

[0068] Among them, E max and E min respectively represent the maximum and minimum values of energy, g i is the gradient of each independent pixel, H max is the maximum value of the pixel gradient in a single picture, M is the total number of pixels, and σ max is the variance value of the pixel gradient.

[0069] Step 5.4: Design a new comparison mechanism to determine whether to retain this part of the image. Define a 4-dimensional sampling window and calculate the energy of each sampling window according to where i and j are the information of this pixel point, L(i, j) is the energy value of this pixel point, E max (i, j) is the maximum energy threshold, E min (i, j) is the minimum energy threshold. When E max (i, j) > L(i, j) > E min (i, j), transmit the image data of this window; when E max (i, j) ≤ L(i, j), it means that the background noise part of the ship gas turbine has not been sufficiently dimensionally reduced, then repeat Step 5.1 - Step 5.3; when L(i, j) ≤ E min (i, j), it means that this part is redundant data and can be ignored.

[0070] Step 6: The Beidou satellite sending module arranges all data packets in chronological order, compiles the multi-source time series of the ship collected by different sensors and the video surveillance images collected in real time by the video surveillance equipment in ASCII code. Each transmission no longer needs to repeat the data type and unit in the message segment of the Beidou 2.1 protocol, but is based on the keywords in the "Ship Situation Awareness Dictionary" defined in ASCII code. The Beidou satellite receiving module also queries the keywords in the "Ship Situation Awareness Dictionary" to judge the types of sensors and video surveillance images.

[0071] Step 7: The shore-end intelligent high-packet-loss interpolation module processes the data of the Beidou satellite receiving module. The inevitable packet loss of the Beidou satellite is reconstructed and recovered in this module. The specific steps are as follows:

[0072] Step 7.1: The shore-end intelligent high-packet-loss interpolation module receives the incomplete ship gas turbine data output by the Beidou satellite receiving module and calculates the distance between any two multi-source time series within a Beidou satellite short message data packet:

[0073]

[0074] Among them, H represents the distance, P and Q are two different time series of marine gas turbine sensors, k is the k-th time series, and p and q are the probability density functions of two different time series.

[0075] Step 7.2: Since the distance formula in Step 7.1 only calculates the difference in the time domain, the distribution distance D L (P, Q) is defined to optimize the distance between the two time series of marine gas turbines:

[0076]

[0077] where m and n are their lengths, x i , y i are the values of two different time series at the i-th moment, and are the expectations of the two time series, f(x) and f(y) are the numerical magnitudes of the time series x and y, and f(x i ) and f(y i ) are the values of the time series x and y at the i-th moment.

[0078] Step 7.3: Substitute two time series with different lengths into D L (P, Q), and further calculate the time series distance with distribution characteristics. The formula is as follows:

[0079]

[0080] where D L (P, Q) is the newly proposed distribution distance.

[0081] Step 7.4: Calculate the distance between the same kind of marine time series at different moments. Let f(x i ) T f(x j ) = k(x i , x j ) represent it, then there is:

[0082]

[0083] where f(x i ) T f(x j ) = k(x i , x j ) is a simplified representation method.

[0084] Step 7.5: In the existing GAN model, the incomplete actual data of the marine gas turbine is used as the input of the GAN model in sequence, and the reconstructed and restored data is also the input of the GAN model. Different from the existing GAN model, the distance calculation is more accurate, and the distance function between the generated data and the adversarial data is redefined as follows:

[0085]

[0086] Preferably, |m - n| < 76 is satisfied to ensure that there is no serious length offset phenomenon between the two multivariate time series of the marine gas turbine, so as to prevent the GAN model from being unable to mine the correlation between different attributes.

[0087] Step 8: The shore - end intelligent situation awareness module analyzes the data processed by the shore - end intelligent high - packet - loss interpolation module to realize ship situation awareness. The specific steps are as follows:

[0088] Step 8.1: The shore - end intelligent perception module perceives the output of the shore - end intelligent high - packet - loss interpolation module in real time, adds sine and cosine function terms on the basis of the existing fuzzy neural network, which is more suitable for the real - time requirements of ship remote situation awareness, and is realized by the following formula:

[0089]

[0090] Among them, are the mean and variance of the i - th multivariate time series f of the ship sensor i 1 at the j - th network node, N hide is the number of neurons and the output value of this aggregation attribution layer, and the superscript represents the layer number of the network where it is located;

[0091] Step 8.2: Optimize the existing fuzzy neural network to establish a sine - cosine optimization layer suitable for ship situation awareness:

[0092]

[0093] Among them, Q is the maximum number of functions composed of sine and cosine, Ω(:, 1), Ω(:, 2), Ω(:, 3) are composed of 1, 2, 3 basis functions respectively, w lj is the weight of Ω′ l and the j - th neuron;

[0094] Step 8.3: Design the newly proposed sine - cosine optimization layer, and the weight update method is as follows:

[0095]

[0096] Among them, Ω l is the l - th basis function value of the input variable, w ljis Ω l and the weight of the j-th neuron, w ij (k - 1) is the connection weight between the positive and cosine optimization layer and the fuzzy inference layer at time k - 1, Δw ij used to represent gradient iteration, α is a given coefficient, η is the learning rate, E, e r are the loss function and the single-layer network error respectively, are the output values of the output layer and the fuzzy inference layer respectively, is the output value of the input layer and the positive and cosine optimization layer, ν jr is the connection weight between the fuzzy inference layer and the output layer, N in , N out is the number of neurons in the input / output layer, w lj is the weight between the l, j neurons, N hide is the number of neurons in the hidden layer, b js is the bias value between the j, s layers, exp is the exponential calculation symbol, r = 1, 2,..., N out is the neuron sorting situation of the output layer.

[0097] Multiple situations of the output layer correspond to different states of ship situation awareness. Online update the model parameters of the shore-end intelligent perception module, and finally complete the implementation of the ship situation awareness system suitable for the high packet loss rate scenario of Beidou satellites. The perception results are as Figure 5 and Figure 6 shown.

Claims

1. A ship situation awareness system suitable for Beidou satellite high packet loss rate scenarios, characterized by The system includes ship sensors, video monitoring equipment, ship time series fusion module, ship time series retrieval module, ship image dimension reduction module, Beidou satellite transmission module, Beidou satellite receiving module, shore intelligent high packet loss interpolation module, shore intelligent situation awareness module, among which: The ship sensor is responsible for collecting the multivariate time series of the ship in real time and providing the status of the power plant of the deep-sea ship; The video monitoring equipment is responsible for collecting the video monitoring images of the gas turbine in real time; The ship time series fusion module is responsible for preprocessing the ship multi-source time series collected in real time by the ship sensors; The ship time series retrieval module is responsible for receiving the ship multi-source time series preprocessed by the ship time series fusion module, and sending the ship multi-source time series to the Beidou satellite sending module in an optimized retrieval order; The ship image dimension reduction module is responsible for performing dimension reduction processing on each video surveillance image collected in real time by the video surveillance equipment, and sending the processed images that meet the Beidou satellite communication frequency to the Beidou satellite sending module; The Beidou satellite sending module is responsible for compiling the multi-source time series of ships collected by different sensors and the video surveillance images collected in real time by video surveillance equipment into Beidou satellite data packets, and arranging all data packets in chronological order; The Beidou satellite receiving module is responsible for receiving data from the Beidou satellite sending module and sending the data to the shore-side intelligent high-packet loss interpolation module; The shore-side intelligent high-packet loss interpolation module is responsible for further processing the received multi-source time series of the ship to achieve joint reconstruction and recovery of sensor data and video images; The shore-side intelligent perception module is responsible for receiving data from the shore-side intelligent high-packet loss interpolation module, parsing the ship's gas turbine data and displaying it in real time, and realizing ship situation awareness by designing and optimizing the fuzzy neural network.

2. The ship situation awareness system suitable for Beidou satellite high packet loss rate scenarios according to claim 1 is characterized in that The ship image dimensionality reduction module includes an image preprocessing module, a background grayscale calculation module and an energy judgment module, wherein: the image preprocessing module is responsible for extracting the significant target part, the background grayscale calculation module is responsible for optimizing the background of the useless ship gas turbine monitoring picture, and the energy judgment module is responsible for judging whether to retain the part of the image.

3. The ship situation awareness system suitable for Beidou satellite high packet loss rate scenarios according to claim 1 is characterized in that The Beidou satellite transmission module consists of two parts: hardware structure and Beidou communication protocol, among which: The hardware structure includes a 24V main / backup power supply module, a Beidou / Tiantong satellite antenna, a SIM communication card parallel module, and a GPU deep learning module, wherein: the 24V main / backup power supply module is responsible for providing power energy, the Beidou / Tiantong satellite antenna is responsible for receiving the radio frequency signal of the Beidou satellite or Tiantong satellite, the SIM communication card parallel module is responsible for integrating 5G communication to realize mobile multimedia communication, and the GPU deep learning module is responsible for calculating the deep learning algorithm; The satellite communication protocol adopts the Beidou 2.1 / 4.0 protocol, including a Beidou message compilation module and a data queuing and sending module, wherein: the Beidou message compilation module is responsible for compiling the information flow into the format specified by the Beidou protocol, and the data queuing and sending module is responsible for temporarily storing the information flow in a buffer to adapt to the communication frequency limit of the Beidou short message.

4. The ship situation awareness system suitable for Beidou satellite high packet loss rate scenarios according to claim 1 is characterized in that The Beidou satellite receiving module consists of two parts: hardware structure and Beidou 2.1 / 4.0 protocol, among which: The hardware structure includes a 24V main / backup power supply module, a Beidou / Tiantong satellite antenna, a GPU deep learning module, and a shore-side intelligent information processing server, wherein: the 24V main / backup power supply module is responsible for providing power energy, the Beidou / Tiantong satellite antenna is responsible for receiving the radio frequency signal of the Beidou satellite or Tiantong satellite, the GPU deep learning module is responsible for calculating the deep learning algorithm, and the shore-side intelligent information processing server is responsible for receiving the Beidou short message at the shore end; The satellite communication protocol adopts Beidou 2.1 / 4.0 protocol, including Beidou message parsing module and situation awareness calculation module, wherein: Beidou message parsing module is responsible for parsing ship situation information using satellite protocol, and situation awareness calculation module is responsible for analyzing the real-time situation of remote ships.

5. The ship situation awareness system suitable for Beidou satellite high packet loss rate scenarios according to claim 1 is characterized in that The shore-side intelligent high-packet loss interpolation module consists of an input layer, a sine-cosine optimization layer, a fuzzy reasoning layer, and an output layer. The input layer is responsible for inputting the multivariate time series information of the ship, the sine-cosine optimization layer is responsible for processing the time series, the fuzzy reasoning layer is responsible for improving the module's generalization ability for different time series information by introducing fuzziness, and the output layer is responsible for judging the different states of ship situation awareness.

6. A method for ship situation awareness applicable to Beidou satellite high packet loss rate scenarios using the system described in any one of claims 1 to 5, characterized in that The method comprises the following steps: Step 1: Using the ship gas turbine power turbine exhaust temperature, nitrogen oxide emissions, power turbine power, and compressor inlet pressure sensors as the acquisition end, the video monitoring equipment collects the operating status of the ship gas turbine in real time, realizing the simultaneous acquisition of multivariate time series and video images; Step 2: In the C / S LAN communication structure, the Modbus TCP and Zigbee fusion communication protocol is used to package the real-time collected ship multivariate time series and the video in image frames and send them to the ship time series fusion module; Step 3: In the ship time series fusion module, the Pearson correlation coefficient is used to calculate the correlation between different sensor data and divided into four sub-parts: [0, 0.25] , (0.25,0.5),[0.5,0.75),[0.75,1], each sub-part data is transmitted and processed separately, and the processed multivariate time series is sent to the ship time series retrieval module; Step 4: The ship time series retrieval module performs information retrieval of different sub-parts. First, all sub-nodes are completely randomly hashed to form new tree nodes, and then the balance of the left and right sub-trees is calculated. Only the nodes with asymmetric balance are added to the tree. Finally, the retrieved data is stored in the sending queue of the Beidou satellite sending module. Step 5: The ship image dimension reduction module processes the real-time monitored ship gas turbine operation picture in the format of a frame, and sends the dimension-reduced picture to the Beidou satellite sending module at each effective communication frequency of the Beidou satellite; Step 6: The Beidou satellite sending module arranges all data packets in chronological order, and compiles the multi-source time series of ships collected by different sensors and the video surveillance images collected in real time by video surveillance equipment in ASCII code. Each transmission no longer needs to repeat the data type and unit in the Beidou 2.1 protocol message segment, but is based on the keywords in the "Ship Situation Awareness Dictionary" defined by ASCII code. The Beidou satellite receiving module also queries the keywords in the "Ship Situation Awareness Dictionary" to determine the types of sensors and video surveillance images; Step 7: The shore-side intelligent high-packet loss interpolation module processes the data of the Beidou satellite receiving module. The inevitable data packet loss of the Beidou satellite is reconstructed and restored in this module; Step 8: The shore-side intelligent situational awareness module analyzes the data processed by the shore-side intelligent high packet loss interpolation module to achieve ship situational awareness.

7. The ship situation awareness method applicable to Beidou satellite high packet loss rate scenario according to claim 6 is characterized in that The specific steps of step 5 are as follows: Step 5.1: Preprocess the original image in frames to extract the salient target parts: Among them, I1, I2…, I i …,I k …,I N is each pixel, S(I k ) is pixel I k The salient feature of D(I k ,I N ) are two pixels I k ,I N The distance between m ) is a significant feature of pixel color size, c i is the pixel color size, m is the mth pixel, n is the total number of pixels, and f i is the pixel c i The possibility of occurrence; Step 5.2: Design a ship image dimensionality reduction module for ship situation awareness and optimize the background of useless ship gas turbine monitoring images: Among them, (x, y) is the standard for each pixel, x, y are the features of the pixel, A(x, y) is the image after grayscale processing, G(x, y) is the Gaussian filter, * is the convolution operation, exp is the exponential calculation symbol, F(x, y) is the result of the background grayscale calculation module, and β is the established adaptive coefficient; Step 5.3: The convolution processed image is further used to calculate the condition E that triggers Beidou satellite communication max and E min : Among them, E max and E min Represent the maximum and minimum energy, g i is the gradient of each individual pixel, H max is the maximum value of the pixel gradient in a single image, M is the total number of pixels, σ max is the variance value of pixel gradient; Step 5.4: Design a comparison mechanism to determine whether to keep the image, define a 4-dimensional sampling window, and Calculate the energy of each sampling window, where i, j is the information of the pixel, L(i, j) is the energy value of the pixel, E max (i,j) is the maximum energy threshold, E min (i,j) is the minimum energy threshold, when E max (i,j)>L(i,j)>E min (i, j) when the image data of the window is transmitted; when E max When L(i,j)≤E(i,j), it means that the background noise of the ship gas turbine has not been fully reduced in dimension, so repeat steps 5.1 to 5.3; when L(i,j)≤E min (i,j) means that this part is redundant data and is ignored.

8. The ship situation awareness method applicable to Beidou satellite high packet loss rate scenario according to claim 6 is characterized in that The specific steps of step 7 are: Step 7.1: The shore-side intelligent high packet loss interpolation module receives the incomplete ship gas turbine number output by the Beidou satellite receiving module, and calculates the distance between any two multivariate time series in a Beidou satellite short message data packet: Where H represents the distance, P and Q are two different ship gas turbine sensor time series, k is the kth time series, and p and q are the probability density functions of two different time series; Step 7.2: Define the distribution distance D L (P,Q) to optimize the distance between the two time series of ship gas turbines: Among them, mn is the length of the two, x i ,y i are the values ​​of two different time series at time i, and is the expectation of two time series, f(x) and f(y) are the numerical values ​​of the time series x and y, f(x i ) and f(y i ) is the value of the time series x, y at time i; Step 7.3: Bring two time series of different lengths into D L (P,Q), further calculate the time series distance D with distribution characteristics L (P,Q), the formula is as follows: Step 7.4: Calculate the distance of the same ship time series at different times, using f(x i ) T f(x j )=k(x i ,x j ) to represent it, then: Among them, f(x i ) T f(x j )=k(x i ,x j ) is a simplified representation method; Step 7.5: The distance function between generated data and adversarial data is redefined as follows:

9. The ship situation awareness method applicable to Beidou satellite high packet loss rate scenarios according to claim 6 is characterized in that The specific steps of step 8 are as follows: Step 8.1: The shore intelligent perception module perceives the output of the shore intelligent high packet loss interpolation module in real time, and adds sine and cosine function terms on the basis of the existing fuzzy neural network, which is more suitable for the real-time requirements of ship remote situation awareness, and is realized by the following formula: Among them, m ij , are the multivariate time series f of the i-th ship sensor i 1 The mean and variance at the jth network node, N hide is the number of neurons and output value of the layer to which the aggregation belongs, and the upper right corner indicates the number of layers in the network; Step 8.2: Optimize the existing fuzzy neural network and establish a sine-cosine optimization layer suitable for ship situation awareness: Among them, Q is the maximum number of functions composed of sine and cosine, Ω(:,1),Ω(:,2),Ω(:,3) are composed of 1, 2, and 3 basis functions respectively, and w lj is Ω′ l and the weight of the jth neuron; Step 8.3: Design the newly proposed sine-cosine optimization layer, and the weight update method is as follows: Among them, Ω l is the lth basis function value of the input variable, w lj is Ω l and the weight of the jth neuron, w ij (k-1) is the connection weight between the sine-cosine optimization layer and the fuzzy reasoning layer at time k-1, Δw ij It is used to represent gradient iteration, α is a given coefficient, η is the learning rate, E,e r They are the loss function and the single-layer network error, are the output values ​​of the output layer and the fuzzy inference layer respectively, is the output value of the input layer and the sine-cosine optimization layer, ν jr is the connection weight between the fuzzy inference layer and the output layer, N in ,N out is the number of neurons in the input / output layer, w lj is the weight between the lth and jth neurons, N hide is the number of neurons in the hidden layer, b js is the bias value between layers j and s, exp is the exponential calculation symbol, r=1,2,...,N out is the neuron arrangement of the output layer.

Images