Lightweight ocean data acquisition and processing system and method based on edge calculation

Abstract

The invention provides a lightweight ocean data acquisition and processing system and method based on edge computing, and the system comprises a self-adaptive data compression module which is used for carrying out the self-adaptive real-time compression of original ocean data acquired by a buoy terminal, obtaining compressed ocean monitoring data, and transmitting the compressed ocean monitoring data to an edge computing node; the data processing module is used for decompressing and preprocessing the compressed marine monitoring data by the edge computing node and inputting the preprocessed marine monitoring data into a marine anomaly diagnosis lightweight model for real-time anomaly detection; and the data transmission module is used for sending the ocean monitoring data processed by the edge node and the anomaly detection result to a shore data center by adopting an incremental transmission mode. By integrating a sensor data compression technology and an ocean anomaly diagnosis lightweight model, real-time acquisition, efficient compression, rapid processing and anomaly detection of ocean data in an open sea environment are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of marine environment monitoring data transmission, and particularly to a lightweight marine data acquisition and processing system and method based on edge computing. Background Art

[0002] With the continuous deepening of marine exploration and development activities, the demand for real-time and accurate monitoring of marine environment data is becoming increasingly urgent. However, the marine environment is complex and changeable, and traditional data acquisition and processing methods face many challenges: on the one hand, the communication conditions in the open sea area are poor, the network bandwidth is limited, and the transmission of a large amount of raw data is extremely difficult, which is likely to cause data transmission delays or even interruptions, and cannot meet the real-time requirements; on the other hand, if all the massive marine data is transmitted to the cloud or land data centers for processing, it will not only consume huge communication resources and storage resources, but also increase the time cost of data processing, and it is difficult to achieve timely response to abnormal situations in the marine environment.

[0003] Currently, marine data acquisition terminals usually have high power consumption and are difficult to operate stably for a long time in the open sea environment. Moreover, the data collected by sensors has not been effectively compressed, and the data volume is huge, which brings great pressure to transmission and subsequent processing. At the same time, in terms of marine environment anomaly detection, existing detection models are often highly complex and cannot operate efficiently on edge devices with limited resources, resulting in the inability to identify abnormal conditions in the marine environment in a timely and accurate manner. Summary of the Invention

[0004] The present invention provides a lightweight marine data acquisition and processing system and method based on edge computing, which realizes real-time acquisition, high-efficiency compression, rapid processing and anomaly detection of marine data in the open sea environment by integrating sensor data compression technology and a lightweight marine anomaly diagnosis model, so as to solve the above problems existing in the prior art.

[0005] The present invention provides a lightweight marine data acquisition and processing system based on edge computing, including: An adaptive data compression module, which is used to adaptively and real-time compress the original marine data collected by the buoy terminal, and obtain compressed marine monitoring data to be sent to the edge computing node; A data processing module, which is used for the edge computing node to decompress and preprocess the compressed marine monitoring data, and input the preprocessed marine monitoring data into a lightweight marine anomaly diagnosis model for real-time anomaly detection; A data transmission module, which is used to adopt an incremental transmission method to send the marine monitoring data and anomaly detection results processed by the edge node to the onshore data center.

[0006] Preferably, in a lightweight marine data acquisition and processing system based on edge computing, the adaptive data compression module includes: A data acquisition unit for acquiring the original marine monitoring data collected by the buoy terminal; A data feature extraction unit for extracting data change features from the original marine monitoring data corresponding to the current buoy terminal to obtain data change features and data representation features; A data compression unit for determining the compression ratio of the original marine monitoring data based on the data change features and data representation features, in combination with the mapping relationship between data features and compression parameters, and compressing the original marine data according to the compression ratio to obtain compressed marine monitoring data; An internal transmission unit for sending the compressed marine monitoring data to the edge node of the buoy terminal.

[0007] Preferably, in a lightweight marine data acquisition and processing system based on edge computing, the data compression unit further includes: A data analysis subunit for acquiring the original marine data collected by multiple sensors within a preset acquisition period based on the edge node, generating multiple time series according to the time axis, and respectively comparing adjacent values of each time series to obtain the data error between adjacent data of the same type of original marine data; Obtain the data error corresponding to the same time series, generate a data floating sequence, randomly select the data floating sequence based on a preset byte window, and calculate the data information entropy of the selected byte segment based on the selected byte segment; A first mapping relationship determination subunit for, when the data information entropy is less than a preset value, acquiring the data representation features of the original marine data of the data floating sequence corresponding to the data information entropy, and based on the data representation features, using big data acquisition technology in combination with the data information entropy to obtain multiple compressed similar data, and calculating the hash value corresponding to each compressed similar data, and based on the hash value, judging the Hamming distance between the hash value and its corresponding standard hash value; Perform screening based on the Hamming distance to obtain the minimum-loss compressed similar data, and obtain the compression parameter corresponding to the minimum-loss compressed similar data as the optimal compression parameter; Classify the data floating sequence based on the data information entropy and data representation features, and determine the target data types of the original marine data corresponding to the data floating sequences of the same category; Extract and compare the data change features of the target data types respectively to obtain public data features, and obtain the data features of the target data types based on the public data features and data representation features corresponding to the target data types; Establish a mapping relationship between data features and compression parameters based on the data features and the optimal compression parameters.

[0008] Preferably, in a lightweight ocean data acquisition and processing system based on edge computing, the data compression unit further includes: The second mapping relationship determination subunit is used for determining the mutation node of the corresponding original ocean data based on the data error between adjacent data in the data floating sequence when the data information entropy is greater than or equal to a preset value, and based on the data error, segmenting the original ocean data in the time series based on the mutation node to obtain time subsequences corresponding to multiple time periods, and obtaining each time subsequence respectively: Combined with the data length of all time subsequences, the preset window is adaptively adjusted to obtain the full sequence adaptation window. Based on the full sequence adaptation window, the data sub-information entropy corresponding to each time subsequence is calculated respectively, and the data sub-information entropy of each time subsequence is compared to obtain the information entropy error. When the information entropy errors are all within the preset range, the data realization characteristics of the original ocean data of the current type are obtained, based on the data realization characteristics, combined with the average information entropy corresponding to the data sub-information entropy, a large data collection technology is used to obtain a variety of compressed similar data, and the hash value corresponding to each compressed similar data is calculated, and based on the hash value, the Hamming distance between the hash value and its corresponding standard hash value is determined; Screening is performed based on the Hamming distance to obtain minimum loss compressed similar data, and compression parameters corresponding to the minimum loss compressed similar data are obtained as optimal compression parameters; The data change features of the original ocean data corresponding to each time subsequence are extracted and compared to obtain the public data sub-features. Based on the public data sub-features, data performance features and the time points corresponding to the mutation nodes, the data features of the current type of ocean data are obtained. Based on the data characteristics and the optimal compression parameters, a mapping relationship between the data characteristics and the compression parameters is established; When the information entropy errors are not within the preset interval, based on the time point corresponding to the mutation point, determine the data mutation distribution characteristics within the preset collection period, based on the data mutation distribution characteristics, combined with the performance characteristics of the current data, adopt big data collection technology to obtain a variety of compressed similar data, and calculate the hash value corresponding to each compressed similar data, based on the hash value, determine the Hamming distance between the hash value and its corresponding standard hash value; Screening is performed based on the Hamming distance to obtain minimum loss compressed similar data, and compression parameters corresponding to the minimum loss compressed similar data are obtained as optimal compression parameters; Based on the data mutation distribution characteristics and data performance characteristics, the data characteristics of the current type of ocean data are obtained; Based on the data characteristics and the optimal compression parameters, establish the mapping relationship between the data characteristics and the compression parameters.

[0009] Preferably, in a lightweight marine data acquisition and processing system based on edge computing, the data processing module includes: A training set generation unit, configured to obtain a large amount of historical marine data as marine sample data, and perform abnormal data marking on the marine sample data to obtain an abnormal marine data set; A model training and deployment module, configured to train a lightweight marine anomaly diagnosis model based on the abnormal marine data set, and deploy the trained lightweight marine anomaly diagnosis model to the edge computing nodes of each sensor.

[0010] Preferably, in a lightweight marine data acquisition and processing system based on edge computing, the data processing module includes: A data decompression processing unit, configured to decompress the compressed marine monitoring data received by the edge node to obtain decompressed data, and compare the decompressed data with the previous received data of its corresponding type to determine whether the data has changed; If a change occurs, perform corresponding preprocessing on the decompressed data based on the type of the decompressed data, and store and record the decompressed data; Otherwise, skip the decompressed data, control the edge node to maintain the previous anomaly detection result, and only record the receiving time point for storage; An anomaly monitoring unit, configured to input the preprocessed decompressed data into the lightweight marine anomaly diagnosis model to determine whether the current marine data is abnormal.

[0011] Preferably, in a lightweight marine data acquisition and processing system based on edge computing, the data transmission module includes: A transmission processing unit, configured to generate a warning report based on the abnormal marine data, and generate an instruction for unchanged data for the type of unchanged data; Generate a transmission data packet based on the warning report, the instruction for unchanged data, and the preprocessed changed marine data; A shore data update unit, configured to parse the transmission data packet after receiving it at the shore data center, and determine the type of unchanged marine data based on the instruction for unchanged data; Respectively obtain the receiving time points corresponding to the unchanged marine data and the changed marine data, and update the unchanged marine data and the changed marine data respectively based on the receiving time points; A shore automatic warning unit, configured to generate a corresponding anomaly warning signal based on the abnormal marine data and issue a warning when the shore data center receives the warning report.

[0012] The present invention provides a lightweight marine data acquisition and processing method based on edge computing, including: Step 1: Adaptive real-time compression is performed on the original marine data collected by the buoy terminal to obtain compressed marine monitoring data and send it to the edge computing node; Step 2: The edge computing node decompresses and preprocesses the compressed marine monitoring data, and inputs the preprocessed marine monitoring data into the lightweight marine anomaly diagnosis model for real-time anomaly detection; Step 3: Adopting an incremental transmission method, the marine monitoring data and anomaly detection results processed by the edge node are sent to the onshore data center.

[0013] Preferably, in a lightweight marine data acquisition and processing method based on edge computing, Step 1 includes: Obtain the original marine monitoring data collected by the buoy terminal; Extract the data change characteristics of the original marine monitoring data corresponding to the current buoy terminal to obtain the data change characteristics and data performance characteristics; Based on the data change characteristics and data performance characteristics, combined with the mapping relationship between data characteristics and compression parameters, determine the compression ratio of the original marine monitoring data, and compress the original marine data according to the compression ratio to obtain compressed marine monitoring data; Send the compressed marine monitoring data to the edge node of the buoy terminal.

[0014] Compared with the prior art, the present invention has at least the following beneficial effects: The present invention adaptively and real - time compresses the original ocean data, which can significantly reduce the data volume and be transmitted to the edge computing node through the communication link faster, effectively shortening the time interval from data acquisition to subsequent processing, improving the response speed of the entire system to changes in the ocean environment, providing strong support for real - time monitoring and early warning, and adjusting the compression strategy according to the dynamic characteristics of the data to ensure that various ocean data can effectively reduce the data volume while maintaining the integrity and availability of the data, enhancing the reliability of data processing. Subsequently, by decompressing the compressed ocean monitoring data at the edge computing node, it ensures that the data is restored to a state close to the original, can guarantee the accuracy and integrity of the data input into the lightweight ocean anomaly diagnosis model, thereby improving the reliability of the anomaly detection results, effectively avoiding misjudgment or missed judgment caused by data quality problems. At the same time, the lightweight processing of the model also reduces the storage requirements during the operation of the edge node, and ensures the timely discovery of ocean anomalies. By adopting the incremental transmission method, only the changed part of the data processed by the edge node relative to the previous transmission and the anomaly detection results are transmitted, which can avoid repeated transmission of a large amount of unchanged data, significantly reduce the data transmission volume, thereby greatly shortening the data transmission time, improving the data transmission efficiency, further enhancing the real - time performance of the system. At the same time, the smaller data transmission volume reduces the probability of data loss or error during transmission, improving the stability and reliability of data transmission in the complex communication environment of the open sea where signals are vulnerable to interference.

[0015] Other features and advantages of the present invention will be described in the following specification, and, in part, will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention can be achieved and obtained through the structure specifically pointed out in this application document.

[0016] The technical solution of the present invention will be further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the specification, and are used to explain the present invention together with the embodiments of the present invention, and do not constitute a limitation to the present invention. In the accompanying drawings: Figure 1 is a structural diagram of a lightweight ocean data acquisition and processing system based on edge computing according to the present invention; Figure 2 is a structural diagram of an adaptive data compression module of a lightweight ocean data acquisition and processing system based on edge computing according to the present invention; Figure 3 is a structural diagram of a data processing module of a lightweight ocean data acquisition and processing system based on edge computing according to the present invention; Figure 4This is a structural diagram of the data transmission module of a lightweight ocean data acquisition and processing system based on edge computing according to the present invention. Specific embodiments

[0019] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only for the purpose of illustrating and explaining the present invention, and are not intended to limit the present invention.

[0020] Embodiment 1: The present invention provides a lightweight ocean data acquisition and processing system based on edge computing, as Figure 1 shown, including: An adaptive data compression module for adaptively and real-time compressing the original ocean data collected by the buoy terminal to obtain compressed ocean monitoring data and sending it to the edge computing node; A data processing module for the edge computing node to decompress and preprocess the compressed ocean monitoring data, and input the preprocessed ocean monitoring data into the lightweight ocean anomaly diagnosis model for real-time anomaly detection; A data transmission module for using the incremental transmission method to send the ocean monitoring data and anomaly detection results processed by the edge node to the onshore data center.

[0021] The beneficial effects of the above technical solutions: The present invention adaptively and real-time compresses the original ocean data, which can greatly reduce the data volume, can be transmitted to the edge computing node faster through the communication link, effectively shortens the time interval from data acquisition to subsequent processing, improves the response speed of the entire system to changes in the ocean environment, provides strong support for real-time monitoring and early warning, and adjusts the compression strategy according to the dynamic characteristics of the data to ensure that various ocean data can effectively reduce the data volume during the compression process, while maintaining the integrity and availability of the data, improving the reliability of data processing. Subsequently, by decompressing the compressed ocean monitoring data at the edge computing node, it ensures that the data is restored to a state close to the original, can ensure the accuracy and integrity of the data input into the lightweight ocean anomaly diagnosis model, thereby improving the reliability of the anomaly detection results, effectively avoiding misjudgment or missed judgment caused by data quality problems. At the same time, the lightweight processing of the model also reduces the storage requirements during the operation of the edge node, and also ensures the timely discovery of ocean anomalies. By using the incremental transmission method, only the changed part of the data processed by the edge node relative to the previous transmission and the anomaly detection results are transmitted, which can avoid repeated transmission of a large amount of unchanged data, significantly reduce the data transmission volume, thereby greatly shortening the data transmission time, improving the data transmission efficiency, further enhancing the real-time performance of the system. At the same time, the smaller data transmission volume reduces the probability of data loss or error during transmission, and improves the stability and reliability of data transmission in the complex communication environment of the open sea where the signal is vulnerable to interference. Example 2:

[0022] Based on Example 1, the adaptive data compression module, as Figure 2 shown, includes: A data acquisition unit for acquiring the original ocean monitoring data collected by the buoy terminal; A data feature extraction unit for extracting data change features from the original ocean monitoring data corresponding to the current buoy terminal, to obtain data change features and data representation features; A data compression unit for determining the compression ratio of the original ocean monitoring data based on the data change features according to the data representation features, in combination with the mapping relationship between data features and compression parameters, and compressing the original ocean data according to the compression ratio to obtain compressed ocean monitoring data; An internal transmission unit for sending the compressed ocean monitoring data to the edge node of the buoy terminal.

[0023] Beneficial effects of the above technical solution: The present invention directly acquires the original ocean monitoring data from the buoy terminal, can timely capture the dynamic changes of the ocean environment, and by extracting the data change features from the original ocean monitoring data, can deeply understand the internal laws and change trends of the data. For example, for seawater temperature data, the change rate of temperature over time, periodic fluctuation characteristics, etc. can be extracted; for wave height data, features such as the change amplitude and frequency of wave crests and troughs can be obtained. At the same time, the extraction of data representation features, such as the distribution and dispersion degree of data, further enriches the understanding of the data. Different types of ocean monitoring data have different characteristics, and the data feature extraction unit can analyze the uniqueness of each type of data, enabling the entire adaptive data compression module to better adapt to various data situations, improving the flexibility and reliability of data processing, and determining the compression ratio of the original ocean monitoring data based on the data change features and data representation features, in combination with the mapping relationship between data features and compression parameters, achieving the optimization of data compression, and quickly sending the compressed ocean monitoring data to the edge node of the buoy terminal, ensuring the timeliness of ocean anomaly data. Example 3:

[0024] Based on Example 2, the data compression unit includes: A data analysis subunit for, based on the edge node, acquiring the original ocean data collected by multiple sensors within a preset acquisition period, generating multiple time series according to the time axis, and respectively comparing adjacent values of each time series to obtain the data error between adjacent data of the same type of original ocean data; Obtain the data error corresponding to the same time series, generate a data floating sequence, randomly select from the data floating sequence based on a preset byte window, and calculate the data information entropy of the selected byte segment based on the selected byte segment; The first mapping relationship determination subunit is configured to, when the data information entropy is less than a preset value, obtain the data representation characteristics of the original ocean data of the data floating sequence corresponding to the data information entropy, and based on the data representation characteristics, combine the data information entropy and use big data acquisition technology to obtain multiple compressed similar data, calculate the hash value corresponding to each compressed similar data, and based on the hash value, judge the Hamming distance between the hash value and its corresponding standard hash value; Perform screening based on the Hamming distance to obtain the minimum-loss compressed similar data, and obtain the compression parameter corresponding to the minimum-loss compressed similar data as the optimal compression parameter; Classify the data floating sequence based on the data information entropy and the data representation characteristics, and determine the target data types of the original ocean data corresponding to the data floating sequences of the same category; Extract and compare the data change characteristics of the target data types respectively to obtain the public data characteristics, and obtain the data characteristics of the target data types based on the public data characteristics and the data representation characteristics corresponding to the target data types; Based on the data characteristics and the optimal compression parameter, establish a mapping relationship between the data characteristics and the compression parameter; The second mapping relationship determination subunit is configured to, when the data information entropy is greater than or equal to the preset value, based on the data error between adjacent data in the data floating sequence, determine the mutation nodes of the corresponding original ocean data based on the data error, segment the original ocean data in the time series based on the mutation nodes to obtain time subsequences corresponding to multiple time periods, and respectively obtain each time subsequence: Combine the data lengths of all time subsequences to adaptively adjust the preset window to obtain a full-sequence adaptation window, calculate the data sub-information entropy corresponding to each time subsequence based on the full-sequence adaptation window, and compare the data sub-information entropy of each time subsequence to obtain the information entropy error; When the information entropy errors are all within the preset interval, obtain the data representation characteristics of the original ocean data of the current type, and based on the data representation characteristics, combine the average information entropy corresponding to the data sub-information entropy, use big data acquisition technology to obtain multiple compressed similar data, calculate the hash value corresponding to each compressed similar data, and based on the hash value, judge the Hamming distance between the hash value and its corresponding standard hash value; Perform screening based on the Hamming distance to obtain the minimum-loss compressed similar data, and obtain the compression parameter corresponding to the minimum-loss compressed similar data as the optimal compression parameter; Extract the data change characteristics of the original ocean data corresponding to each time subsequence respectively and compare them to obtain the public data sub-characteristics. Based on the public data sub-characteristics, data performance characteristics, and the time points corresponding to the mutation nodes, obtain the data characteristics of the current type of ocean data; Based on the data characteristics and the optimal compression parameters, establish the mapping relationship between the data characteristics and the compression parameters; When the information entropy errors are not within the preset intervals, based on the time points corresponding to the mutation points, determine the data mutation distribution characteristics within the preset acquisition period. Based on the data mutation distribution characteristics, combined with the performance characteristics of the current data, use big data acquisition technology to obtain multiple compressed similar data, and calculate the hash values corresponding to each compressed similar data. Based on the hash values, judge the Hamming distance between the hash values and their corresponding standard hash values; Perform screening based on the Hamming distance to obtain the compressed similar data with the minimum loss, and obtain the compression parameters corresponding to the compressed similar data with the minimum loss as the optimal compression parameters; Based on the data mutation distribution characteristics and data performance characteristics, obtain the data characteristics of the current type of ocean data; Based on the data characteristics and the optimal compression parameters, establish the mapping relationship between the data characteristics and the compression parameters.

[0025] In this embodiment, the selected byte segment refers to the data segment randomly selected by the element and the serial port in the actual sequence.

[0026] In this embodiment, the mutation node refers to the point where the data suddenly increases or decreases rapidly.

[0027] In this embodiment, the standard hash value refers to the preset ideal hash value corresponding to each compressed similar data. The compressed similar data refers to the data with the same data information entropy as that within the length corresponding to the preset window in the data performance characteristics (including the writing form of the data and the relative situation of the monitoring time, such as images, numbers, discrete, continuous, etc.) of the original ocean data.

[0028] The beneficial effects of the above technical solution are as follows: the present invention efficiently starts from the original ocean data collected by various sensors, generates a time series according to the time axis, and obtains data error by comparing adjacent values, and then generates a data floating sequence, preliminarily sorts out the change law of the data, and lays the foundation for subsequent in-depth analysis. For example, when analyzing seawater temperature data, abnormal temperature fluctuations can be quickly discovered by comparing adjacent values, providing clues for subsequent judgment of whether there is a mutation in the data. At the same time, the data information entropy of the data floating sequence is calculated, and different strategies are used for processing according to the size of the information entropy to further explore the potential characteristics of the data, such as the randomness and regularity of the data, so as to provide a basis for subsequent data classification and compression; in determining the compression parameters, different methods are adopted for different situations: when the data information entropy is less than the preset value, the data performance characteristics are obtained, and the compressed similar data is obtained by combining the big data collection technology, and the Hamming distance between the hash value and the standard hash value is used to screen out the compressed similar data with the minimum loss, so as to determine the optimal compression parameters, which can realize efficient data compression under the premise of ensuring the minimum loss of data information. For example, for some relatively stable marine data, such as long-term stable salinity data of a certain sea area, this method can find the most suitable compression method, reduce data storage space, and retain the key features of the data to the greatest extent. When the data information entropy is greater than or equal to the preset value, the mutation node of the data is further considered, and the time series is processed in segments. By calculating the data sub-information entropy and information entropy error of each time sub-sequence, the optimal compression parameters are determined according to different situations, and the data mutation situation is processed. It can adapt to complex and changeable marine data more accurately, ensuring that high-quality compression can be achieved when there is a mutation in the data, avoiding information loss or poor compression effect caused by data mutation. At the same time, based on the data information entropy and data performance characteristics, the data floating sequence is classified, the target data type is determined, and the public data features and data features are extracted, which helps to reasonably classify the complex and diverse marine data according to their inherent characteristics, and facilitate the subsequent targeted processing and analysis of different types of data. For example, the marine biological quantity data and marine hydrological data are distinguished, and the respective data change characteristics are extracted respectively, providing more targeted data support for marine ecological research and marine environmental monitoring. At the same time, by extracting and comparing the data features of different time subsequences, we can understand the changes of data in different time periods in more detail, especially when there are mutations in the data, we can accurately grasp the mutation distribution characteristics of the data, and provide strong data support for in-depth research on the dynamic changes of the marine environment. The present invention establishes an effective mapping relationship: by establishing a mapping relationship between data features and compression parameters, it provides great convenience for the subsequent storage, transmission and processing of marine data. In practical applications, when new raw marine data arrives, the system can quickly find the corresponding optimal compression parameters from the established mapping relationship according to its data features, realize rapid data compression, and improve data processing efficiency.Meanwhile, this mapping relationship also helps to accurately restore the original features of the data according to the compression parameters during data decompression, ensuring the accuracy and integrity of the data. In addition, for different types of data, different compression parameters are adopted according to their characteristics, which can achieve a better balance between storage space and data quality, meeting the requirements of the marine environment monitoring system for massive data storage and efficient processing; Establishing a mapping relationship between data features and compression parameters is beneficial to shortening the time interval from data acquisition to subsequent processing, improving the response speed of the entire system to changes in the marine environment, while ensuring that the data is restored to a state close to the original, ensuring the accuracy and integrity of the data input into the marine anomaly diagnosis lightweight model, thereby improving the reliability of the anomaly detection results and effectively avoiding misjudgment or missed judgment caused by data quality problems. Embodiment 4:

[0029] Based on Embodiment 1, the data processing module, such as Figure 3 shown, includes: A training set generation unit, configured to obtain a large amount of historical marine data as marine sample data, and perform anomaly data marking on the marine sample data to obtain an abnormal marine data set; A model training and deployment module, configured to train a marine anomaly diagnosis lightweight model based on the abnormal marine data set, and deploy the trained marine anomaly diagnosis lightweight model to the edge computing nodes of each sensor.

[0030] Beneficial effects of the above technical solution: The present invention realizes the training and deployment of the marine anomaly diagnosis lightweight model, realizes the integration of the sensor data compression technology and the marine anomaly diagnosis lightweight model, and provides a basis for the rapid processing and timely discovery of marine anomalies. Embodiment 5:

[0031] Based on Embodiment 1, the data processing module, such as Figure 3 shown, includes: A data decompression processing unit, configured to decompress the compressed marine monitoring data received by the edge node to obtain decompressed data, and compare the decompressed data with the previous received data of its corresponding type to determine whether the data has changed; If a change occurs, corresponding preprocessing is performed on the decompressed data based on the type of the decompressed data, and the decompressed data is stored and recorded Otherwise, the decompressed data is skipped, and the edge node is controlled to maintain the previous anomaly detection result, and only the time point is received for storage and recording; An anomaly monitoring unit, configured to input the preprocessed decompressed data into the marine anomaly diagnosis lightweight model to determine whether the current marine data is abnormal.

[0032] Beneficial effects of the above technical solution: After receiving and decompressing the compressed ocean monitoring data at the edge node, the present invention ensures the integrity of the data used for abnormal monitoring judgment. At the same time, the decompressed data is compared with the previous received data of its corresponding type respectively to determine whether the data has changed, providing a basis for the selection of subsequent incremental transmission and the judgment of abnormal monitoring. The preprocessed decompressed data is input into the lightweight ocean anomaly diagnosis model to determine whether the current ocean data is abnormal, realizing the judgment of the automatic monitoring result before the data is sent to the onshore data center and ensuring the timely discovery of ocean anomalies. Embodiment 6:

[0033] Based on Embodiment 1, the data transmission module, as Figure 4 shown, includes: A transmission processing unit, configured to generate a warning report based on abnormal ocean data, and generate an instruction for unchanged data for the unchanged data type; Generate a transmission data packet based on the warning report, the instruction for unchanged data, and the preprocessed changed ocean data; An onshore data update unit, configured to parse the transmission data packet after receiving it at the onshore data center, and determine the type of unchanged ocean data based on the instruction for unchanged data; Respectively obtain the reception time points corresponding to the unchanged ocean data and the changed ocean data, and update the unchanged ocean data and the changed ocean data respectively based on the reception time points; An onshore automatic warning unit, configured to generate a corresponding abnormal warning signal based on the abnormal ocean data and issue a warning when the onshore data center receives the warning report.

[0034] Beneficial effects of the above technical solution: The present invention adopts an incremental transmission method, which effectively reduces the data transmission volume, shortens the data transmission time, ensures the real-time and timeliness of the data received by the onshore data center, is conducive to improving the response speed to sudden ocean conditions, and parses the transmission data packet after the onshore data center receives it, determines the type of unchanged ocean data based on the instruction for unchanged data; respectively obtain the reception time points corresponding to the unchanged ocean data and the changed ocean data, and update the unchanged ocean data and the changed ocean data respectively based on the reception time points, ensuring that even in the case of only transmitting changed data, the unchanged data is recorded in a timely manner, providing comprehensive and complete data support for the subsequent analysis of the ocean activity rules by relevant departments. At the same time, when the onshore data center receives the warning report, a corresponding abnormal warning signal is generated based on the abnormal ocean data and a warning is issued, ensuring the timely discovery of ocean anomalies by relevant departments. Embodiment 7:

[0035] The present invention provides a lightweight marine data acquisition and processing method based on edge computing, including: Step 1: Adaptive real-time compression is performed on the original marine data collected by the buoy terminal to obtain compressed marine monitoring data and send it to the edge computing node; Step 2: The edge computing node decompresses and preprocesses the compressed marine monitoring data, and inputs the preprocessed marine monitoring data into a lightweight marine anomaly diagnosis model for real-time anomaly detection; Step 3: Adopting an incremental transmission method, the marine monitoring data and anomaly detection results processed by the edge node are sent to the onshore data center.

[0036] Beneficial effects of the above technical solution: The present invention performs adaptive real-time compression on the original marine data, which can greatly reduce the data volume and can be transmitted to the edge computing node through the communication link faster, effectively shortening the time interval from data acquisition to subsequent processing, improving the response speed of the entire system to changes in the marine environment, providing strong support for real-time monitoring and early warning, and adjusting the compression strategy according to the dynamic characteristics of the data to ensure that various marine data can effectively reduce the data volume during the compression process while maintaining the integrity and availability of the data, improving the reliability of data processing. Subsequently, by decompressing the compressed marine monitoring data at the edge computing node, it is ensured that the data is restored to a state close to the original, which can ensure the accuracy and integrity of the data input into the lightweight marine anomaly diagnosis model, thereby improving the reliability of the anomaly detection results, effectively avoiding misjudgment or missed judgment caused by data quality problems. At the same time, the lightweight processing of the model also reduces the storage requirements during the operation of the edge node, and also ensures the timely discovery of marine anomalies. By adopting the incremental transmission method, only the changed part of the data and the anomaly detection results processed by the edge node relative to the previous transmission are transmitted, which can avoid repeated transmission of a large amount of unchanged data, significantly reduce the data transmission volume, thereby greatly shortening the data transmission time, improving the data transmission efficiency, further enhancing the real-time performance of the system. At the same time, the smaller data transmission volume reduces the probability of data loss or error during transmission, and improves the stability and reliability of data transmission in the complex communication environment of the open sea where the signal is vulnerable to interference. Embodiment 8:

[0037] Based on Embodiment 7, a lightweight marine data acquisition and processing method based on edge computing, Step 1 includes: Obtain the original marine monitoring data collected by the buoy terminal; Extract the data change characteristics and data representation characteristics of the original marine monitoring data corresponding to the current buoy terminal; Based on the data change characteristics and data presentation characteristics, in combination with the mapping relationship between data characteristics and compression parameters, determine the compression ratio of the original ocean monitoring data, and compress the original ocean data according to the compression ratio to obtain compressed ocean monitoring data; Send the compressed ocean monitoring data to the edge node of the buoy terminal.

[0038] Beneficial effects of the above technical solution: The present invention directly collects the original ocean monitoring data from the buoy terminal, can timely capture the dynamic changes of the ocean environment, and by extracting the data change characteristics of the original ocean monitoring data, can deeply understand the internal laws and change trends of the data. For example, for seawater temperature data, the change rate of temperature over time, periodic fluctuation characteristics, etc. can be extracted; for wave height data, characteristics such as the change amplitude and frequency of wave crests and troughs can be obtained. At the same time, the extraction of data presentation characteristics, such as the distribution and dispersion degree of data, further enriches the understanding of the data. Different types of ocean monitoring data have different characteristics, and the data feature extraction unit can analyze the uniqueness of each type of data, enabling the entire adaptive data compression module to better adapt to various data situations, improving the flexibility and reliability of data processing, and determining the compression ratio of the original ocean monitoring data based on the data change characteristics and data presentation characteristics, in combination with the mapping relationship between data characteristics and compression parameters, realizing the optimization of data compression, and quickly sending the compressed ocean monitoring data to the edge node of the buoy terminal to ensure the timeliness of ocean anomaly data.

[0039] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these changes and modifications.

Claims

1. A lightweight marine data acquisition and processing system based on edge computing, characterized in that, Including: An adaptive data compression module, which is used to adaptively and real-time compress the original ocean data collected by the buoy terminal, and send the compressed ocean monitoring data to the edge computing node; A data processing module, which is used for the edge computing node to decompress and preprocess the compressed ocean monitoring data, and input the preprocessed ocean monitoring data into the ocean anomaly diagnosis lightweight model for real-time anomaly detection; A data transmission module, which is used to adopt the incremental transmission method to send the ocean monitoring data and anomaly detection results processed by the edge node to the onshore data center.

2. The lightweight marine data acquisition and processing system based on edge computing according to claim 1, wherein, The adaptive data compression module includes: A data acquisition unit, which is used to acquire the original ocean monitoring data collected by the buoy terminal; A data feature extraction unit, which is used to extract the data change features of the original ocean monitoring data corresponding to the current buoy terminal, and obtain the data change features and data performance features; A data compression unit, which is used to determine the compression ratio of the original ocean monitoring data based on the data change features and data performance features, and in combination with the mapping relationship between the data features and the compression parameters, and compress the original ocean data according to the compression ratio to obtain the compressed ocean monitoring data; An internal transmission unit, which is used to send the compressed ocean monitoring data to the edge node of the buoy terminal.

3. A lightweight marine data acquisition and processing system based on edge computing according to claim 2, characterized in that, The data compression unit further includes: A data analysis subunit, which is used to obtain the original ocean data collected by multiple sensors within a preset acquisition period based on the edge node, generate multiple time series according to the time axis, and respectively compare the adjacent values of each time series to obtain the data error between the adjacent data of the same kind of original ocean data; Obtain the data error corresponding to the same time series, generate a data floating sequence, randomly select the data floating sequence based on a preset byte window, and calculate the data information entropy of the selected byte segment based on the selected byte segment; A first mapping relationship determination subunit, which is used to, when the data information entropy is less than a preset value, obtain the data performance features of the original ocean data of the data floating sequence corresponding to the data information entropy, and based on the data performance features, combine the data information entropy and adopt the big data acquisition technology to obtain multiple compression similar data, and calculate the hash value corresponding to each compression similar data, and based on the hash value, judge the Hamming distance between the hash value and its corresponding standard hash value; Perform screening based on the Hamming distance to obtain the minimum loss compression similar data, and obtain the compression parameter corresponding to the minimum loss compression similar data as the optimal compression parameter; Classify the data floating sequence based on the data information entropy and data performance features, and determine the target data type of the original ocean data corresponding to the data floating sequence of the same category; Extract and compare the data change features of the target data types respectively to obtain the public data features, and based on the public data features and data performance features corresponding to the target data types, obtain the data features of the target data types; Based on the data features and the optimal compression parameters, establish a mapping relationship between the data features and the compression parameters.

4. A lightweight marine data acquisition and processing system based on edge computing according to claim 3, characterized in that, The data compression unit further includes: A second mapping relationship determination subunit, configured to, when the data information entropy is greater than or equal to a preset value, based on the data error between adjacent data in the data floating sequence, determine mutation nodes of the corresponding original ocean data based on the data error, and segment the original ocean data in the time series based on the mutation nodes to obtain time subsequences corresponding to multiple time periods, and respectively obtain each time subsequence: Combined with the data lengths of all time subsequences, adaptively adjust a preset window to obtain a full-sequence adaptation window, and based on the full-sequence adaptation window, calculate the data sub-information entropy corresponding to each time subsequence respectively, and compare the data sub-information entropy of each time subsequence to obtain an information entropy error; When the information entropy errors are all within a preset interval, obtain the data performance characteristics of the current type of original ocean data, and based on the data performance characteristics, combined with the average information entropy corresponding to the data sub-information entropy, use big data acquisition technology to obtain multiple compressed similar data, and calculate the hash value corresponding to each compressed similar data, and based on the hash value, judge the Hamming distance between the hash value and its corresponding standard hash value; Perform screening based on the Hamming distance to obtain the minimum-loss compressed similar data, and obtain the compression parameter corresponding to the minimum-loss compressed similar data as the optimal compression parameter; Extract and compare the data change characteristics of the original ocean data corresponding to each time subsequence respectively to obtain public data sub-characteristics, and based on the public data sub-characteristics, data performance characteristics, and the time points corresponding to the mutation nodes, obtain the data characteristics of the current type of ocean data; Based on the data characteristics and the optimal compression parameter, establish a mapping relationship between the data characteristics and the compression parameter.

5. The lightweight marine data acquisition and processing system based on edge computing according to claim 4, wherein The second mapping relationship determination subunit is further configured to: When the information entropy errors are not all within the preset interval, based on the time points corresponding to the mutation points, determine the data mutation distribution characteristics within a preset acquisition period, and based on the data mutation distribution characteristics, combined with the performance characteristics of the current data, use big data acquisition technology to obtain multiple compressed similar data, and calculate the hash value corresponding to each compressed similar data, and based on the hash value, judge the Hamming distance between the hash value and its corresponding standard hash value; Perform screening based on the Hamming distance to obtain the minimum-loss compressed similar data, and obtain the compression parameter corresponding to the minimum-loss compressed similar data as the optimal compression parameter; Based on the data mutation distribution characteristics and data performance characteristics, obtain the data characteristics of the current type of ocean data; Based on the data characteristics and the optimal compression parameter, establish a mapping relationship between the data characteristics and the compression parameter.

6. The lightweight marine data acquisition and processing system based on edge computing according to claim 1, characterized in that, The data processing module includes: A training set generation unit, configured to obtain a large amount of historical ocean data as ocean sample data, and mark abnormal data in the ocean sample data to obtain an abnormal ocean data set; A model training and deployment module, configured to train an ocean anomaly diagnosis lightweight model based on the abnormal ocean data set, and deploy the trained ocean anomaly diagnosis lightweight model to the edge computing nodes of each sensor.

7. A lightweight marine data acquisition and processing system based on edge computing according to claim 1, characterized in that, The data processing module includes: A data decompression processing unit, which is used to decompress the compressed ocean monitoring data received by the edge node to obtain decompressed data, and compare the decompressed data with the previous received data of its corresponding type to determine whether the data has changed; If a change occurs, corresponding preprocessing is performed on the decompressed data based on the type of the decompressed data, and the decompressed data is stored and recorded; Otherwise, the decompressed data is skipped, and the edge node is controlled to maintain the previous anomaly detection result, and only the receiving time point is stored and recorded; An anomaly monitoring unit, which is used to input the preprocessed decompressed data into the ocean anomaly diagnosis lightweight model to determine whether the current ocean data is abnormal.

8. A lightweight marine data acquisition and processing system based on edge computing according to claim 1, characterized in that, A data transmission module, including: A transmission processing unit, which is used to generate a warning report based on the abnormal ocean data, and generate an instruction for unchanged data types for the unchanged data; Generate a transmission data packet based on the warning report, the unchanged data instruction, and the preprocessed changed ocean data; A shore data update unit, which is used to parse the transmission data packet received by the shore data center, and determine the types of unchanged ocean data based on the unchanged data instruction; Respectively obtain the receiving time points corresponding to the unchanged ocean data and the changed ocean data, and update the unchanged ocean data and the changed ocean data respectively based on the receiving time points; A shore automatic warning unit, which is used to generate a corresponding anomaly warning signal based on the abnormal ocean data and give a warning when the shore data center receives the warning report.

9. A lightweight marine data acquisition and processing method based on edge computing, characterized in that, Including: Step 1: Perform adaptive real-time compression on the original ocean data collected by the buoy terminal to obtain compressed ocean monitoring data and send it to the edge computing node; Step 2: The edge computing node decompresses and preprocesses the compressed ocean monitoring data, and inputs the preprocessed ocean monitoring data into the ocean anomaly diagnosis lightweight model for real-time anomaly detection; Step 3: Adopt an incremental transmission method to send the ocean monitoring data and the anomaly detection result processed by the edge node to the shore data center.

10. A lightweight marine data acquisition and processing method based on edge computing according to claim 9, characterized in that, Step 1 includes: Obtain the original ocean monitoring data collected by the buoy terminal; Extract the data change characteristics of the original ocean monitoring data corresponding to the current buoy terminal to obtain the data change characteristics and the data performance characteristics; Based on the data change characteristics and the data performance characteristics, combined with the mapping relationship between the data characteristics and the compression parameters, determine the compression ratio of the original ocean monitoring data, and compress the original ocean data according to the compression ratio to obtain compressed ocean monitoring data; Send the compressed ocean monitoring data to the edge node of the buoy terminal.

Images