High-frequency data remote transmission method and system

Abstract

The invention discloses a high-frequency data remote transmission method and system, and the method comprises the steps: dynamically adjusting the collection frequency and mode according to environment conditions, and obtaining high-frequency data in real time; based on the data type and the transmission network state, a compression algorithm, a filtering mode and a coding strategy are automatically selected; the congestion condition of the transmission network is predicted through a built-in machine learning model, and a packaging strategy and transmission parameters are dynamically adjusted according to network conditions; in the data receiving process, the bit error rate in the transmission process is monitored in real time, abnormity in transmission is automatically detected, repair is carried out through an error correction algorithm, and if repair fails, the system automatically requests the data packet again; and the recovered high-frequency data is classified and stored according to the importance, real-time performance and storage cost of the data, and automatic storage strategy adjustment is realized in a remote database. According to the technical scheme, the high efficiency, reliability and safety of remote transmission of high-frequency data are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data transmission, and in particular to a high-frequency data remote transmission method and system. Background Art

[0002] High-frequency data remote transmission technology mainly involves wireless communication and data coding technology, which aims to solve the problems of signal attenuation and interference. With the popularization of the Internet of Things and smart devices, the demand for efficient and stable remote data transmission continues to increase. Common methods include the use of modulation and demodulation technology, optical fiber communication, and low-frequency signal forwarding. However, high-frequency signals are easily affected by environmental factors during transmission, such as rain, buildings, and other obstacles, resulting in signal attenuation and loss; they are easily interfered in dense environments, especially in shared spectrum situations, which may affect the reliability of data; and there are significant delay issues due to the large signal transmission distance. Summary of the invention

[0003] The main purpose of the present invention is to provide a high-frequency data remote transmission method and system, aiming to improve the efficiency, reliability and security of high-frequency data remote transmission.

[0004] To achieve the above object, the present invention proposes the following technical solution.

[0005] In a first aspect, the present invention provides a high-frequency data remote transmission method, comprising:

[0006] Step S1: dynamically adjusting the acquisition frequency and mode according to environmental conditions to acquire high-frequency data in real time, wherein the environmental conditions include but are not limited to signal strength, noise level, and sensor status;

[0007] Step S2: Automatically select compression algorithm, filtering method and encoding strategy based on data type and transmission network status to maximize transmission efficiency and data fidelity;

[0008] Step S3: The built-in machine learning model is used to predict the congestion of the transmission network, and the packaging strategy and transmission parameters are dynamically adjusted according to the network conditions to optimize the bandwidth utilization while ensuring data integrity.

[0009] Step S4: During the data reception process, the bit error rate of the transmission process is monitored in real time, and the abnormality in the transmission is automatically detected and repaired through the error correction algorithm. If the repair fails, the system automatically requests the data packet again;

[0010] Step S5: The recovered high-frequency data is classified and stored according to the importance, real-time nature and storage cost of the data, and automatic storage strategy adjustment is implemented in the remote database to optimize storage resources.

[0011] In a possible implementation, the step S1 includes:

[0012] Automatically detect and identify available high-frequency data sources in the environment through sensor networks, including but not limited to audio, video, vibration, temperature and humidity;

[0013] The built-in environmental sensors monitor the surrounding environmental parameters in real time and dynamically adjust the data collection parameters according to the evaluation results;

[0014] Adjust the frequency and resolution of data collection in real time according to changes in environmental conditions and data types to optimize collection performance and reduce redundant data;

[0015] During the data collection process, data validity detection is performed to identify and discard noise interference or invalid data to ensure that the collected data is accurate and reliable;

[0016] The collected valid data is preliminarily cached and packaged for the next step of preprocessing and transmission.

[0017] In a possible implementation, step S2 includes:

[0018] Automatically select a suitable compression mode according to the type of high-frequency data collected and the transmission network conditions. The compression algorithm includes but is not limited to differential compression, lossless compression or lossy compression;

[0019] Perform real-time filtering on the data before data compression to remove high-frequency noise and abnormal data during transmission. The filtering techniques used include but are not limited to low-pass filtering, high-pass filtering or adaptive filtering;

[0020] Encode data according to data transmission requirements and encrypt data according to security level requirements to ensure data security during transmission;

[0021] The compressed, filtered and encrypted data is packaged into multiple data packets, and the size and structure of the data packets are optimized according to the requirements of the transmission protocol to improve transmission efficiency.

[0022] In a possible implementation, step S3 includes:

[0023] The built-in machine learning model evaluates the latency, bandwidth, and reliability of different transmission paths in real time and selects the best transmission path based on the evaluation results.

[0024] During the transmission process, the size and sending frequency of the data packet are dynamically adjusted according to the current network status to ensure stable transmission performance;

[0025] When a data packet is detected to be lost or damaged during transmission, the data retransmission mechanism is automatically triggered, and key data packets are retransmitted first to ensure data integrity and consistency;

[0026] During the data transmission process, the transmission status is monitored in real time, and feedback information is generated and sent to the source end so that the source end can adjust the subsequent data packaging and transmission strategies based on the feedback.

[0027] In a possible implementation, step S4 includes:

[0028] Monitor the error rate and transmission delay in real time during data transmission through the transmission protocol and network monitoring module, and identify transmission failures that may cause data corruption or loss;

[0029] When an erroneous packet is detected, the processing unit identifies and marks the damaged packet through a built-in error detection algorithm;

[0030] Attempts to automatically repair damaged packets using predefined error correction algorithms;

[0031] If the error cannot be repaired automatically, the source end is automatically requested to retransmit the relevant data packets, and the critical data that failed to be repaired is transmitted first.

[0032] In a possible implementation, step S5 includes:

[0033] According to the importance, real-time nature and access frequency of data, data is divided into high-priority data and low-priority data, and stored in different types of storage media. High-priority data is stored in high-speed access storage devices, while low-priority data is stored in permanent storage media.

[0034] Automatically generate data backups and automatically restore critical data from backups when data loss or corruption is detected;

[0035] Regularly analyze storage resource usage and automatically optimize storage resource allocation based on actual data access frequency and storage requirements.

[0036] In a second aspect, the present invention provides a system, comprising:

[0037] Data acquisition module, used to obtain high-frequency data in real time;

[0038] The data preprocessing module is used to compress, filter and encode the data after collecting high-frequency data to reduce the data volume and optimize the transmission efficiency;

[0039] The data packaging and transmission module is used to package the pre-processed data according to the preset transmission protocol and transmit the data remotely to the receiving end using the high-frequency wireless communication network;

[0040] A data receiving and decoding module is used to decode and decompress the remotely received data through a decoding function to restore the original high-frequency data;

[0041] The data storage module is used to store the decoded high-frequency data in a remote database for subsequent analysis and processing.

[0042] The technical solution of the present invention adopts data source identification, environmental condition evaluation and dynamic acquisition frequency adjustment, so that the system can automatically adjust the acquisition strategy according to the needs and environmental conditions of different data sources; through multiple preprocessing steps such as intelligent compression algorithm, real-time filtering, encoding and encryption, the system can automatically select the optimal processing strategy according to the data type and network conditions; through network condition prediction based on machine learning and dynamic adjustment of data packet size, the system can adapt to network fluctuations in real time, maximize bandwidth utilization and reduce transmission delay; real-time monitoring of errors in data transmission and automatic repair through built-in algorithms, reduce transmission failures caused by data packet loss or errors; the system can intelligently classify and store data according to the importance, real-time and access frequency of the data. In this way, high efficiency of data transmission, reliability of system operation, low power consumption, intelligence of data management and improvement of overall security are achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0044] Figure 1 The figure is a flow chart of an embodiment of a high-frequency data remote transmission method of the present invention.

[0045] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0047] Embodiment 1

[0048] The present invention proposes a method for high-frequency data remote transmission, which is executed by a processing unit, which integrates hardware and / or software systems with various functions and is responsible for executing each step in the entire high-frequency data remote transmission method. Exemplarily, a multifunctional system integrating hardware (such as CPU, GPU, DSP, sensor interface, communication module, etc.) and software (such as data processing algorithms, AI models, power consumption management and error detection tools, etc.). It is responsible for the entire process from data acquisition to transmission, reception, decoding and storage. According to actual needs, the processing unit can be an embedded system, a server, a cloud computing unit, or an integrated module of a distributed system. Specifically comprising the following steps:

[0049] Step S1: dynamically adjust the acquisition frequency and mode according to environmental conditions to acquire high-frequency data in real time. The environmental conditions include but are not limited to signal strength, noise level, and sensor status.

[0050] In this step, the intelligent data acquisition system is applied to the environmental monitoring scenario. The system uses multiple sensors to collect high-frequency data in the environment and dynamically adjusts the acquisition frequency based on the type of data source, environmental conditions, etc. The system includes multiple types of sensors, covering data sources such as temperature, humidity, vibration, sound, and light intensity. Each sensor is connected to the main processing unit through an I2C or SPI interface. The main processing unit is a hardware platform that integrates a microcontroller (MCU) and a signal processing module, and is equipped with a built-in environmental assessment module. First, the sensor automatically detects the type of surrounding environmental data source. For example, when the system detects a significant temperature change, it automatically activates the temperature sensor and starts collecting temperature data. Similarly, when a vibration signal is detected, the vibration sensor will be mobilized first. During the data acquisition process, the main processing unit dynamically adjusts the acquisition frequency by monitoring environmental parameters (such as temperature fluctuations and signal strength changes) in real time. For example, when the sensor detects frequent temperature fluctuations, the acquisition frequency will increase to capture more details of the changes; when the environment is stable, the acquisition frequency will decrease to reduce unnecessary data load. In order to avoid collecting invalid data, the processing unit will perform validity checks on the data. For example, the processing unit will detect whether the collected temperature data is within a reasonable range. If an abnormal value is detected (such as a temperature far exceeding the set threshold), it will be identified as noise and discarded. The processing unit will initially cache the collected valid data and package the data into small data packets according to the transmission protocol, ready for transmission to the remote server. When caching, the data is stored in a ring buffer to improve the efficiency of the cache.

[0051] Step S2: Based on the data type and transmission network status, automatically select the compression algorithm, filtering method and encoding strategy to maximize transmission efficiency and data fidelity.

[0052] In this step, in the preprocessing stage, the main processing unit selects different compression algorithms according to the data type. For example, for high-frequency audio data, the system selects a lossy compression algorithm, such as MP3 encoding, to reduce the amount of data and ensure sufficient sound quality; for temperature and humidity data, a lossless compression algorithm is selected to ensure the integrity of the data. For the data collected by the vibration sensor, the system performs filtering in real time to remove high-frequency noise and environmental interference. The system uses an adaptive filter, which can dynamically adjust the filtering parameters according to the characteristics of the signal to improve the clarity of the signal. Before data transmission, the processing unit encodes the data. For highly sensitive data (such as user privacy data), the system uses the AES-256 encryption algorithm for encryption to ensure the security of the data during transmission. After compression, encoding and encryption, the processing unit packages the data into multiple small data packets. The system dynamically adjusts the size of the data packet according to the current network conditions (such as network delay, bandwidth utilization) to optimize the transmission efficiency. For example, when the bandwidth is wide, the system will increase the size of a single data packet to reduce the overhead of the data packet header.

[0053] Step S3: The built-in machine learning model is used to predict the congestion of the transmission network, and the packaging strategy and transmission parameters are dynamically adjusted according to the network conditions to optimize the bandwidth utilization while ensuring data integrity.

[0054] In this step, the system analyzes the current network conditions, including available bandwidth, latency, packet loss rate, etc., through the built-in machine learning model. Based on these parameters, the system automatically selects the best transmission path. For example, when there are multiple network options (such as 5G, Wi-Fi), the system will give priority to the path with the smallest latency and the largest bandwidth. When the system detects changes in network conditions (such as weakened Wi-Fi signals and decreased bandwidth), the processing unit will automatically reduce the size of the data packet and reduce the frequency of data transmission to avoid network congestion. For example, when the network bandwidth becomes narrower, the system will choose to transmit in smaller data packets to reduce the load of one transmission and avoid network congestion. If packet loss or transmission errors are detected, the system will trigger the retransmission mechanism. The system prioritizes retransmission of key data packets, such as key frames in video streams, to ensure the integrity of important data. The retransmission mechanism uses adaptive FEC (forward error correction coding) technology to automatically repair some data packets at a certain error rate and reduce the number of retransmissions. The system monitors the transmission status in real time and sends feedback information to the source. The source can dynamically adjust the transmission strategy based on these feedbacks, such as increasing data packet redundancy or changing the transmission path to adapt to changes in network conditions.

[0055] Step S4: During the data reception process, the bit error rate of the transmission process is monitored in real time, and anomalies in the transmission are automatically detected and repaired through an error correction algorithm. If the repair fails, the system automatically requests the data packet again.

[0056] In this step, the system monitors the errors in the transmission process in real time through the built-in fault detection module. If an increase in transmission delay or packet loss rate is detected, the system will record these error events and notify the system administrator. The system automatically detects and repairs damaged data packets in transmission through the forward error correction (FEC) mechanism. For example, the system can repair slightly damaged data packets through redundant data without retransmission. If the error data packet cannot be repaired automatically, the system will prioritize the retransmission of key data packets and skip unnecessary auxiliary data to reduce retransmission time and improve repair efficiency.

[0057] Step S5: The recovered high-frequency data is classified and stored according to the importance, real-time nature and storage cost of the data, and automatic storage strategy adjustment is implemented in the remote database to optimize storage resources.

[0058] In this step, the system divides the collected data into high-priority data and low-priority data based on the importance, real-time nature, and access frequency of the data. High-priority data (such as real-time monitoring data) is stored in solid-state drives (SSDs) for quick access; low-priority data is stored in larger disks or cloud storage. The system can dynamically select storage media based on the usage of storage resources. For example, when the local storage is about to be full, the system can automatically migrate some low-priority data to cloud storage. The system regularly backs up critical data, and when storage damage or data loss is detected, the system can automatically restore relevant data from the backup to ensure data security.

[0059] In summary, the specific implementation methods of intelligent data acquisition, adaptive preprocessing, intelligent data packaging and transmission, remote automatic fault detection and repair, and dynamic data storage optimization are demonstrated. Each step combines the technical means of intelligence, dynamic adjustment and adaptive optimization, enabling the system to efficiently and reliably transmit high-frequency data remotely in a complex and changing environment. It not only improves the efficiency and reliability of data transmission, but also greatly enhances the adaptability and intelligence of the system, making it have a wide range of application potential in application scenarios such as the Internet of Things, remote monitoring, and intelligent environmental control.

[0060] In a possible implementation, the step S1 includes:

[0061] S101. Automatically detect and identify available high-frequency data sources in the environment through a sensor network, including but not limited to audio, video, vibration, temperature and humidity.

[0062] In this step, the system forms a sensor network through the deployment of various sensors (such as audio, video, vibration, temperature and humidity, etc.) to automatically detect and identify available data sources in the surrounding environment. When a data source is activated (such as sound, vibration, or temperature and humidity changes in the environment), the system first determines whether the data source has high-frequency characteristics through parameters such as the sensor's signal strength, frequency, and duration. For example, continuous high-frequency vibration signals detected by vibration sensors will be processed first, while temperature and humidity sensors will be activated when the environment changes significantly. In this way, the system can respond only to important and frequently changing data sources without consuming a lot of energy, reducing the processing burden of irrelevant data.

[0063] S102, monitor the surrounding environmental parameters in real time through the built-in environmental sensors, and dynamically adjust the data collection parameters according to the evaluation results.

[0064] In this step, the system's built-in environmental sensors continuously monitor key parameters of the surrounding environment (such as temperature, humidity, air pressure, light intensity, etc.). Based on the real-time evaluation results of environmental changes, the system automatically adjusts the relevant parameters of data acquisition, including acquisition frequency, resolution, and sampling duration. For example, when the system detects a sharp change in ambient temperature, it automatically increases the sampling frequency of the temperature and humidity sensors to capture these changes more frequently. When the environment is stable, the sampling frequency will be reduced to save system resources. In addition, when the ambient light intensity changes greatly, the video acquisition resolution will also be automatically adjusted to ensure that the collected video data can maintain clarity without generating too much redundancy.

[0065] S103. According to changes in environmental conditions and data types, adjust the frequency and resolution of data collection in real time to optimize collection performance and reduce redundant data.

[0066] In this step, for audio data sources, when the system detects that the background noise is large, the sampling resolution will be automatically adjusted to increase noise filtering or reduce audio quality to adapt to the background environment; in a quieter environment, the sampling rate will be increased to capture more details. The acquisition frequency of the vibration data source will also be adaptively adjusted according to environmental conditions. For example, when the system detects vibration from the operation of mechanical equipment, it will increase the sampling frequency to monitor the equipment status in real time; after the equipment stops, the frequency will automatically decrease to reduce the collection of irrelevant data.

[0067] S104. During the data collection process, data validity detection is performed simultaneously to identify and discard noise interference or invalid data to ensure that the collected data is accurate and reliable.

[0068] In this step, for example, the system will determine whether the collected data is within the expected range by setting a reasonable signal threshold. When the collected data exceeds the set range, the system will mark it as invalid data and discard it. For audio and vibration data, the system uses a noise filtering algorithm to automatically detect and remove background noise or signal fluctuations to ensure that the final data is clean and has analytical value.

[0069] S105: Preliminary caching of the collected valid data is performed, and the data is packaged for the next step of preprocessing and transmission.

[0070] In this step, after the data is cached, the system will group and package it for the next step of preprocessing and remote transmission. The packaging process is optimized according to the data type, transmission priority and network conditions. For example, important data will be packaged and transmitted first, while low-priority data can be processed later.

[0071] In a possible implementation, step S2 includes:

[0072] S201. Automatically select a suitable compression mode according to the type of high-frequency data collected and the transmission network conditions. The compression algorithm includes but is not limited to differential compression, lossless compression or lossy compression.

[0073] In this step, after collecting high-frequency data, the system automatically selects a suitable compression mode according to the data type and transmission network conditions (such as bandwidth, delay, transmission rate, etc.). Compression algorithms include but are not limited to differential compression, lossless compression, and lossy compression.

[0074] S202: Perform real-time filtering on the data before data compression to remove high-frequency noise and abnormal data during transmission. The filtering technology used includes but is not limited to low-pass filtering, high-pass filtering or adaptive filtering.

[0075] In this step, before data compression, the system performs real-time filtering on the data to remove high-frequency noise and abnormal data in the data to ensure that the transmitted data is accurate and reliable. Filtering techniques include but are not limited to low-pass filtering, high-pass filtering and adaptive filtering.

[0076] S203: Encode the data according to data transmission requirements and encrypt the data according to security level requirements to ensure data security during transmission.

[0077] In this step, the system uses appropriate encoding methods, such as Huffman encoding, Run-length encoding, etc., to further compress data redundancy and reduce the size of the data packet. The system uses different encryption algorithms for encryption processing according to security requirements. For sensitive data, such as user identity information or private data, the system uses high-level encryption algorithms such as AES-256 to ensure the security of data during transmission. For general data, a lighter encryption algorithm can be used to increase processing speed.

[0078] S204: Pack the compressed, filtered and encrypted data into multiple data packets, and optimize the size and structure of the data packets according to the requirements of the transmission protocol to improve transmission efficiency.

[0079] In this step, the system dynamically adjusts the size of the data packet according to the current network conditions. For example, when the bandwidth is sufficient, the data packet size can be increased to reduce the header overhead and improve transmission efficiency; when the network conditions are poor, the system will reduce the size of the data packet to reduce the probability of retransmission. The system optimizes the structure of the data packet by analyzing the transmission protocol to make it more in line with the requirements of the transmission protocol. For example, the data packet can be divided into several small fragments to facilitate segmented transmission and error retransmission in an unreliable network environment.

[0080] In summary, through this implementation method, the system can intelligently select the compression mode and perform effective filtering before data transmission to remove interference and noise and ensure the quality of the transmitted data. At the same time, the encoding and encryption steps enhance the security of the data, making it difficult for the data to be stolen or tampered with during transmission. Finally, the mechanism of data packet packaging and transmission optimization improves the efficiency of data transmission, allowing the system to flexibly adjust the transmission strategy under different network conditions and adapt to various complex network environments. This design has significant advantages in high-frequency data transmission applications and is suitable for scenarios such as remote monitoring, the Internet of Things, and big data analysis.

[0081] In a possible implementation, step S3 includes:

[0082] S301. Use the built-in machine learning model to evaluate the delay, bandwidth, and reliability of different transmission paths in real time, and select the best transmission path based on the evaluation results.

[0083] In this step, the system has a built-in machine learning model that can evaluate the network parameters of different transmission paths in real time, including latency, bandwidth, and reliability. By learning from historical transmission data, the machine learning model can predict the future performance of each path and automatically select the best transmission path based on the evaluation results. The model predicts the delay changes in future time periods by analyzing the delay fluctuations of different paths. If a path is expected to have a large delay in the short term, the system will avoid selecting that path. The system estimates the available bandwidth of each path based on historical data and selects the path with the best current bandwidth to ensure a stable transmission rate. Based on past transmission records, the machine learning model predicts the packet loss rate, error rate, etc. of the transmission path to ensure that a data transmission channel with higher reliability is selected.

[0084] S302. During the transmission process, the size and transmission frequency of the data packet are dynamically adjusted according to the current network status to ensure stable transmission performance.

[0085] In this step, when the network condition is good, the system automatically increases the size of the data packet to improve transmission efficiency and reduce network load. When the network is congested or the packet loss rate increases, the system will reduce the size of the data packet to reduce the retransmission cost caused by the loss of large packets. The system adjusts the frequency of data transmission according to the dynamic change of bandwidth. If the bandwidth reduction is detected, the system will reduce the frequency of data packet transmission to ensure that the network is not overloaded; when the bandwidth is sufficient, the system will increase the frequency of transmission to speed up data transmission.

[0086] S303. When it is detected that a data packet is lost or damaged during transmission, a data retransmission mechanism is automatically triggered, and key data packets are retransmitted first to ensure data integrity and consistency.

[0087] In this step, the system will prioritize the key data packets that need to be retransmitted (such as important status information, commands, etc.) according to the data priority strategy to ensure that the core data transmitted is prioritized. The system will select the best retransmission path and time based on the packet loss path and network conditions to reduce the number of retransmissions and resource consumption.

[0088] S304: During the data transmission process, the transmission status is monitored in real time, and feedback information is generated and sent to the source end, so that the source end can adjust subsequent data packaging and transmission strategies according to the feedback.

[0089] In this step, the system generates a detailed transmission status report by monitoring key indicators such as transmission delay, bandwidth fluctuation, and packet loss rate. The system feeds back the monitoring results to the source in real time, and the source dynamically adjusts the packet size, transmission frequency, and path selection based on the feedback information to optimize subsequent transmission strategies. For example, in the case of a high packet loss rate, the source can reduce the transmission of non-critical data or use stronger error correction coding.

[0090] In a possible implementation, step S4 includes:

[0091] S401. Monitor the error rate and transmission delay in the data transmission process in real time through the transmission protocol and network monitoring module, and identify transmission failures that may cause data damage or loss.

[0092] In this step, the system collects the proportion of error packets in transmission in real time. If the error rate exceeds the set threshold, the system automatically marks the transmission channel as unstable and evaluates alternative paths or performs error correction on the packets. The system monitors the latency of the transmission path, especially in high-latency environments, where latency changes may cause packet loss or out-of-order transmission. When the latency is abnormal, the system automatically switches the path or adjusts the packet transmission strategy to ensure that the data arrives intact.

[0093] S402: When an erroneous data packet is detected, the processing unit identifies and marks the damaged data packet through a built-in error detection algorithm.

[0094] In this step, the system verifies the integrity of the received data packets through algorithms such as CRC. When the error bits in the data packet are detected to exceed the error tolerance range, the system will automatically mark the data packet as damaged. All packets detected to be damaged will be marked so that the system can identify and process them. These marked data packets will enter the priority queue and wait for further processing and repair.

[0095] S403: Use a predefined error correction algorithm to try to automatically repair the damaged data packet.

[0096] In this step, the system can use forward error correction (FEC) algorithms, such as Hamming code, Reed-Solomon code, etc., to repair partially damaged data. These algorithms allow the system to correct a small number of bit errors by appending redundant information, so that the receiving end does not need to request retransmission from the source end. When the system detects an error in a data packet, it will try to repair the packet through redundant correction codes. If the error bit is within the repair range, the system will successfully recover the packet and mark its repair status.

[0097] S404: If the error cannot be repaired automatically, automatically request the source end to retransmit the relevant data packets, and give priority to transmitting the key data that failed to be repaired.

[0098] In this step, the system automatically sends a retransmission request to the source end based on the marking status of the data packet, including the ID, type, and other metadata of the damaged packet. After receiving the request, the source end resends the corresponding data packet according to the priority. During the retransmission process, the system places critical data packets (such as important instructions or sensor data) in a priority queue to ensure that they can be retransmitted as soon as possible to ensure the transmission integrity of the core data.

[0099] In a possible implementation, step S5 includes:

[0100] S501. According to the importance, real-time nature and access frequency of the data, the data is divided into high-priority data and low-priority data, and stored in different types of storage media respectively. The high-priority data is stored in a high-speed access storage device, and the low-priority data is stored in a permanent storage medium.

[0101] In this step, high-priority data refers to real-time data that has a direct impact on system stability and operation, such as critical sensor data, real-time control instructions, etc. These data require high-speed access and are therefore stored in high-speed access storage devices, such as SSDs or memory (RAM). Low-priority data refers to data that is infrequently accessed or has low timeliness requirements, such as historical records, log files, etc. The system stores this data in low-cost, long-term storage media, such as mechanical hard disks (HDDs) or cloud storage. The system automatically prioritizes data according to predefined rules by analyzing the real-time and importance of the data. For example, the current status data of a sensor would be defined as high priority, while historical temperature records would be defined as low priority.

[0102] S502: Automatically generate data backup, and automatically restore key data from the backup when data loss or damage is detected.

[0103] In this step, the system automatically generates regular backups based on the importance and priority of the data. High-priority data will be backed up more frequently, and a multi-copy backup strategy will be adopted to ensure high data availability; low-priority data can adopt a periodic or incremental backup strategy. When the system detects a storage medium failure, data loss or damage, the processing unit will automatically extract the corresponding data from the backup for recovery. For high-priority data, the system gives priority to fast recovery to ensure that the real-time and continuous operation of the system are not affected. To enhance data security, the system will store backup data in different physical locations or remote clouds to prevent the loss of all data due to the failure of a single storage device.

[0104] S503: Regularly analyze the usage of storage resources, and automatically optimize the allocation of storage resources according to the actual access frequency and storage demand of data.

[0105] In this step, the system monitors the access frequency of each data set, regularly compiles access logs, and identifies which data is frequently read and which data has not been accessed for a long time. Based on the analysis results of the access frequency, the system automatically migrates data with low access frequency from high-speed storage devices to long-term storage devices, freeing up high-speed storage resources to store more high-priority data. At the same time, for data with increased access frequency, the system will migrate it to faster storage devices. The system dynamically adjusts the resource allocation of various storage devices based on the storage requirements of the data, such as data size, access frequency, life cycle, etc. The storage resource optimization strategy ensures the best use efficiency of storage media through adaptive algorithms, avoiding excessive use of high-performance devices or inefficient use of low-cost devices.

[0106] In a second aspect, the present invention provides a system, comprising:

[0107] Data acquisition module, used to obtain high-frequency data in real time;

[0108] The data preprocessing module is used to compress, filter and encode the data after collecting high-frequency data to reduce the data volume and optimize the transmission efficiency;

[0109] The data packaging and transmission module is used to package the pre-processed data according to the preset transmission protocol and transmit the data remotely to the receiving end using the high-frequency wireless communication network;

[0110] A data receiving and decoding module is used to decode and decompress the remotely received data through a decoding function to restore the original high-frequency data;

[0111] The data storage module is used to store the decoded high-frequency data in a remote database for subsequent analysis and processing.

[0112] Those skilled in the art should understand that the division of the various modules in the embodiment is only a division of logical functions, and can be fully or partially integrated into one or more actual carriers in actual application, and these modules can be implemented entirely in the form of software called by the processing unit, or entirely in the form of hardware, or in the form of a combination of software and hardware. It should be noted that the modules in the system in this embodiment correspond one-to-one to the steps in the high-frequency data remote transmission method in the aforementioned embodiment. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned high-frequency data remote transmission method, which will not be repeated here.

[0113] The above are only preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A high-frequency data remote transmission method, characterized in that: include: Step S1: dynamically adjusting the acquisition frequency and mode according to environmental conditions to acquire high-frequency data in real time, wherein the environmental conditions include but are not limited to signal strength, noise level, and sensor status; Step S2: Automatically select compression algorithm, filtering method and encoding strategy based on data type and transmission network status to maximize transmission efficiency and data fidelity; Step S3: The built-in machine learning model is used to predict the congestion of the transmission network, and the packaging strategy and transmission parameters are dynamically adjusted according to the network conditions to optimize the bandwidth utilization while ensuring data integrity. Step S4: During the data reception process, the bit error rate of the transmission process is monitored in real time, and the abnormality in the transmission is automatically detected and repaired through the error correction algorithm. If the repair fails, the system automatically requests the data packet again; Step S5: The recovered high-frequency data is classified and stored according to the importance, real-time nature and storage cost of the data, and automatic storage strategy adjustment is implemented in the remote database to optimize storage resources.

2. The high-frequency data remote transmission method according to claim 1, characterized in that: The step S1 comprises: Automatically detect and identify available high-frequency data sources in the environment through sensor networks, including but not limited to audio, video, vibration, temperature and humidity; The built-in environmental sensors monitor the surrounding environmental parameters in real time and dynamically adjust the data collection parameters according to the evaluation results; Adjust the frequency and resolution of data collection in real time according to changes in environmental conditions and data types to optimize collection performance and reduce redundant data; During the data collection process, data validity detection is performed to identify and discard noise interference or invalid data to ensure that the collected data is accurate and reliable; The collected valid data is preliminarily cached and packaged for the next step of preprocessing and transmission.

3. The high-frequency data remote transmission method according to claim 1, characterized in that: The step S2 comprises: Automatically select a suitable compression mode according to the type of high-frequency data collected and the transmission network conditions. The compression algorithm includes but is not limited to differential compression, lossless compression or lossy compression; Perform real-time filtering on the data before data compression to remove high-frequency noise and abnormal data during transmission. The filtering techniques used include but are not limited to low-pass filtering, high-pass filtering or adaptive filtering; Encode data according to data transmission requirements and encrypt data according to security level requirements to ensure data security during transmission; The compressed, filtered and encrypted data is packaged into multiple data packets, and the size and structure of the data packets are optimized according to the requirements of the transmission protocol to improve transmission efficiency.

4. The high-frequency data remote transmission method according to claim 1, characterized in that: The step S3 comprises: The built-in machine learning model evaluates the latency, bandwidth, and reliability of different transmission paths in real time and selects the best transmission path based on the evaluation results. During the transmission process, the size and sending frequency of the data packet are dynamically adjusted according to the current network status to ensure stable transmission performance; When a data packet is detected to be lost or damaged during transmission, the data retransmission mechanism is automatically triggered, and key data packets are retransmitted first to ensure data integrity and consistency; During the data transmission process, the transmission status is monitored in real time, and feedback information is generated and sent to the source end so that the source end can adjust the subsequent data packaging and transmission strategies based on the feedback.

5. The high-frequency data remote transmission method according to claim 1, characterized in that: The step S4 comprises: Monitor the error rate and transmission delay in the data transmission process in real time through the transmission protocol and network monitoring module, and identify transmission failures that may cause data corruption or loss; When an erroneous packet is detected, the processing unit identifies and marks the damaged packet through a built-in error detection algorithm; Attempts to automatically repair damaged packets using predefined error correction algorithms; If the error cannot be repaired automatically, the source end is automatically requested to retransmit the relevant data packets, and the critical data that failed to be repaired is transmitted first.

6. The high-frequency data remote transmission method according to claim 1, characterized in that: The step S5 comprises: According to the importance, real-time nature and access frequency of data, data is divided into high-priority data and low-priority data, and stored in different types of storage media. High-priority data is stored in high-speed access storage devices, while low-priority data is stored in permanent storage media. Automatically generate data backups and automatically restore critical data from backups when data loss or corruption is detected; Regularly analyze storage resource usage and automatically optimize storage resource allocation based on actual data access frequency and storage requirements.

7. A system, characterized in that: include: Data acquisition module, used to obtain high-frequency data in real time; The data preprocessing module is used to compress, filter and encode the data after collecting high-frequency data to reduce the data volume and optimize the transmission efficiency; The data packaging and transmission module is used to package the pre-processed data according to the preset transmission protocol and transmit the data remotely to the receiving end using the high-frequency wireless communication network; A data receiving and decoding module is used to decode and decompress the remotely received data through a decoding function to restore the original high-frequency data; The data storage module is used to store the decoded high-frequency data in a remote database for subsequent analysis and processing.

Images