Data acquisition method, system and device and mobile terminal

Abstract

The invention discloses a data acquisition method, system and device and a mobile terminal. The method comprises the following steps: acquiring original data; determining a working mode and compressed data according to the original data; preprocessing the compressed data according to priority and real-time requirements to obtain data features; analyzing according to the data characteristics to obtain a data transmission scheme; performing real-time monitoring according to the transmission scheme to obtain energy consumption and resource use conditions of the equipment; performing analysis and prediction according to the energy consumption, the resource use condition and the working mode to obtain an energy consumption prediction result; according to an energy consumption prediction result, dynamic generation is carried out to obtain an optimal working parameter and a data transmission strategy, the operation state of equipment is adjusted in real time through the data transmission strategy and the optimal working parameter, and the overall energy consumption of the system is minimized while the data quality and the real-time performance are ensured. According to the method, the problem that in the prior art, energy consumption is too large under various complex and changeable data collection scenes can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of data acquisition, and particularly to a data acquisition method, system, device and mobile terminal. Background Art

[0002] Currently, in the process of data acquisition by a mobile terminal, power consumption management of the device is a key technical issue. To achieve the goal of energy conservation, it is necessary to reasonably control the power consumption of the device on the premise of ensuring the quality of data acquisition. However, the requirements of data acquisition tasks are diverse, and different tasks have different requirements for data acquisition frequency, data volume size, and real-time performance. How to dynamically adjust the working mode and parameter configuration of the device according to the characteristics of the acquisition task to achieve the purpose of power consumption optimization is a difficult problem to be solved urgently.

[0003] In an existing technology, the collected original data is often large in volume, and a large amount of energy and bandwidth resources are required for acquisition and transmission.

[0004] In the existing technology, there is a problem of excessive energy consumption in various complex and changeable data acquisition scenarios. Summary of the Invention

[0005] The present invention provides a data acquisition method, system, device and mobile terminal to solve the problem of excessive energy consumption in various complex and changeable data acquisition scenarios in the existing technology.

[0006] In a first aspect, to solve the above technical problem, the present invention provides a data acquisition method, including: Obtaining the original data of the current data acquisition task; wherein, the original data includes acquisition frequency, data volume size, real-time requirement and priority; Determining the working mode of the device according to the original data; Compressing the original data according to the original data to obtain compressed data; Preprocessing the compressed data according to the priority and the real-time requirement to obtain data features; Analyzing the data features to obtain the optimal data transmission scheme; Performing real-time monitoring during data transmission according to the optimal transmission scheme to obtain the energy consumption and resource usage of the device; Analyzing and predicting according to the energy consumption, the resource usage and the working mode to obtain an energy consumption prediction result; Dynamically generating the optimal working parameters and data transmission strategy of the device according to the energy consumption prediction result, and adjusting the running state of the device in real time by the data transmission strategy and the optimal working parameters.

[0007] In an alternative embodiment, the determination based on the original data to obtain the working mode of the device includes: Analyze the original data to obtain a high-performance mode with a higher priority; Allocate computing resources and network resources according to the high-performance mode to obtain an allocation ratio; Monitor the execution of real-time monitoring data collection tasks according to the allocation ratio to obtain an execution ratio; Dynamically adjust the collection frequency and resource allocation according to the execution ratio to obtain the working mode of the device.

[0008] In an alternative embodiment, the preprocessing of the compressed data according to the priority and the real-time requirement to obtain data features includes: Calculate according to the priority and the real-time requirement to obtain a dynamic index of the data; Determine according to the dynamic index to obtain the parameter value range of the preprocessing algorithm; Calculate according to the parameter value range and a preset parameter threshold to obtain an adjusted parameter value; Calculate according to the priority and the real-time requirement to obtain a quality value and an efficiency value of the data; Calculate according to the quality value and the efficiency value to obtain a balance value; Perform preprocessing according to the balance value and the adjusted parameter to obtain data features.

[0009] In an alternative embodiment, the calculation according to the priority and the real-time requirement to obtain the dynamic index of the data includes: Calculate the dynamic index in the following manner: wherein, is the dynamic index, is the priority weight coefficient, is the real-time requirement weight coefficient, is an extremely small positive number, is the priority of the data, is the real-time requirement of the data.

[0010] In an alternative embodiment, the calculation according to the parameter value range and the preset parameter threshold to obtain the adjusted parameter value includes: Calculate the adjusted parameter value in the following manner: wherein, is the adjusted parameter value, is the parameter value range, is the adjustment coefficient, is the data volume size, is the preset threshold.

[0011] In an alternative embodiment, the calculating, according to the priority and the real-time requirement, to obtain the quality value and the efficiency value of the data includes: Calculating the quality value in the following manner: Wherein, is the quality value, δ1 is the priority weight coefficient, is the real-time requirement weight coefficient, is the priority of the data, is the real-time requirement of the data, is an extremely small positive number.

[0012] In an alternative embodiment, the calculating, according to the priority and the real-time requirement, to obtain the quality value and the efficiency value of the data includes: Calculating the efficiency value in the following manner: Wherein, is the quality value, δ1 is the priority weight coefficient, is the real-time requirement weight coefficient, is the priority of the data, is the real-time requirement of the data.

[0013] In an alternative embodiment, the calculating, according to the quality value and the efficiency value, to obtain the balance value includes: Calculating the balance value in the following manner: Wherein, is the balance value, is the weight coefficient, is the quality value, is the quality value.

[0014] In an alternative embodiment, the analyzing, according to the data characteristics, to obtain the optimal transmission scheme of the data includes: Determining according to the data characteristics to obtain the minimum bandwidth and the maximum delay threshold of the transmission link; Calculating according to the minimum bandwidth and the maximum delay threshold to obtain the comprehensive score of the transmission link; Comparing the comprehensive score with the preset score threshold to obtain the transmission protocol; Dynamically adjust according to the transmission protocol to obtain the optimal data transmission scheme.

[0015] In a second aspect, the present invention provides a data acquisition device, including: A data acquisition module, configured to acquire the original data of the current data acquisition task; wherein, the original data includes the acquisition frequency, the data volume size, the real-time requirement, and the priority; A mode confirmation module, configured to determine according to the original data to obtain the working mode of the device; A data compression module, configured to compress according to the original data to obtain compressed data; A data processing module, configured to preprocess the compressed data according to the priority and the real-time requirement to obtain data features; A data analysis module, configured to analyze according to the data features to obtain the optimal data transmission scheme; A data monitoring module, configured to perform real-time monitoring during the data transmission process according to the optimal transmission scheme to obtain the energy consumption and resource usage of the device; A data prediction module, configured to perform analysis and prediction according to the energy consumption, the resource usage, and the working mode to obtain an energy consumption prediction result; A data generation module, configured to dynamically generate according to the energy consumption prediction result to obtain the optimal working parameters and data transmission strategy of the device, and through strategy distribution and a remote control interface, adjust the running state of the device in real time, while ensuring data quality and real-time performance, minimizing the overall energy consumption of the system; In a third aspect, the present invention further provides an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the data acquisition method described in any one of the above.

[0016] In a fourth aspect, the present invention further provides a computer-readable storage medium, where the computer-readable storage medium includes a stored computer program. When the computer program runs, it controls the device where the computer-readable storage medium is located to execute the data acquisition method described in any one of the above.

[0017] Compared with the prior art, the present invention has the following beneficial effects: The present invention discloses a data acquisition method, including obtaining the original data of the current data acquisition task; wherein, the original data includes the acquisition frequency, the data volume size, the real-time requirement, and the priority; determining according to the original data to obtain the working mode of the device; compressing according to the original data to obtain compressed data; preprocessing the compressed data according to the priority and the real-time requirement to obtain data features; analyzing according to the data features to obtain the optimal data transmission scheme; performing real-time monitoring during the data transmission process according to the optimal transmission scheme to obtain the energy consumption and resource usage of the device; analyzing and predicting according to the energy consumption, the resource usage, and the working mode to obtain an energy consumption prediction result; dynamically generating according to the energy consumption prediction result to obtain the optimal working parameters and data transmission strategy of the device, and adjusting the operation state of the device in real time through the data transmission strategy and the optimal working parameters. The present invention dynamically adjusts the device working mode and resource allocation according to the real-time requirements, priorities, and data volume sizes of different tasks, and performs compression processing on the collected original data, combines the data features and task requirements, and adjusts the transmission strategy in real time. Compared with the prior art, the collected original data often has a large volume and requires a large amount of energy and bandwidth resources for acquisition and transmission. The present invention can solve the problem of excessive energy consumption in various complex and changeable data acquisition scenarios in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a schematic flowchart of the data acquisition method provided by the first embodiment of the present invention; Figure 2 is a schematic diagram of the data acquisition and transmission structure provided by the present invention; Figure 3 is a schematic diagram of the structure of the data acquisition device provided by the second embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0020] Referring to Figure 1 , the first embodiment of the present invention provides a data acquisition method, including the following steps: S11, obtaining the original data of the current data acquisition task; wherein, the original data includes the acquisition frequency, the data volume size, the real-time requirement, and the priority; S12. Determine the working mode of the device based on the original data; S13. Compress the original data to obtain compressed data; S14. Preprocess the compressed data according to the priority and the real-time requirement to obtain data features; S15. Analyze according to the data features to obtain the optimal data transmission scheme; S16. Monitor in real time during data transmission according to the optimal transmission scheme to obtain the energy consumption and resource usage of the device; S17. Analyze and predict according to the energy consumption, the resource usage and the working mode to obtain an energy consumption prediction result; S18. Dynamically generate according to the energy consumption prediction result to obtain the optimal working parameters and data transmission strategy of the device, and issue the optimal working parameters and the data transmission strategy to adjust the running state of the device in real time.

[0021] In step S11, obtain the original data of the current data acquisition task; wherein, the original data includes the acquisition frequency, the data volume size, the real-time requirement and the priority.

[0022] The acquisition frequency refers to the number of times the data acquisition task is executed per unit time, which determines the periodicity of data acquisition. For example, it can be acquired once per hour, per day, per week or per month. The choice of acquisition frequency depends on the data update speed and business requirements. For example, for financial transaction data with high real-time requirements, the acquisition frequency needs to be set to once per minute or even higher; while for some static data with slow changes, such as census data, the acquisition frequency is once per year. The acquisition frequency can be divided into high acquisition frequency and low acquisition frequency. High acquisition frequency is suitable for scenarios with high requirements for data continuity, such as real-time monitoring systems, high-frequency trading systems, etc. High-frequency acquisition can capture more details, but it will increase the data volume size and processing energy consumption. Low acquisition frequency is suitable for scenarios with low requirements for data real-time, such as environmental monitoring, equipment status monitoring, etc. Low-frequency acquisition can reduce the data volume and lower the storage and processing energy consumption. The choice of acquisition frequency needs to be determined according to the data characteristics. For example, in the process of signal acquisition, the sampling frequency must be greater than twice the signal frequency (Nyquist theorem) to avoid signal distortion.

[0023] The size of the data volume refers to the amount of data obtained in each data collection task, measured in bytes, megabytes (MB), gigabytes (GB), etc. The size of the data volume depends on the collection frequency, collection time, and the complexity of the collected data. Determining the size of the data volume requires considering storage resources and processing capabilities. For example, for large-scale Internet of Things sensor data collection, the data volume is extremely large and requires an efficient storage and processing system to support. At the same time, the size of the data volume also affects the data transmission time and energy consumption, so it needs to be reasonably configured according to requirements. Among them, a large data volume is suitable for scenarios that require high-precision and high-resolution data, such as high-definition video surveillance, large-scale Internet of Things data collection, etc. A large data volume requires stronger storage and processing capabilities. A small data volume is suitable for scenarios with low requirements for data accuracy, such as simple status monitoring, log recording, etc. A small data volume can reduce storage and processing energy consumption.

[0024] The real-time requirement refers to the timeliness requirement for data update in the data collection task. Tasks with high real-time requirements need to complete data collection and processing as soon as possible after the data is generated. For example, in an autonomous driving system, the data of vehicle sensors needs to be collected and processed in real time to ensure that the vehicle can respond in a timely manner. The real-time requirement can be achieved by setting the data collection interval time and the priority of data transmission. Among them, high real-time is suitable for scenarios that require quick response, such as autonomous driving, industrial automation control, etc. High real-time requirements need high-performance collection devices and low-latency network transmission. Low real-time is suitable for scenarios with low requirements for data timeliness, such as historical data analysis, log recording, etc. Low-real-time tasks can tolerate a certain amount of data delay. Achieving the real-time requirement needs to consider factors such as the performance of the collection device, network bandwidth, and data processing ability.

[0025] Priority refers to the ranking of the importance of each task when multiple data collection tasks are carried out simultaneously. Setting the priority can help the system give priority to processing more important tasks when resources are limited. For example, in an enterprise, the data collection task of financial data has a higher priority than the data collection task of market research data. The priority allocation can be determined according to factors such as the urgency of the task and the impact on the business. Among them, high priority is suitable for tasks that have a greater impact on the business, such as financial transaction data collection, key equipment status monitoring, etc. High-priority tasks require more resource support. Low priority is suitable for tasks that have a smaller impact on the business, such as the status record of non-critical equipment, auxiliary data collection, etc. Low-priority tasks can be processed when resources permit.

[0026] In step S12, it is determined according to the original data to obtain the working mode of the device.

[0027] In one implementation, the original data is analyzed to obtain a high-performance mode with a higher priority; Allocate computing resources and network resources according to the high-performance mode to obtain an allocation ratio; Execute the real-time monitoring data collection task according to the allocation ratio to obtain an execution ratio; Dynamically adjust the collection frequency and resource allocation according to the execution ratio to obtain the working mode of the device.

[0028] In one implementation, determine the working mode and parameter configuration of the device according to the collection frequency, data volume size, real-time requirement, and priority. If the real-time requirement is high and the priority is high, adopt the high-performance mode and increase the collection frequency. Allocate more computing resources and network resources from low-priority tasks to support the data collection task in the high-performance mode. Adjust the allocation ratio of computing resources and network resources according to the working mode and parameter configuration of the device. Dynamically adjust the collection frequency and resource allocation by monitoring the execution of the data collection task in real time. Re-evaluate the working mode and parameter configuration of the device according to the change of task requirements to ensure the efficient execution of the data collection task.

[0029] The allocation ratio refers to allocating computing resources and network resources to different tasks or modules according to a certain ratio according to the high-performance mode. For example: Suppose the device has two tasks to execute: Task A (data processing) and Task B (data transmission). According to the analysis of the high-performance mode, Task A requires more computing resources, while Task B requires more network resources. Therefore, the resource allocation ratio may be as follows: Computing resources: Task A accounts for 70%, and Task B accounts for 30%. Network resources: Task A accounts for 30%, and Task B accounts for 70%. The execution ratio refers to monitoring the execution of the task in real time after resource allocation, reflecting the actual resource ratio occupied by the task. For example: After resource allocation, the device starts to execute Task A and Task B. Through real-time monitoring, it is found that Task A actually occupies 65% of the computing resources and 35% of the network resources, while Task B actually occupies 35% of the computing resources and 65% of the network resources. Therefore, the execution ratio may be as follows: Computing resources: Task A accounts for 65%, and Task B accounts for 35%. Network resources: Task A accounts for 35%, and Task B accounts for 65%.

[0030] For tasks with higher priorities, further analyze their performance requirements. For example, tasks with high real-time requirements need a response time in milliseconds, and tasks with high data volumes need to process several gigabytes of data per second. Based on the above analysis, define the high-performance mode. The high-performance mode includes the following characteristics: high computing resource allocation, high network bandwidth, and high acquisition frequency. High computing resource allocation will allocate more CPU cores and memory resources. High network bandwidth can ensure high-speed and low-latency data transmission. High acquisition frequency can adjust the acquisition frequency according to task requirements, such as increasing from once per second to ten times per second. Evaluate the current resource situation of the system, including the number of available CPU cores, memory capacity, network bandwidth, etc. According to the requirements of the high-performance mode, formulate a resource allocation strategy. For example, allocate more resources to high-priority tasks and fewer resources to low-priority tasks. Calculate the resource allocation ratio for each task based on the task's priority and resource requirements. Allocate the system resources to each task according to the calculated allocation ratio. For example, if a task's allocation ratio is 0.3, then it will obtain 30% of the total system resources.

[0031] Exemplarily, data acquisition task management is a complex process that requires dynamic adjustment of strategies based on multiple factors. First, obtaining task requirement information is the crucial first step. For example, for a smart city project, data such as traffic flow, air quality, and energy consumption need to be collected. Traffic flow data needs to be collected once every minute and has a large data volume; air quality data is collected once every hour and has a small data volume; while energy consumption data is collected once a day and has a medium data volume. Based on these requirements, the working mode and parameter configuration of the device can be determined. For example, due to the high frequency and large data volume of traffic flow data collection, a high-performance mode needs to be adopted. The high-performance mode requires the use of a more powerful processor, larger storage space, and faster network connection. In contrast, standard modes can be used for air quality and energy consumption data collection. In terms of resource allocation, it needs to be adjusted according to the priority of the task. Suppose a traffic accident occurs, and the real-time requirement and priority of traffic flow data suddenly increase. In this case, a part of the computing resources and network bandwidth need to be allocated from the energy consumption data collection task to support the high-frequency collection of traffic flow data. Dynamic adjustment is the key to ensuring the efficient execution of data acquisition tasks. For example, if it is found that during the morning and evening rush hours, the collection frequency of traffic flow data is insufficient to capture rapidly changing situations, the collection frequency needs to be increased from once every minute to once every 30 seconds. At the same time, the storage space and network bandwidth allocated to this task need to be increased accordingly. Changes in task requirements also lead to the need to re-evaluate the working mode of the entire system. For example, if the city decides to launch a large-scale air quality improvement plan, then the collection of air quality data needs to be upgraded to a high priority and a high-performance mode is adopted. This requires reconfiguring the device, including upgrading hardware, adjusting software parameters, etc. Through this dynamic and responsive management method, it can be ensured that the data acquisition system can flexibly respond to various situations, maximize resource utilization efficiency, and meet the requirements of different tasks at the same time.

[0032] In step S13, the original data is compressed to obtain compressed data.

[0033] The collected original data is initially compressed using a preset compression algorithm. If the data contains continuously varying time series information, the differences between adjacent data are encoded using a differential coding algorithm to obtain compressed difference data. Based on the compressed difference data, the dynamic range of the data is determined, and the dynamic range is further optimized through preset rules to obtain an optimized data range. For the optimized data range, a time series analysis method is used to segment the continuously varying data to obtain segmented time series data. According to the segmented time series data, the differences between adjacent data within each segment are calculated using a differential coding algorithm to obtain the difference sequence for each segment. For the difference sequence of each segment, a preset compression algorithm is used for secondary compression processing to obtain the final compressed data. Based on the final compressed data, it is judged whether the data meets the preset compression ratio requirement. If it meets the requirement, the compression result is output; if it does not meet the requirement, the compression algorithm parameters are readjusted. Through the above steps, the compression processing of the original data is completed to obtain a compressed data result that meets the requirements.

[0034] Exemplarily, data compression is an important means to optimize data storage and transmission efficiency. For the original data containing continuously changing time series information, the differential coding algorithm can effectively reduce data redundancy. For example, in meteorological monitoring, temperature data usually shows a continuous changing trend. If the original temperature data is 20°C, 21°C, 23°C, 22°C, the difference sequence of 1°C, 2°C, -1°C can be obtained through differential coding, significantly reducing the data storage space. Determining the data dynamic range and optimizing it are key steps in the compression process. Taking stock price data as an example, if the intraday trading price fluctuation range of a certain stock is from 98 yuan to 102 yuan, the data range can be optimized to 95 yuan to 105 yuan, which not only ensures data integrity but also reserves space for subsequent compression. This optimization can improve the adaptability and efficiency of the compression algorithm. The time series analysis method segments the continuously changing data, which can more accurately capture data characteristics. In seismic waveform data processing, the waveform can be segmented according to different phases such as P-waves and S-waves. The data change characteristics within each segment are similar, which is beneficial to improving the compression effect. Applying the differential coding to the segmented data again can further reduce data redundancy. Secondary compression of each segment difference sequence is an effective means to improve the overall compression ratio. In the field of video coding, the difference information between adjacent frames can be compressed. For example, the motion estimation and compensation technology is a typical application based on inter-frame differences, which can significantly reduce the video data volume. Judging the compression ratio requirement and adjusting parameters ensure the quality of the compression result. In medical image compression, if the size of the CT image after the first compression does not reach the expected target, the compression ratio can be increased by adjusting the quantization parameter or increasing the threshold of the transform coefficient. This iterative optimization process can minimize the file size to the greatest extent while ensuring image quality. Through this series of steps, efficient compression of the original data can be achieved. In practical applications, such as remote sensing image processing, the above methods can be combined with domain-specific compression technologies. For example, for multi-spectral remote sensing images, spectral inter-correlation analysis and decorrelation processing are first performed, and then the compression process described in this article is applied, which can make full use of data characteristics and obtain better compression effects.

[0035] In step S14, preprocess the compressed data according to the priority and the real-time requirement to obtain data characteristics.

[0036] In one implementation, calculate according to the priority and the real-time requirement to obtain the dynamic index of the data; Determine according to the dynamic index to obtain the parameter value range of the preprocessing algorithm; Calculate according to the parameter value range and the preset parameter threshold to obtain the adjusted parameter value; Calculate according to the priority and the real-time requirement to obtain the quality value and efficiency value of the data; Calculate according to the quality value and the efficiency value to obtain a balance value; Perform preprocessing according to the balance value and the adjustment parameter to obtain data features.

[0037] In one implementation, the dynamicity index is calculated in the following way: where, is the dynamicity index, is the priority weight coefficient, is the real-time requirement weight coefficient, is an extremely small positive number, is the priority of the data, is the real-time requirement of the data.

[0038] In one implementation, the adjustment parameter value is calculated in the following way: where, is the adjustment parameter value, is the parameter value range, is the adjustment coefficient, is the data volume size, is the preset threshold.

[0039] In one implementation, the quality value is calculated in the following way: where, is the quality value, δ1 is the priority weight coefficient, is the real-time requirement weight coefficient, is the priority of the data, is the real-time requirement of the data, is an extremely small positive number.

[0040] In one implementation, the efficiency value is calculated in the following way: where, is the quality value, δ1 is the priority weight coefficient, is the real-time requirement weight coefficient, is the priority of the data, is the real-time requirement of the data.

[0041] In one implementation, the balance value is calculated in the following way: where, is the balance value, is the weight coefficient, is the quality value, is the quality value.

[0042] Dynamicity metrics are used to measure the frequency and amplitude of data changes over time. Common dynamicity metrics include: rate of change, amplitude of fluctuation, and frequency. The rate of change refers to the amount of change in data per unit time. The amplitude of fluctuation refers to the difference between the maximum and minimum values of the data. Frequency refers to the frequency of data changes, which is calculated using Fourier transform. By analyzing the time series characteristics of data, metrics such as the rate of change and amplitude of fluctuation of the data are calculated. For example, a sliding window is used to calculate the mean and standard deviation within each window. Statistical characteristics of the data (such as mean, variance) are calculated within a fixed time window, and the changes of these statistical characteristics over time are observed. For example, a moving average is used to smooth the data. The preprocessing algorithm usually contains multiple parameters, such as smoothing parameter, sampling rate, filter parameters, etc. The value ranges of these parameters need to be adjusted according to the dynamicity metrics. For data with higher dynamicity, the smoothing parameter can be set smaller to reduce data fluctuations. The sampling rate is adjusted according to the dynamicity of the data. Higher sampling rates are required for data with high dynamicity. A suitable filter (such as a low-pass filter, high-pass filter) is selected and its parameters are adjusted to remove noise.

[0043] The preset threshold is the upper and lower limits of the parameter range set according to the experiment. For example, the threshold range of the smoothing parameter is [0.1, 1.0]. If the calculated parameter value exceeds the preset threshold range, it is adjusted to within the threshold range. For example, if the calculated value of the smoothing parameter is 0.05 and the threshold range is [0.1, 1.0], it is adjusted to 0.1. According to the dynamicity of the real-time data, the parameter values are dynamically adjusted to adapt to the changes in the data. Preprocessing is performed according to the balance value and adjusted parameters. Duplicate data is removed through a hash table or sorting algorithm. Missing values are filled using the mean, median, or interpolation method. Z-score or IQR methods are used to detect and remove outliers. By extracting the cleaned data, data characteristics are obtained.

[0044] The value range of data priority is determined according to the task preset. The larger the value, the higher the priority. Data timeliness is the time interval from data generation to the current time. The dynamicity index is an index calculated by comprehensively considering data priority and timeliness. The preprocessing algorithm parameters are represented by the vector Params = [p1, p2, …, pn], where pi is the i-th parameter. For each parameter pi, the value range is [Li, Ui], where Li is the lower limit and Ui is the upper limit. The data volume size is the number of data in the dataset. The data quality value reflects the accuracy, completeness, etc. of the data. The data efficiency value reflects the efficiency of data processing, such as speed. Balance value: used to balance data quality and processing efficiency. α and β are weight coefficients used to adjust the influence degree of priority and timeliness on the dynamicity index, and α + β = 1; ε is an extremely small positive number used to avoid the denominator being zero. ω is a weight coefficient used to adjust the importance of quality in the balance value.

[0045] In step S15, analyze according to the data characteristics to obtain the optimal data transmission scheme.

[0046] In one implementation, determine according to the data characteristics to obtain the minimum bandwidth and maximum delay threshold of the transmission link; Calculate according to the minimum bandwidth and the maximum delay threshold to obtain the comprehensive score of the transmission link; Compare the comprehensive score with the preset score threshold to obtain the transmission protocol; Perform dynamic adjustment according to the transmission protocol to obtain the optimal data transmission scheme.

[0047] Obtain the compressed data characteristics, and extract the data volume size, real-time requirement, and transmission distance parameters. Determine the minimum bandwidth and maximum delay threshold of the transmission link according to the data volume size and real-time requirement. Use a multi-objective optimization algorithm to calculate the comprehensive score of the transmission scheme by combining the transmission distance and compression ratio. If the comprehensive score is higher than the preset threshold, select a low-latency transmission protocol; otherwise, select a high-throughput transmission protocol. Dynamically adjust the priority and protocol parameter configuration of the transmission link according to the task scenario and demand changes. Predict the optimal transmission scheme under different task scenarios through a machine learning algorithm to generate a dynamic configuration table. Update the transmission link and protocol parameters in real time according to the dynamic configuration table to adapt to the task scenario and demand changes.

[0048] It should be noted that the time required for transmission is determined according to the data volume size in the data characteristics. For example, if the data volume is 10MB and the transmission is required to be completed within 10 seconds, the minimum bandwidth needs to reach 1MB / s.

[0049] Determine the maximum delay threshold according to the real-time requirement of the data. For example, for data with high real-time requirements (such as financial transaction data), the maximum delay threshold is set to 10 milliseconds; while for non-real-time data (such as log data), the maximum delay threshold can be relaxed to 100 milliseconds. Calculate the minimum bandwidth by analyzing the data volume and transmission time requirement. For example, if the data volume is 10MB and it is required to complete the transmission within 10 seconds, the minimum bandwidth is 1MB / s. In practical applications, redundancy and burst traffic in the network are also considered, and the bandwidth is increased to cope with emergencies. Determine the maximum delay threshold according to the real-time requirement of the data. For example, for data with high real-time requirements, the maximum delay threshold is set to 10 milliseconds; for data with low real-time requirements, the maximum delay threshold can be set to 100 milliseconds. In the actual network, the delay fluctuates, so the maximum delay threshold needs to be adjusted according to the network delay situation. Set the preset threshold for the comprehensive score according to historical data and practical application experience. For example, if the comprehensive score is higher than 80 points, it indicates that the link performance is good; if the comprehensive score is lower than 60 points, it indicates that the link performance is poor.

[0050] If the link performance is good, select a low-latency transmission protocol, such as Cubic or BBR of TCP. These protocols can effectively reduce the transmission delay through congestion control mechanisms. If the link performance is poor, select a high-throughput transmission protocol, such as UDP or RTP. These protocols do not perform congestion control and can provide higher throughput, but will increase the packet loss rate. Monitor the bandwidth usage and delay situation of the network in real time through a network monitoring system. Dynamically adjust the bandwidth allocation according to the real-time monitoring data. For example, if the bandwidth utilization rate of a certain link is low, part of the bandwidth can be allocated to other links. Select an appropriate congestion control algorithm according to the transmission protocol. For example, for the TCP protocol, the Cubic or BBR algorithm can be selected. Adjust the transmission rate in real time according to the actual state of the network. Analyze the network topology structure through a network topology analysis tool to determine the available transmission paths. Select the optimal transmission path according to the comprehensive score and the preset threshold. Optimize the transmission scheme comprehensively according to the dynamically adjusted bandwidth, delay, and path.

[0051] In step S16, perform real-time monitoring during the data transmission process according to the optimal transmission scheme to obtain the energy consumption and resource usage of the device.

[0052] The power consumption value and resource value of the device are obtained through sensors, and the device status is judged whether it is within the normal range by combining with a preset threshold. If the device status is abnormal, a real-time monitoring mechanism is triggered to collect the timeliness and importance of the data stream, and a priority list is generated. According to the priority list, a dynamic adjustment algorithm is used to adjust the transmission policy, and the optimal link quality is selected for data transmission. During the data transmission process, the change of the link quality is continuously monitored. If the link quality is lower than the preset threshold, the backup link is switched. For critical data, a redundant transmission mechanism is adopted to ensure its reliability, while reducing the transmission priority of non-critical data. The historical data stream is analyzed by a machine learning algorithm to predict the change trend of the future link quality, and the adjustment frequency of the transmission policy is optimized. According to the prediction result and the device status, a resource allocation plan is generated to dynamically adjust the power consumption value and resource value of the device to achieve efficient transmission.

[0053] In step S17, based on the energy consumption, the resource usage, and the working mode, an analysis and prediction are performed to obtain an energy consumption prediction result.

[0054] Based on the energy consumption, the resource usage, and the working mode, a data mining algorithm is used to preprocess the data to remove noise and redundant information. For the preprocessed data, the association rules between the task features and the device mode are extracted, and an association rule mining algorithm is used to generate a feature mode association set. According to the feature mode association set, combined with the resource usage, a clustering algorithm is used to classify the device working mode to obtain a device mode classification result. For the device mode classification result, the association relationship between the resource usage and the energy consumption data is extracted, and a regression algorithm is used to establish an energy consumption prediction model. According to the energy consumption prediction model, an energy consumption analysis is performed on the device mode classification result to generate a device mode energy consumption prediction value. For the device mode energy consumption prediction value, combined with the resource allocation situation, an optimization algorithm is used to dynamically adjust the resource usage to generate a resource optimized allocation plan. According to the resource optimized allocation plan, the energy consumption prediction model is updated, and the device mode energy consumption prediction value is recalculated to form a closed-loop optimization process.

[0055] In step S18, based on the energy consumption prediction result, the optimal working parameters and data transmission strategy of the device are dynamically generated, and the operation state of the device is adjusted in real time through the data transmission strategy and the optimal working parameters.

[0056] According to the energy consumption prediction results, a regression algorithm is used to calculate the energy consumption values of the device under different operating parameters, and the combination of operating parameters with the lowest energy consumption is determined. According to the output results of the energy consumption optimization model and combined with the data transmission requirements, a dynamic programming algorithm is used to generate a data transmission strategy, and the optimal combination of data transmission frequency and data compression ratio is determined. The operating parameters and transmission strategy are sent through the remote control interface to adjust the device operating state in real time. If the device operating state deviates from the preset threshold, the operating parameters and transmission strategy are recalculated. The device operating state data is obtained, and a clustering algorithm is used to analyze the data quality to determine whether the data meets the requirements of real-time and accuracy. If the data quality does not meet the standard, the transmission strategy is adjusted. According to the device operating state and data quality analysis results, the energy consumption optimization model is updated, and the operating parameters and transmission strategy are recalculated to form a closed-loop optimization process. The system energy consumption is monitored in real time through the remote control interface. If the system energy consumption exceeds the preset range, a dynamic adjustment mechanism is triggered to regenerate the operating parameters and transmission strategy. The optimized operating parameters and transmission strategy are stored in a preset database as a reference for subsequent energy consumption prediction and optimization adjustment.

[0057] For the convenience of understanding the present invention, some preferred embodiments of the present invention will be further described below.

[0058] The working process of the present invention is described below by taking a relatively common scenario as an example. Please also refer to Figure 2 , which is Figure 1 a schematic diagram of the working scenario of the method.

[0059] In an intelligent factory, in order to optimize the energy consumption and data transmission efficiency of devices, a comprehensive technical solution is adopted. First, by analyzing the acquisition frequency, data volume, real-time requirements, and priority of the devices, the original data of the current data acquisition task is obtained. For example, the acquisition frequency of key devices is 1 time per second, the data volume is large (such as 1MB for a high-definition camera), the real-time requirement is high (within 10 milliseconds), and the priority is 90; while the acquisition frequency of auxiliary devices is 1 time per minute, the data volume is small (such as 1KB for a temperature sensor), the real-time requirement is low (within 1 second), and the priority is 30. According to these original data, the working mode of the devices is determined: key devices adopt a high-performance mode to ensure sufficient resources, and auxiliary devices adopt a normal mode to save resources.

[0060] Next, the collected original data is compressed. Key devices use the JPEG algorithm to compress the image data by about 50%, and auxiliary devices use the LZW algorithm to compress the numerical data by about 30% to reduce the data transmission volume. Then, the compressed data is preprocessed according to the priority and real-time requirements of the devices, and data features are extracted. For example, the change rate and fluctuation amplitude of key device data are calculated, the smoothing parameter is dynamically adjusted to reduce data fluctuations, and features such as timestamps, means, and variances are extracted.

[0061] Based on these data characteristics, analyze and determine the optimal data transmission scheme. For example, the minimum bandwidth required by the critical device is 1MB / s and the maximum latency is 10 milliseconds, so the Cubic protocol of TCP is selected to ensure low-latency transmission; the minimum bandwidth of the auxiliary device is 0.5MB / s and the maximum latency is 100 milliseconds, and the UDP protocol is selected to improve transmission efficiency. At the same time, dynamically adjust the bandwidth allocation and congestion control according to the real-time network status to ensure the efficiency and stability of data transmission.

[0062] During the data transmission process, monitor the energy consumption and resource usage of the devices in real time. Monitor the energy consumption of the critical device (such as 100W) and the auxiliary device (such as 50W) through an intelligent electricity meter, and monitor the CPU utilization, memory consumption, and network bandwidth usage of the devices through system resource monitoring tools. When the energy consumption or resource usage exceeds the preset threshold, the system will issue an alarm to remind the administrator to handle it in time.

[0063] Furthermore, based on the monitored energy consumption, resource usage, and the working mode of the devices, use machine learning algorithms to establish an energy consumption prediction model, analyze the trends of energy consumption and resource usage of the devices, and predict future energy consumption. For example, predict that the energy consumption of the critical device will be 120kWh within the next 24 hours. According to these prediction results, dynamically generate the optimal working parameters and data transmission strategies for the devices. For example, adjust the operating frequency of the critical device from 1 time per second to 0.5 times per second, and at the same time adjust the data transmission bandwidth from 2MB / s to 1.5MB / s. Through these strategies, adjust the operating status of the devices in real time to ensure that while ensuring data quality and real-time performance, the overall energy consumption of the system is minimized.

[0064] In summary, the present invention discloses a data acquisition method, which includes obtaining the original data of the current data acquisition task; wherein, the original data includes the acquisition frequency, the data volume size, the real-time requirement, and the priority; determining according to the original data to obtain the working mode of the device; compressing according to the original data to obtain compressed data; preprocessing the compressed data according to the priority and the real-time requirement to obtain data features; analyzing according to the data features to obtain the optimal data transmission scheme; performing real-time monitoring during the data transmission process according to the optimal transmission scheme to obtain the energy consumption and resource usage of the device; analyzing and predicting according to the energy consumption, the resource usage, and the working mode to obtain an energy consumption prediction result; dynamically generating according to the energy consumption prediction result to obtain the optimal working parameters and data transmission strategy of the device, and adjusting the operating state of the device in real time through the data transmission strategy and the optimal working parameters. By aiming at the real-time requirements, priorities, and data volume sizes of different tasks, the present invention dynamically adjusts the device working mode and resource allocation, and compresses the collected original data, combines the data features and task requirements, and adjusts the transmission strategy in real time. Compared with the prior art, the collected original data often has a large volume and requires a large amount of energy and bandwidth resources for acquisition and transmission. The present invention can solve the problem of excessive energy consumption in various complex and changeable data acquisition scenarios in the prior art.

[0065] Referring to Figure 3 , the second embodiment of the present invention provides a data acquisition device, including: A data acquisition module, configured to obtain the original data of the current data acquisition task; wherein, the original data includes the acquisition frequency, the data volume size, the real-time requirement, and the priority; A mode confirmation module, configured to determine according to the original data to obtain the working mode of the device; A data compression module, configured to compress according to the original data to obtain compressed data; A data processing module, configured to preprocess the compressed data according to the priority and the real-time requirement to obtain data features; A data analysis module, configured to analyze according to the data features to obtain the optimal data transmission scheme; A data monitoring module, configured to perform real-time monitoring during the data transmission process according to the optimal transmission scheme to obtain the energy consumption and resource usage of the device; A data prediction module, configured to analyze and predict according to the energy consumption, the resource usage, and the working mode to obtain an energy consumption prediction result; A data generation module is used to dynamically generate based on the energy consumption prediction result to obtain the optimal working parameters and data transmission strategy of the device. Through the policy distribution and remote control interface, the running state of the device is adjusted in real time, minimizing the overall energy consumption of the system while ensuring data quality and real-time performance.

[0066] It should be noted that a data acquisition device provided in an embodiment of the present invention is used to execute all the process steps of a data acquisition method in the above embodiment. The working principles and beneficial effects of the two correspond one by one, so they will not be elaborated here.

[0067] An embodiment of the present invention also provides an electronic device. The electronic device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, such as an algorithm program. When the processor executes the computer program, the steps in the above-mentioned various data acquisition method embodiments are implemented, such as Figure 1 the step S11 shown. Alternatively, when the processor executes the computer program, the functions of each module / unit in the above-mentioned device embodiments are implemented, such as the data generation module.

[0068] Exemplarily, the computer program can be divided into one or more modules / units. The one or more modules / units are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program in the electronic device.

[0069] The electronic device can be a computing device such as a desktop computer, a notebook, a palm computer, and a smart tablet. The electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the above components are only examples of the electronic device and do not constitute a limitation on the electronic device. It may include more or fewer components than the above, or combine some components, or different components. For example, the electronic device may further include input / output devices, network access devices, a bus, etc.

[0070] The so-called processor may be a Central Processing Unit (CPU), or it may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The processor is the control center of the electronic device and connects various parts of the entire electronic device through various interfaces and circuits.

[0071] The memory can be used to store the computer program and / or modules. The processor realizes various functions of the electronic device by running or executing the computer program and / or modules stored in the memory, and by calling the data stored in the memory. The memory mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created according to the use of the mobile phone (such as audio data, phone book, etc.). In addition, the memory can include high-speed random access memory and can also include non-volatile memory, such as a hard disk, memory, plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage devices.

[0072] Among them, if the modules / units integrated in the electronic device are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above-described embodiment methods of the present invention, it can also be completed by a computer program instructing relevant hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice within the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0073] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the attached drawings of the device embodiments provided by the present invention, the connection relationships between the modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0074] The specific embodiments described above further elaborate on the purpose, technical solutions, and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the protection scope of the present invention. It is particularly pointed out that for those skilled in the art, any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data collection method, characterized in that: Executed by a computer, including: Obtaining the original data of the current data collection task; wherein the original data includes collection frequency, data volume, real-time requirements and priority; Determine according to the original data to obtain the working mode of the device; Compressing the original data to obtain compressed data; Preprocessing the compressed data according to the priority and the real-time requirement to obtain data features; Analyze the data characteristics to obtain an optimal data transmission solution; Performing real-time monitoring during the data transmission process according to the optimal transmission scheme to obtain the energy consumption and resource usage of the device; Analyze and predict the energy consumption, resource usage and working mode to obtain an energy consumption prediction result; Dynamically generate the optimal operating parameters and data transmission strategy of the device according to the energy consumption prediction result, and adjust the operating state of the device in real time through the data transmission strategy and the optimal operating parameters.

2. The data collection method according to claim 1, characterized in that: The determining according to the original data to obtain the working mode of the device includes: Analyze the original data to obtain a high-performance mode with a higher priority; Allocate computing resources and network resources according to the high-performance mode to obtain an allocation ratio; According to the allocation ratio, real-time monitoring of the execution of the data collection task is performed to obtain the execution ratio; The acquisition frequency and resource allocation are dynamically adjusted according to the execution ratio to obtain the working mode of the device.

3. The data collection method according to claim 1, characterized in that: The preprocessing of the compressed data according to the priority and the real-time requirement to obtain data features includes: Calculate according to the priority and the real-time requirement to obtain a dynamic index of the data; Determining according to the dynamic index, obtaining a parameter value range of a preprocessing algorithm; Calculate the parameter value based on the parameter value range and the preset parameter threshold to obtain the adjusted parameter value; Calculate according to the priority and the real-time requirement to obtain the quality value and efficiency value of the data; Calculate according to the mass value and the efficiency value to obtain a balance value; Preprocessing is performed according to the balance value and the adjustment parameter to obtain data features.

4. The data collection method according to claim 3, characterized in that: The calculation is performed according to the priority and the real-time requirement to obtain a dynamic index of the data, including: The dynamic index is calculated as follows: in, It is a dynamic indicator. is the priority weight coefficient, is the real-time requirement weight coefficient, is a very small positive number. is the priority of the data, It is the real-time requirement of data.

5. The data collection method according to claim 3, characterized in that: The step of calculating according to the parameter value range and the preset parameter threshold to obtain the adjusted parameter value includes: The tuning parameter value is calculated as follows: in, To adjust the parameter value, is the parameter value range, is the adjustment factor, is the amount of data, is the preset threshold.

6. The data collection method according to claim 3, characterized in that: The calculating according to the priority and the real-time requirement to obtain the quality value and efficiency value of the data includes: The mass value is calculated by: in, is the quality value, δ1 is the priority weight coefficient, is the real-time requirement weight coefficient, is the priority of the data, It is the real-time requirement of data. is a very small positive number.

7. The data collection method according to claim 3, characterized in that: The calculating according to the priority and the real-time requirement to obtain the quality value and efficiency value of the data includes: The efficiency value is calculated by: in, is the quality value, δ1 is the priority weight coefficient, is the real-time requirement weight coefficient, is the priority of the data, It is the real-time requirement of data.

8. The data collection method according to claim 3, characterized in that: The calculating according to the mass value and the efficiency value to obtain a balance value includes: The equilibrium value is calculated by: in, is the equilibrium value, is the weight coefficient, is the quality value, is the quality value.

9. The data collection method according to claim 1, characterized in that: The step of analyzing the data characteristics to obtain an optimal data transmission solution includes: Determine according to the data characteristics to obtain a minimum bandwidth and a maximum delay threshold of the transmission link; Calculate according to the minimum bandwidth and the maximum delay threshold to obtain a comprehensive score of the transmission link; Comparing the comprehensive score with a preset score threshold to obtain a transmission protocol; Dynamic adjustments are made according to the transmission protocol to obtain the optimal data transmission solution.

10. A data acquisition device, characterized in that: include: The data acquisition module is used to acquire the original data of the current data acquisition task; wherein the original data includes the acquisition frequency, data volume, real-time requirements and priority; A mode confirmation module, used to determine according to the original data to obtain the working mode of the device; A data compression module, used for compressing the original data to obtain compressed data; A data processing module, used for preprocessing the compressed data according to the priority and the real-time requirement to obtain data features; A data analysis module, used to analyze the data characteristics and obtain an optimal data transmission solution; A data monitoring module is used to perform real-time monitoring during the data transmission process according to the optimal transmission scheme to obtain the energy consumption and resource usage of the device; A data prediction module, used to analyze and predict according to the energy consumption, the resource usage and the working mode, to obtain an energy consumption prediction result; The data generation module is used to dynamically generate the optimal working parameters and data transmission strategy of the equipment according to the energy consumption prediction result, and adjust the operating state of the equipment in real time through the data transmission strategy and the optimal working parameters.

Images