Data processing method and system based on edge computing

Abstract

The invention provides a data processing method and system based on edge computing, and relates to the technical field of data processing.The method comprises the steps that an original data set of edge computing is subjected to noise reduction and standardization processing to obtain a data fluctuation factor, and then the expected speed of data transmission is calculated according to real-time network state characteristics and the data fluctuation factor; and finally, gradually and smoothly adjusting the actual uploading speed to an expected target uploading speed through a speed adjusting formula, thereby effectively coping with network fluctuation, avoiding unstable factors possibly caused by sudden and large changes, reducing packet loss and delay in data transmission, improving stability of data transmission, and improving data transmission efficiency while ensuring data transmission quality. According to the method, the available bandwidth resources are utilized to the maximum extent, resource waste is avoided, the integrity and consistency of the data set are ensured through abnormal value detection and replacement, and the robustness and stability of the system are improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data processing method and system based on edge computing. Background Art

[0002] With the rapid development of technologies such as the Internet of Things, big data, and cloud computing, the amount of data has increased dramatically, putting higher requirements on the efficiency and reliability of data transmission. Traditional data processing and transmission methods often rely on centralized servers, which have problems such as high latency and limited bandwidth when facing a large number of dispersed data sources. Edge computing, as an emerging computing model, transfers data processing tasks from central servers to edge devices close to data sources, thereby reducing data transmission delays and improving response speed and overall system performance. However, in actual applications, the network environment in which edge devices are located is often complex and changeable. Factors such as network bandwidth, latency, and packet loss rate can significantly affect the quality of data transmission. Therefore, how to dynamically adjust the data upload speed according to the real-time network status to ensure the stability and efficiency of data transmission has become an important research topic.

[0003] Therefore, it is necessary to provide a data processing method and system based on edge computing to solve the above technical problems. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a data processing method and system based on edge computing, which can achieve the beneficial effect of ensuring the stability and efficiency of edge computing data transmission.

[0005] The present invention provides a data processing method based on edge computing, the data processing method comprising the following steps: Perform noise reduction processing on the original data set uploaded to the data receiving end, and obtain the network status characteristics of the local end storing the original data set, the network status characteristics including the current available bandwidth value, the historical average available bandwidth value, the current network delay, the historical average network delay, the current packet loss rate, and the historical average packet loss rate; The z-score statistical method was used to standardize the original data set after noise reduction; Calculate the data fluctuation factor based on the original data set after standardization; Calculate the target upload speed based on the network status characteristics and the data fluctuation factor, wherein the target upload speed represents the expected speed of uploading to the data receiving end; A speed adjustment difference is calculated based on the target upload speed and the actual upload speed. If the speed adjustment difference is not within the speed adjustment tolerance range, the actual upload speed is adjusted to the target upload speed. Otherwise, the existing actual upload speed is kept unchanged, wherein the speed adjustment difference is the difference between the target upload speed and the actual upload speed.

[0006] Preferably, the noise reduction process performed on the original data set uploaded to the data receiving end is one of a filter, a wavelet transform, and a Fourier transform.

[0007] Preferably, the specific steps of using the z-score statistical method to standardize the original data set after noise reduction are: Calculate the mean of the original data set after denoising; Calculate the standard deviation of the original data set after denoising; The z-score formula is applied to standardize all data in the original data set after noise reduction. The standardization formula is:

[0008] in, The first data points, is the original data set after denoising. data points, is the mean of the original data set after denoising, is the standard deviation of the original data set after denoising.

[0009] Preferably, it also includes: Use the outlier detection algorithm to perform anomaly detection on the standardized original data set; Based on the preset anomaly judgment threshold, mark the data greater than the anomaly judgment threshold as an outlier, calculate the median of all non-outlier values, and replace all marked outliers with the median.

[0010] Preferably, the calculation formula of the data fluctuation factor is:

[0011] in, is the data fluctuation factor, is the adjustment factor, is the variance of the original data set after standardization, is the original data set after standardization.

[0012] Preferably, the target upload speed is calculated as follows:

[0013] in, is the target upload speed, is the maximum upload speed, For the current moment Available bandwidth value, is the historical average available bandwidth value, For the current moment Network latency, is the historical average network delay, For the current moment Packet loss rate.

[0014] Preferably, the formula for gradually adjusting the actual upload speed to the target upload speed is:

[0015] in, For the current moment The actual upload speed, For the current moment The moment before The actual upload speed, is the target upload speed, is the speed adjustment factor.

[0016] Preferably, in the process of adjusting the actual upload speed to the target upload speed one by one, a maximum adjustment number is set. If the adjustment number reaches the maximum adjustment number and the actual upload speed is not adjusted to the target upload speed, the adjustment is stopped.

[0017] The present invention also provides a data processing system based on edge computing, which is applied to a data processing method based on edge computing. The data processing system includes: A data acquisition module is used to perform noise reduction processing on the original data set uploaded to the data receiving end, and obtain the network status characteristics of the local end storing the original data set, the network status characteristics including the current available bandwidth value, the historical average available bandwidth value, the current network delay, the historical average network delay, the current packet loss rate, and the historical average packet loss rate; A standardization processing module is used to standardize the original data set after noise reduction using the z-score statistical method; A data fluctuation factor calculation module is used to calculate the data fluctuation factor based on the original data set after standardization; A target upload speed calculation module, used to calculate a target upload speed based on network status characteristics and data fluctuation factors, wherein the target upload speed represents a speed of uploading to a data receiving end; The upload speed adjustment module is used to calculate the speed adjustment difference based on the target upload speed and the actual upload speed. If the speed adjustment difference is not within the speed adjustment tolerance range, the actual upload speed is adjusted to the target upload speed; otherwise, the existing actual upload speed is kept unchanged, wherein the speed adjustment difference is the difference between the target upload speed and the actual upload speed.

[0018] Preferably, it also includes: An anomaly processing module is used to perform anomaly detection on the original data set after standardization using an outlier detection algorithm; Based on the preset anomaly judgment threshold, mark the data greater than the anomaly judgment threshold as an outlier, calculate the median of all non-outlier values, and replace all marked outliers with the median.

[0019] Compared with the related art, the data processing method and system based on edge computing provided by the present invention have the following beneficial effects: The present invention obtains the data fluctuation factor by denoising and standardizing the original data set of edge computing, and then calculates the expected speed of data transmission according to the real-time network status characteristics and the data fluctuation factor, and finally smoothly adjusts the actual upload speed to the expected target upload speed through the speed adjustment formula. It can effectively cope with network fluctuations, avoid unstable factors that may be caused by sudden and drastic changes, reduce packet loss and delay in data transmission, and improve the stability of data transmission. While ensuring the quality of data transmission, it maximizes the use of available bandwidth resources and avoids resource waste. It also ensures the integrity and consistency of the data set through outlier detection and replacement, improves the robustness and stability of the system, and sets the maximum number of adjustments. When the number of adjustments reaches the maximum value and the actual upload speed has not been adjusted to the target upload speed, the adjustment is stopped to prevent unlimited adjustments from causing system instability. The above technical measures effectively ensure the reliable operation of data transmission in edge computing. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of a data processing method based on edge computing of the present invention; Figure 2 This is a schematic diagram of the module structure of a data processing system based on edge computing of the present invention. DETAILED DESCRIPTION

[0021] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only the parts related to the present invention, rather than all structures, are shown in the accompanying drawings. In addition, the embodiments of the present invention and the features in the embodiments may be combined with each other without conflict.

[0022] It should also be noted that, for ease of description, only the parts related to the present invention, but not all of the contents, are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the various operations (or steps) as being processed sequentially, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the various operations can be rearranged. The processing can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The processing can correspond to methods, functions, procedures, subroutines, subprograms, etc.

[0023] Embodiment 1 A data processing method based on edge computing, in the specific implementation process, such as Figure 1 As shown, it shows a flow chart of a data processing method based on edge computing, including: Step S1: perform noise reduction processing on the original data set uploaded to the data receiving end, and obtain the network status characteristics of the local end storing the original data set, the network status characteristics include the current available broadband value, the historical average available broadband value, the current network delay, the historical average network delay, the current packet loss rate, the historical average packet loss rate, use the network monitoring tool to obtain the current network status characteristics in real time, store the network status characteristics obtained each time in the database, and use it to calculate the historical average.

[0024] During the specific implementation process, the original data set uploaded to the data receiving end is first preprocessed. The preprocessing includes noise reduction. The original data set after noise reduction is convenient for other subsequent edge computing processing. In addition, in order to ensure the stability and efficiency of the subsequent upload of the original data set, it is necessary to obtain the network status characteristics of the local end. The network status characteristics include the current available broadband value, the historical average available broadband value, the current network delay, the historical average network delay, the current packet loss rate, and the historical average packet loss rate. These characteristics are currently available and are effective resources related to the efficiency of uploading the original data set. Therefore, it is necessary to obtain this information to facilitate the subsequent adjustment of the upload speed of the original data set.

[0025] Specifically, in step S1, the original data set uploaded to the data receiving end is subjected to noise reduction processing such as a filter, a wavelet transform, or a Fourier transform.

[0026] In the specific implementation process, for example, discrete wavelet transform technology is used to reduce the noise of the original data set, db4 wavelet basis function and 3 decomposition levels are selected to process the original data set, threshold processing is performed on the obtained wavelet coefficients to remove the noise of the original data set, and an inverse discrete wavelet transform is applied to reconstruct the data to obtain the original data set after noise reduction.

[0027] Step S2: Use the z-score statistical method to standardize the original data set after noise reduction.

[0028] Specifically, the specific steps of using the z-score statistical method to standardize the original data set after noise reduction are as follows: Calculate the mean of the original data set after denoising; Calculate the standard deviation of the original data set after denoising; The z-score formula is applied to standardize all data in the original data set after noise reduction. The standardization formula is:

[0029] in, The first data points, is the original data set after denoising. data points, is the mean of the original data set after denoising, is the standard deviation of the original data set after denoising.

[0030] In the specific implementation process, the original data set after noise reduction is standardized using the z-score statistical method. The z-score statistical method first calculates the mean and standard deviation of all data points in the original data set, and then uses the z-score standardization processing formula in the above steps to calculate the standardized original data set.

[0031] Step S3: Use an outlier detection algorithm to perform anomaly detection on the normalized original data set.

[0032] Based on the preset anomaly judgment threshold, mark the data greater than the anomaly judgment threshold as an outlier, calculate the median of all non-outlier values, and replace all marked outliers with the median.

[0033] During the specific implementation process, after the original data set is standardized, it is necessary to detect and deal with the abnormal data in the data set to reduce the impact of abnormal data points on the accuracy of subsequent calculations. An abnormality judgment threshold is preset, and the absolute value of the standardized data point greater than the abnormality judgment threshold is marked as an outlier. The median of the remaining non-outlier values ​​is calculated, and the data points marked as outliers are replaced with the obtained median. Finally, the corrected original data set is obtained, which improves the overall quality of the data and the reliability of subsequent processing.

[0034] Step S4: Calculate the data fluctuation factor based on the normalized original data set.

[0035] Specifically, the calculation formula of the data fluctuation factor is:

[0036] in, is the data fluctuation factor, is the adjustment factor, is the variance of the original data set after standardization, is the original data set after standardization.

[0037] In the specific implementation process, in order to calculate the expected upload speed, it is necessary to consider the statistical characteristics of the original data set, and obtain a factor affecting the upload speed based on the statistical characteristics of the standardized original data set. In the above steps, the calculation formula of the data fluctuation factor is given, where the adjustment coefficient Controls the sensitivity of the data volatility factor to the degree of data volatility, and a larger adjustment coefficient Will make the data fluctuation factor more sensitive to data fluctuations, and the appropriate adjustment coefficient The value can balance the stability of the system and the response speed to data fluctuations. For example, the adjustment coefficient can be determined by test data. size.

[0038] Step S5: Calculate the target upload speed based on the network status characteristics and the data fluctuation factor, wherein the target upload speed represents the expected speed of uploading to the data receiving end, and the calculation formula of the target upload speed is:

[0039] in, is the target upload speed, is the maximum upload speed, For the current moment Available bandwidth value, is the historical average available bandwidth value, For the current moment Network latency, is the historical average network delay, For the current moment Packet loss rate.

[0040] In the specific implementation process, it is necessary to calculate a target upload speed based on the network status characteristics and data fluctuation factors. The target upload speed represents the expected speed of uploading to the data receiving end. The target upload speed takes into account the resources currently available for data upload and the statistical characteristics of the original data set. It is the optimal upload speed selection during data transmission. In the data processing process of edge computing, the target upload speed is the basis for the direction and amplitude of the current actual upload speed adjustment. The target upload speed in the formula The maximum upload speed is an upper limit set during system design to ensure that data upload does not exceed the physical limitations of the network or hardware. For example, The factors to be considered include but are not limited to the maximum upload bandwidth of the local network device, the reading speed of the storage device, the maximum upload bandwidth provided by the Internet service provider, and the needs of the actual application scenario.

[0041] Step S6: Calculate the speed adjustment difference based on the target upload speed and the actual upload speed. If the speed adjustment difference is not within the speed adjustment tolerance range, adjust the actual upload speed to the target upload speed. Otherwise, keep the existing actual upload speed unchanged. The speed adjustment difference is the difference between the target upload speed and the actual upload speed. The formula for adjusting the actual upload speed to the target upload speed is:

[0042] in, For the current moment The actual upload speed, For the current moment The moment before The actual upload speed, is the target upload speed, is the speed adjustment factor.

[0043] In the specific implementation process, the speed adjustment difference is first calculated according to the target upload speed and the actual upload speed. The speed adjustment difference has positive and negative values. If it is positive, the actual upload speed is increased to the target upload speed. If it is negative, the actual upload speed is reduced to the target upload speed. For example, the speed adjustment factor is determined according to the size of the speed adjustment difference and experimental experience. The value of is as large as possible while ensuring that the data fluctuation is not large. , the actual upload speed can be adjusted closer to the target upload speed more quickly, improving the efficiency of data transmission. For example, the speed adjustment factor is set The value is 0.1, assuming that the current target upload speed is calculated according to the target upload speed calculation formula The actual upload speed at the previous moment is 100Mbps. The speed adjustment difference is 20 Mbps, and the actual upload speed is adjusted to the target upload speed. If the actual upload speed is 82Mbps, the actual upload speed is adjusted to 82Mbps this time. After the preset response time interval, the formula for successively adjusting the actual upload speed to the target upload speed is used again to calculate and adjust the actual upload speed until the actual upload speed reaches the target upload speed, then the adjustment is stopped.

[0044] Specifically, in the process of adjusting the actual upload speed to the target upload speed one by one, a maximum adjustment number is set. If the adjustment number reaches the maximum adjustment number and the actual upload speed is not adjusted to the target upload speed, the adjustment is stopped.

[0045] During the specific implementation process, in order to prevent inefficiency and infinite adjustment loopholes, a maximum number of adjustments is set. Even if the actual upload speed does not reach the target upload speed, but the number of adjustments has reached the maximum number of adjustments, the adjustment will stop to ensure the stability of data upload and transmission efficiency, and prevent excessive adjustments from affecting transmission performance.

[0046] The working principle of a data processing method based on edge computing provided by the present invention is as follows: First, the original data set uploaded to the data receiving end is subjected to noise reduction processing in the edge computing environment, and the network status characteristics of the local end are obtained, including the current available broadband value, the historical average available broadband value, the current network delay, the historical average network delay, the current packet loss rate, and the historical average packet loss rate. Secondly, the z-score statistical method is used to standardize the noise-reduced data set. Thirdly, after the original data set is standardized, it is necessary to detect and deal with the abnormal data in the data set to reduce the impact of abnormal data points on the accuracy of subsequent calculations. An abnormal judgment threshold is preset, and the absolute value of the standardized data point greater than the abnormal judgment threshold is marked as an abnormal value. The median of the remaining non-abnormal values ​​is calculated, and the data points marked as abnormal values ​​are replaced with the obtained median to finally obtain the corrected original data set. Then, the data fluctuation factor is calculated based on the standardized data set. Because the statistical characteristics of the original data set need to be considered when uploading data, a factor affecting the upload speed is obtained based on the statistical characteristics of the standardized original data set. In the calculation formula of the data fluctuation factor, the adjustment coefficient Controls the sensitivity of the data volatility factor to the degree of data volatility, and a larger adjustment coefficient Will make the data fluctuation factor more sensitive to data fluctuations, and the appropriate adjustment coefficient The value can balance the stability of the system and the response speed to data fluctuations. With the data fluctuation factor, a target upload speed is calculated in combination with the network status characteristics. The target upload speed represents the expected speed of uploading to the data receiving end. The target upload speed comprehensively considers the resources currently available for data upload and the statistical characteristics of the original data set. It is the optimal upload speed selection during data transmission. In the data processing process of edge computing, the target upload speed is the basis for the direction and amplitude of the current actual upload speed adjustment. Finally, after the target upload speed is obtained, the actual upload speed is adjusted to the target upload speed to complete the optimization of the data transmission speed during edge computing. Adjusting the actual upload speed to the target upload speed is not completed in one step. In order to ensure the stability of data transmission and prevent the impact caused by network fluctuations, it is necessary to adjust smoothly. According to the formula for adjusting the actual upload speed to the target upload speed one by one, the actual upload speed is adjusted to the target upload speed one by one smoothly. In order to prevent inefficiency and infinite adjustment vulnerabilities, a maximum number of adjustments is also set. Even if the actual upload speed does not reach the target upload speed, but the number of adjustments has reached the maximum number of adjustments, the adjustment will stop to ensure the stability of data upload and transmission efficiency, and prevent excessive adjustments from affecting transmission performance.

[0047] Embodiment 2 A data processing system based on edge computing is applied to a data processing method based on edge computing. In the specific implementation process, Figure 2 As shown, it shows a schematic diagram of the module structure of a data processing system based on edge computing, including: The data acquisition module 100 is used to perform noise reduction processing on the original data set uploaded to the data receiving end, and obtain the network status characteristics of the local end storing the original data set, the network status characteristics include the current available bandwidth value, the historical average available bandwidth value, the current network delay, the historical average network delay, the current packet loss rate, and the historical average packet loss rate; A standardization processing module 200 is used to perform standardization processing on the original data set after noise reduction using a z-score statistical method; Anomaly processing module 300, used to perform anomaly detection on the original data set after normalization using an outlier detection algorithm; Based on the preset anomaly judgment threshold, mark the data greater than the anomaly judgment threshold as an outlier, calculate the median of all non-outlier values, and replace all marked outliers with the median.

[0048] A data fluctuation factor calculation module 400 is used to calculate the data fluctuation factor based on the original data set after the standardization process; The target upload speed calculation module 500 is used to calculate the target upload speed based on the network status characteristics and the data fluctuation factor, wherein the target upload speed represents the speed of uploading to the data receiving end; The upload speed adjustment module 600 is used to calculate the speed adjustment difference based on the target upload speed and the actual upload speed. If the speed adjustment difference is not within the speed adjustment tolerance range, the actual upload speed is adjusted to the target upload speed. Otherwise, the existing actual upload speed is kept unchanged, wherein the speed adjustment difference is the difference between the target upload speed and the actual upload speed.

[0049] The working principle of a data processing system based on edge computing provided by the present invention is as follows: The data acquisition module 100 is responsible for collecting the original data set from the data source and performing noise reduction processing on the data. The noise reduction processing uses one of the filter, wavelet transform or Fourier transform methods to remove the noise in the original data set and improve the data quality. At the same time, it also obtains the network status characteristics of the local end storing the original data set, including the current available bandwidth value, the historical average available bandwidth value, the current network delay, the historical average network delay, the current packet loss rate, and the historical average packet loss rate. This information is helpful for calculating the target upload speed in the subsequent steps. The standardization processing module 200 uses the z-score statistical method to standardize the original data set after noise reduction. The standardization processing can eliminate the dimensional differences between different features, so that each feature can be compared and analyzed on the same scale. After data standardization, the abnormality processing module 300 uses the abnormal value detection algorithm to identify abnormal data points. According to the preset abnormality judgment threshold, the data greater than the abnormality judgment threshold is marked as an abnormal value, and then the median of all non-abnormal values ​​is calculated, and all marked abnormal values ​​are replaced with the median. This step helps to reduce the impact of abnormal data on subsequent calculations. The data fluctuation factor calculation module 400 calculates the data fluctuation factor based on the original data set after standardization, quantifies the instability or variation of the data, and provides a basis for dynamically adjusting the upload speed. The data fluctuation factor takes into account the statistical characteristics of the original data set. According to the statistical characteristics of the standardized original data set, a factor data fluctuation factor affecting the upload speed is obtained, which reflects the degree of data fluctuation. The target upload speed calculation module 500 calculates the target upload speed based on the network status characteristics and the data fluctuation factor. The target upload speed represents the speed of uploading to the data receiving end. The network status characteristics and data volatility are comprehensively considered. This module calculates the optimal target upload speed, which is intended to balance the network load, avoid congestion, and ensure the efficiency and reliability of data transmission. It can be adjusted according to the current network status and data characteristics to ensure the efficiency and stability of uploading the original data set. The upload speed adjustment module 600 determines whether the upload speed needs to be adjusted based on the difference between the target upload speed and the actual upload speed monitored in real time. If the speed adjustment difference is not within the speed adjustment tolerance range, the actual upload speed is adjusted to the target upload speed; otherwise, the existing actual upload speed is kept unchanged, so as to achieve rapid response and optimization to network conditions.

[0050] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0051] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, the storage medium including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically-erasable programmable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0052] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

Claims

1. A data processing method based on edge computing, characterized in that: The data processing method comprises the following steps: Perform noise reduction processing on the original data set uploaded to the data receiving end, and obtain the network status characteristics of the local end storing the original data set, the network status characteristics including the current available bandwidth value, the historical average available bandwidth value, the current network delay, the historical average network delay, the current packet loss rate, and the historical average packet loss rate; The z-score statistical method was used to standardize the original data set after noise reduction; Calculate the data fluctuation factor based on the original data set after standardization; Calculate the target upload speed based on the network status characteristics and the data fluctuation factor, wherein the target upload speed represents the expected speed of uploading to the data receiving end; A speed adjustment difference is calculated based on the target upload speed and the actual upload speed. If the speed adjustment difference is not within the speed adjustment tolerance range, the actual upload speed is adjusted to the target upload speed. Otherwise, the existing actual upload speed is kept unchanged, wherein the speed adjustment difference is the difference between the target upload speed and the actual upload speed.

2. The data processing method based on edge computing according to claim 1, characterized in that: The original data set uploaded to the data receiving end is subjected to noise reduction processing by one of filter, wavelet transform and Fourier transform.

3. The data processing method based on edge computing according to claim 2 is characterized in that: The specific steps of using the z-score statistical method to standardize the original data set after noise reduction are as follows: Calculate the mean of the original data set after denoising; Calculate the standard deviation of the original data set after denoising; The z-score formula is applied to standardize all data in the original data set after noise reduction. The standardization formula is: in, The first data points, is the original data set after denoising. data points, is the mean of the original data set after denoising, is the standard deviation of the original data set after denoising.

4. The data processing method based on edge computing according to claim 3 is characterized in that: Also includes: Use the outlier detection algorithm to perform anomaly detection on the standardized original data set; Based on the preset anomaly judgment threshold, mark the data greater than the anomaly judgment threshold as an outlier, calculate the median of all non-outlier values, and replace all marked outliers with the median.

5. The data processing method based on edge computing according to claim 4 is characterized in that: The calculation formula of data fluctuation factor is: in, is the data fluctuation factor, is the adjustment factor, is the variance of the original data set after standardization, is the original data set after standardization.

6. The data processing method based on edge computing according to claim 5, characterized in that: The target upload speed is calculated as: in, is the target upload speed, is the maximum upload speed, For the current moment Available bandwidth value, is the historical average available bandwidth value, For the current moment Network latency, is the historical average network delay, For the current moment Packet loss rate.

7. The data processing method based on edge computing according to claim 6, characterized in that: The formula for gradually adjusting the actual upload speed to the target upload speed is: in, For the current moment The actual upload speed, For the current moment The moment before The actual upload speed, is the target upload speed, is the speed adjustment factor.

8. The data processing method based on edge computing according to claim 7, characterized in that: In the process of adjusting the actual upload speed to the target upload speed one by one, a maximum adjustment number is set. If the adjustment number reaches the maximum adjustment number and the actual upload speed is not adjusted to the target upload speed, the adjustment is stopped.

9. A data processing system based on edge computing, characterized in that: Applied to a data processing method based on edge computing as claimed in any one of claims 1 to 8, the data processing system comprising: A data acquisition module is used to perform noise reduction processing on the original data set uploaded to the data receiving end, and obtain the network status characteristics of the local end storing the original data set, the network status characteristics including the current available bandwidth value, the historical average available bandwidth value, the current network delay, the historical average network delay, the current packet loss rate, and the historical average packet loss rate; A standardization processing module is used to standardize the original data set after noise reduction using the z-score statistical method; A data fluctuation factor calculation module is used to calculate the data fluctuation factor based on the original data set after standardization; A target upload speed calculation module, used to calculate a target upload speed based on network status characteristics and data fluctuation factors, wherein the target upload speed represents a speed of uploading to a data receiving end; The upload speed adjustment module is used to calculate the speed adjustment difference based on the target upload speed and the actual upload speed. If the speed adjustment difference is not within the speed adjustment tolerance range, the actual upload speed is adjusted to the target upload speed; otherwise, the existing actual upload speed is kept unchanged, wherein the speed adjustment difference is the difference between the target upload speed and the actual upload speed.

10. The data processing system based on edge computing according to claim 9, characterized in that: Also includes: An anomaly processing module is used to perform anomaly detection on the original data set after standardization using an outlier detection algorithm; Based on the preset anomaly judgment threshold, mark the data greater than the anomaly judgment threshold as an outlier, calculate the median of all non-outlier values, and replace all marked outliers with the median.

Images