Multi-source heterogeneous data processing method and device
Through wavelet transformation and adaptive noise reduction processing, data standardization and DS reasoning methods, the noise and inconsistency problems of multi-source heterogeneous data processing in the new power system are solved, and the high-quality fusion and accuracy of data are achieved to meet the system various application requirements.
Patent Information
- Application Number
- CN202410784880.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-18
- Publication Date
- 2024-11-08
AI Technical Summary
Multi-source, isomerized and complex data processing in new power systems has brought huge pressure, and the existing technology cannot effectively reduce noise and integrate data, resulting in insufficient data accuracy and reliability.
A multi-source heterogeneous data processing method is proposed to remove noise through wavelet transformation and adaptive noise reduction treatment, followed by Yeo-Johnson transformation and standard score processing of standardized data, and data fusion is used to ensure data accuracy and consistency.
Improve the accuracy and reliability of multi-source data, provide high-quality data fusion results, and meet the different application needs of new power systems, such as power load prediction and equipment health monitoring.
Smart Images

Figure CN118916829A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of data processing technology, and in particular to a method and device for processing multi-source heterogeneous data. Background Art
[0002] With the development of new power systems, the proportion of renewable energy such as photovoltaics and wind power in the power system has increased, which has brought changes to the physical properties of the power system and also made the power system data heterogeneous and complex. At the same time, the access of various external systems will generate massive amounts of data, which are diverse, multi-source and uncertain. Various types of data from multiple sources, heterogeneity and complexity have brought tremendous pressure to the data processing of new power systems, directly affecting the development of new power systems. Summary of the invention
[0003] In view of this, an object of the embodiments of the present application is to provide a multi-source heterogeneous data processing method and device.
[0004] Based on the above objectives, the present application provides a method for processing multi-source heterogeneous data, including:
[0005] Obtain multi-source data for new power systems;
[0006] Performing noise reduction processing on the multi-source data to obtain noise-reduced multi-source data;
[0007] Performing standardization processing on the denoised multi-source data to obtain standardized multi-source data;
[0008] The standardized multi-source data is fused using a preset fusion method to obtain fused multi-source data.
[0009] Optionally, performing noise reduction processing on the multi-source data to obtain the noise-reduced multi-source data includes:
[0010] Performing wavelet transformation on any original data in the multi-source data to obtain wavelet coefficients of each layer corresponding to the original data;
[0011] According to the wavelet coefficients of each layer, determine the wavelet coefficient peak ratio corresponding to each layer;
[0012] According to the wavelet coefficient peak ratio of each layer and the data length of each layer, the peak and ratio adjustment factor corresponding to each layer is determined;
[0013] Determine the threshold value corresponding to each layer according to the peak and ratio adjustment factor of each layer, the noise estimation, the total length of the original data and the energy adjustment factor; wherein the energy adjustment factor is determined according to the energy value of each layer, and the energy value is determined according to the wavelet coefficient and the data length of the corresponding layer;
[0014] According to the threshold value of each layer, the wavelet coefficients of the corresponding layer are subjected to denoising to obtain the wavelet coefficients of each layer after the noise is removed;
[0015] Based on the wavelet coefficients of each layer after noise removal, the original data after noise reduction is reconstructed.
[0016] Optionally, the method for determining the energy adjustment factor includes:
[0017] According to the wavelet coefficients of each layer and the data length of the corresponding layer, the energy value corresponding to each layer is determined;
[0018] According to the energy value of each layer, the critical decomposition layer corresponding to the energy extreme value is determined;
[0019] For layers smaller than the critical decomposition layer, the energy adjustment factor is set to a first energy adjustment factor; for layers greater than or equal to the critical decomposition layer, the energy adjustment factor is set to a second energy adjustment factor, and the second energy adjustment factor is greater than the first energy adjustment factor.
[0020] Optionally, the threshold corresponding to each layer is determined according to the peak sum ratio adjustment factor, noise estimation, total length of original data and energy adjustment factor of each layer, and the method is:
[0021]
[0022] Among them, λ j is the threshold of the jth layer, < is the noise estimation, N is the total length of the original data, r is the energy adjustment factor, F j is the peak-to-peak ratio adjustment factor of the j-th layer.
[0023] Optionally, the value range of the first energy adjustment factor is -0.5-0.3, and the value range of the second energy adjustment factor is 0.1-0.3.
[0024] Optionally, the denoised multi-source data is subjected to standardization processing to obtain standardized multi-source data, including:
[0025] Performing Yeo-Johnson transformation on the denoised multi-source data to obtain transformed multi-source data;
[0026] Performing standard score processing on the transformed multi-source data to obtain multi-source data after standard preprocessing;
[0027] Performing magnitude unification processing on the multi-source data after the standard preprocessing to obtain multi-source data with uniform magnitude;
[0028] The multi-source data with uniform magnitude is subjected to time consistency processing to obtain multi-source data with consistent time.
[0029] Optionally, the standardized multi-source data is fused using a preset fusion method, including:
[0030] The standardized multi-source data are fused using the DS inference method.
[0031] Optionally, the acquiring of multi-source data of the novel power system includes:
[0032] According to the preset target requirements, obtain the corresponding target multi-source data;
[0033] The obtained fused multi-source data is: obtained fused target multi-source data.
[0034] Optionally, the method further includes:
[0035] Based on the fused target multi-source data, a prediction result corresponding to the target demand is determined using a preset prediction model.
[0036] The present application also provides a multi-source heterogeneous data processing device, including:
[0037] An acquisition module, used to acquire multi-source data of the new power system;
[0038] A denoising module is used to perform denoising on multi-source data to obtain denoised multi-source data;
[0039] A standardization module, used for performing standardization processing on the denoised multi-source data to obtain standardized multi-source data;
[0040] The fusion module is used to perform fusion processing on the standardized multi-source data using a preset fusion method to obtain fused multi-source data.
[0041] From the above, it can be seen that the multi-source heterogeneous data processing method and device provided in the embodiment of the present application obtains multi-source data of the new power system, performs noise reduction processing on the multi-source data to obtain the noise-reduced multi-source data, performs standardization processing on the noise-reduced multi-source data to obtain standardized multi-source data, and uses a preset fusion method to fuse the standardized multi-source data to obtain fused multi-source data. The present application can improve the accuracy and reliability of fused data and meet the different applications of new power needs. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0043] Figure 1 A schematic diagram of a method flow of an embodiment of the present application;
[0044] Figure 2 A schematic diagram of the system framework of an embodiment of the present application;
[0045] Figure 3A This is a waveform diagram of the original voltage signal sent by the sending end in one embodiment of the present application;
[0046] Figure 3B This is a waveform diagram of an original voltage signal containing noise received by a receiving end in one embodiment of the present application;
[0047] Figure 3C This is a waveform diagram of a voltage signal after noise reduction according to an embodiment of the present application;
[0048] Figure 4A A schematic diagram of data distribution of original load data according to an embodiment of the present application;
[0049] Figure 4B A data distribution diagram of load data after being processed for standard scores of related technologies;
[0050] Figure 4C This is a data distribution diagram of load data after standard score processing according to an embodiment of the present application;
[0051] Figure 5 This is a block diagram of the device structure of an embodiment of the present application;
[0052] Figure 6 This is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION
[0053] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.
[0054] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connecting" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to represent relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0055] As mentioned in the background technology section, the data of the new power system comes from various types of equipment and systems, and has the characteristics of multiple sources, heterogeneity, and complexity. The unified processing and fusion of data are difficult. In particular, multi-source data may have noise, inconsistency, and missing data. Through noise reduction processing and data fusion, multi-source data can be integrated and calibrated to improve the accuracy and completeness of the data. The constant threshold noise reduction processing method in the related art cannot adaptively adjust the threshold according to the characteristics of the data, so the noise reduction effect is not ideal, and the accuracy of the processed data is not high.
[0056] In view of this, an embodiment of the present application provides a multi-source heterogeneous data processing method, which performs adaptive noise reduction processing on the multi-source data of the new power system, can improve data accuracy, provide high-quality data for data fusion, standardize the noise-reduced multi-source data, and then fuse the standardized multi-source data to obtain fused multi-source data. The new power system can perform subsequent processing such as predictive analysis based on the fused multi-source data to meet the different application requirements of the system.
[0057] The technical solution of the present application is further described in detail below through specific embodiments.
[0058] like Figure 1 As shown, the multi-source heterogeneous data processing method provided in the embodiment of the present application includes:
[0059] S101: Acquire multi-source data of new power systems;
[0060] Combination Figure 2As shown, in this embodiment, the new power system is a power system supported by the interaction of source, grid, load and storage and multi-energy complementarity. The data of the new power system includes real-time monitoring data from various sensors, load data from smart meters and power consumption monitoring equipment, historical power consumption data and historical fault data from the monitoring platform, price data from the management platform, etc. Among them, the real-time monitoring data includes temperature signals collected by temperature sensors, current signals collected by current sensors, voltage signals collected by voltage sensors, air quality signals collected by air quality sensors, circuit breaker status signals monitored by circuit breaker status sensors, transformer status signals monitored by transformer sensors, transmission line status signals monitored by transmission line monitoring sensors, vehicle-related signals collected by various sensors in the vehicle battery management system, etc. The system structure, working principle and specific data types of the new power system are not described in detail.
[0061] In some embodiments, a unified data fusion process is performed on multi-source data from different sources in the new power system, so that the uniformly processed multi-source data is suitable for the application of the new power system. In order to adapt to the different application requirements of the new power system, the multi-source data of the new power system is obtained, including: according to the preset target requirements, the corresponding target multi-source data is obtained. That is, according to the actual needs of the new power system, the target multi-source data that can meet the needs is determined, and the target multi-source data is uniformly fused, so that the fused target multi-source data can meet the corresponding needs.
[0062] For example, in order to realize power load forecasting of new power systems, multi-source data including load data, historical power consumption data and environmental data (for example, temperature signals, air quality signals, etc.) are obtained; in order to realize equipment health monitoring and fault prediction of new power systems, multi-source data including real-time equipment status data (for example, circuit breaker status signals, transmission line status signals, etc.) and historical fault data are obtained; in order to realize power market analysis and forecasting of new power systems, multi-source data including load data, environmental data and price data are obtained; in order to realize electric vehicle charging management of new power systems, multi-source data including load data, vehicle data and environmental data are obtained, etc.
[0063] In some embodiments, target multi-source data is acquired according to target requirements, and after the target multi-source data is processed to obtain fused target multi-source data, a prediction result corresponding to the target requirement is determined based on the fused target multi-source data and a preset prediction model. That is, by fusing data from different sources, more comprehensive and accurate fused data can be obtained, and then accurate prediction and analysis can be performed based on the fused data to achieve refined management and control. For example, based on the fused target multi-source data, a preset power load prediction model is used to predict the power load status for a certain period of time in the future; a preset equipment health monitoring and fault prediction model is used to predict the health status of the equipment, etc.
[0064] S102: performing noise reduction processing on the multi-source data to obtain noise-reduced multi-source data;
[0065] In this embodiment, since the original multi-source data directly obtained from various devices, systems, etc. has noise, it is necessary to first perform noise reduction processing on the multi-source data to improve data accuracy.
[0066] In some embodiments, performing noise reduction processing on multi-source data to obtain noise-reduced multi-source data includes:
[0067] Perform wavelet transform on any original data in the multi-source data to obtain the wavelet coefficients of each layer corresponding to the original data;
[0068] According to the wavelet coefficients of each layer, determine the wavelet coefficient peak ratio corresponding to each layer;
[0069] According to the wavelet coefficient peak ratio of each layer and the data length of each layer, the peak and ratio adjustment factor corresponding to each layer is determined;
[0070] Determine the threshold value corresponding to each layer according to the peak and ratio adjustment factor of each layer, the noise estimation, the total length of the original data and the energy adjustment factor; wherein the energy adjustment factor is determined according to the energy value of each layer, and the energy value is determined according to the wavelet coefficient and the data length of the corresponding layer;
[0071] According to the threshold value of each layer, the wavelet coefficients of the corresponding layer are subjected to denoising to obtain the wavelet coefficients of each layer after the noise is removed;
[0072] Based on the wavelet coefficients of each layer after noise removal, the original data after noise reduction is reconstructed.
[0073] This embodiment provides an adaptive noise reduction processing method, which performs wavelet transform on the original data, decomposes the original data into wavelet coefficients corresponding to each layer of data, determines the corresponding wavelet coefficient peak ratio for each layer of wavelet coefficients, and determines the peak and ratio adjustment factor of each layer according to the wavelet coefficient peak ratio of each layer and the data length of the corresponding layer. Since the peak and ratio adjustment factor is an unadjustable fixed threshold, noise reduction is performed according to this fixed threshold, which affects the noise reduction effect. Therefore, according to the peak and ratio adjustment factor of each layer, the noise estimation and the energy adjustment factor of the corresponding layer, the threshold corresponding to each layer is determined, and then the wavelet coefficients of the corresponding layer are subjected to noise reduction processing according to the threshold of each layer, and the original signal after noise reduction is obtained based on the wavelet coefficients of each layer after noise reduction. Since the adaptive noise reduction processing method is adopted, the corresponding threshold is determined according to the characteristics of each layer of data, and the noise is removed according to the corresponding threshold, which can improve the noise reduction effect, maximize the retention of useful signals, remove noise signals, and improve the accuracy of the data after noise reduction.
[0074] In some embodiments, the method of determining the energy adjustment factor includes:
[0075] According to the wavelet coefficients of each layer and the data length of the corresponding layer, the energy value corresponding to each layer is determined;
[0076] According to the energy value of each layer, the critical decomposition layer corresponding to the energy extreme value is determined;
[0077] For layers smaller than the critical decomposition layer, the energy adjustment factor is set to the first energy adjustment factor. For layers greater than or equal to the critical decomposition layer, the energy adjustment factor is set to the second energy adjustment factor, and the second energy adjustment factor is greater than the first energy adjustment factor.
[0078] In some implementations, any one of the original data in the multi-source data is s(t), and the original data includes a useful signal x(t) and a noise signal n(t), which is expressed as:
[0079] s(t)=x(t)+n(t) (1) The original data is processed by discrete wavelet transform to obtain:
[0080] d j,k =∫s(t)ψ(t)dt=∫x(t)ψ(t)dt+∫n(t)ψ(t)dt (2)
[0081] Among them, ψ(t) is the db4 wavelet basis function; d j,k is the wavelet coefficient corresponding to the j-th layer of data, k is the position of the wavelet coefficient in the j-th layer of decomposition, the characteristics of the wavelet coefficients of the useful signal and the noise signal are different, depending on the number of wavelet decomposition layers. Gaussian white noise is mainly concentrated in the wavelet coefficients with smaller modulus. As the number of decomposition layers increases, the value of the wavelet coefficient corresponding to the noise signal continues to decrease.
[0082] According to the wavelet coefficients d of each layer j,k , determine the peak ratio of wavelet coefficients corresponding to each layer, expressed as:
[0083]
[0084] Among them, L j is the data length of the j-th layer data, PSR j is the wavelet coefficient peak ratio of the jth layer. The wavelet coefficient peak ratio is used to characterize the ratio of the useful signal to the noise signal of the jth layer, PSR j The larger the value of indicates, the larger the proportion of wavelet coefficients with larger moduli in the jth layer, and the larger the proportion of useful signals.
[0085] According to the peak ratio of wavelet coefficients of each layer and the data length of each layer, the peak and ratio adjustment factor corresponding to each layer is calculated as follows:
[0086]
[0087] Considering that the threshold adjustment based on the peak and ratio adjustment factor is relatively fixed and cannot be flexibly adjusted according to the data characteristics, the noise reduction effect is limited. In order to improve the accuracy of threshold estimation, this embodiment provides an adaptive noise reduction method, which combines the energy characteristics of each layer of data to determine the critical decomposition layer, and sets the energy adjustment factors of different layers according to the critical decomposition layer.
[0088] In some methods, the energy value corresponding to each layer is determined according to the wavelet coefficient of each layer and the data length of the corresponding layer. The calculation method is:
[0089]
[0090] Among them, E j is the energy value of the j-th layer data.
[0091] After calculating the energy values of each layer according to formula (5), the layer corresponding to the energy extreme value is determined and this layer is used as the critical decomposition layer j. 0 , for layers smaller than the critical decomposition layer (i.e., j < j 0 ), the noise signal dominates and decreases with the increase of the number of decomposition layers, and the corresponding energy also decreases. For layers greater than the critical decomposition, the useful signal dominates.
[0092] The energy adjustment factor r is set at the critical decomposition layer. The threshold of each layer is determined according to the peak and ratio adjustment factor, noise estimation, total length of original data and corresponding energy adjustment factor of each layer. The threshold calculation method is:
[0093]
[0094] Among them, for layers smaller than the critical decomposition layer, the energy adjustment factor is set to the first energy adjustment factor, and for layers greater than or equal to the critical decomposition layer, the energy adjustment factor is set to the second energy adjustment factor, and the second energy adjustment factor is greater than the first energy adjustment factor; that is, for layers smaller than the critical decomposition layer, a smaller energy adjustment factor is set, and the resulting threshold is larger, and for layers greater than the critical decomposition layer, a larger energy adjustment factor is set, and the resulting threshold is smaller.
[0095] σ is the noise standard deviation, which is used to estimate the noise as a whole, expressed as:
[0096]
[0097] In some embodiments, the value range of the first energy adjustment factor is -0.5-0.3, and the value range of the second energy adjustment factor is 0.1-0.3. The specific values can be adjusted experimentally according to the characteristics of the data to achieve the best noise reduction effect. Optionally, the value of the first energy adjustment factor is -0.3, and the value of the second energy adjustment factor is 0.2.
[0098] After determining the threshold of each layer, the wavelet coefficients of the corresponding layer are denoised according to the threshold of each layer. The method is:
[0099]
[0100] Among them, d j ' ,k is the wavelet coefficient of the jth layer after noise removal.
[0101] After removing the noise from the wavelet coefficients of each layer based on the energy adjustment factor of each layer, the original data after noise reduction is reconstructed based on the wavelet coefficients of each layer after noise removal. The method is:
[0102]
[0103] like Figures 3A-3C As shown, taking the voltage signal collected in real-time operation of the distribution network as an example, the voltage signal contains harmonic disturbances and noise. Using a constant threshold for noise reduction processing is likely to treat the harmonic disturbance as noise and mistakenly remove the useful signal. The adaptive noise reduction method of the present application can not only retain the useful harmonic disturbance characteristics, but also filter out most of the noise signals, thereby improving the noise reduction effect.
[0104] S103: performing standardization processing on the denoised multi-source data to obtain standardized multi-source data;
[0105] In this embodiment, since various data from different sources have different characteristic attributes such as form, level, type and magnitude, it is necessary to solve the problem of inconsistent characteristic attributes of multi-source data through standardization processing.
[0106] In some methods, after the multi-source data is subjected to denoising, the denoised multi-source data is subjected to uniform standardization, including:
[0107] Performing Yeo-Johnson transformation on the denoised multi-source data to obtain transformed multi-source data;
[0108] Performing standard score processing on the transformed multi-source data to obtain multi-source data after standard preprocessing;
[0109] Performing level-unification processing on the multi-source data after standard preprocessing to obtain multi-source data with uniform level;
[0110] Multi-source data with uniform magnitude are processed for time consistency to obtain multi-source data with consistent time.
[0111] In this embodiment, for the multi-source data after denoising, the Yeo-Johnson transformation method is first used to transform the data, which is used to reduce the heteroscedasticity of the multi-source data and amplify its normality, so that the shape of the probability density function is close to the normal distribution, and the accuracy and stability of the multi-source data standardization process are improved; the transformed multi-source data is processed by standard scores to obtain the multi-source data after standardized preprocessing, and then the multi-source data after standardized preprocessing is unified in magnitude and processed in time consistency, and finally the standardized multi-source data is obtained.
[0112] In some embodiments, the method of performing Yeo-Johnson transformation on the denoised multi-source data is:
[0113]
[0114] Where X is the multi-source data after noise reduction, and α is a transformation parameter used to adjust the nonlinearity of the transformation. In practical applications, a statistical library is used to automatically determine the value of α. Optionally, the power transformer PowerTransformer in Python's Scikit-learn library is used to determine the optimal α value.
[0115] For multi-source data after Yeo-Johnson transformation, standard score processing is used, that is, Z-score data transformation processing is performed, the method is:
[0116]
[0117] Among them, YJ_Z is the data point after Yeo-Johnson transformation, μ is the mean of the data after Yeo-Johnson transformation, and σ is the standard deviation of the data after Yeo-Johnson transformation.
[0118] like Figures 4A-4C As shown, the data in this application that is first processed by Yeo-Johnson transformation and then processed by standard score is closer to normal distribution than the data that is directly processed by standard score without Yeo-Johnson transformation, which can improve the standardization effect.
[0119] In some embodiments, since the magnitudes of data from different sources may be different, the magnitudes of multi-source data after standardization preprocessing need to be unified, for example, all voltage data are uniformly converted to volt magnitudes, all current data are uniformly converted to ampere magnitudes, etc.
[0120] In some embodiments, the acquired multi-source data is multi-source data within the same time period, for example, multi-source data within ten minutes, multi-source data within half an hour, considering that the time points of data collection from different sources are different, that is, the time points of data from different sources are not completely consistent. Therefore, after the multi-source data within the same time period is processed in a unified manner, it is necessary to perform time consistency processing, align the time of the multi-source data through time interpolation, and the data of the inserted time point can be determined based on the data of two adjacent time points or determined by fitting the data of multiple time points. In this way, the multi-source data after time consistency processing has the same time granularity.
[0121] S104: Using a preset fusion method to perform fusion processing on the standardized multi-source data to obtain fused multi-source data.
[0122] In this embodiment, after the multi-source data is uniformly standardized, the conflict between data from different sources needs to be considered. To resolve the data conflict, the standardized multi-source data is fused using the DS reasoning method to obtain fused multi-source data.
[0123] In some methods, if the i-th data after normalization at time t is The trust function constraints are:
[0124]
[0125] in, for degree of trust.
[0126] Determine the initial weight of the i-th data after standardization at time t according to the exponentially weighted moving average method Calculate the new weight in real time and regard the weight as the corresponding trust, expressed as:
[0127]
[0128] Among them, β is the smoothing factor.
[0129] The DS inference method is used to fuse the trust function of the standardized multi-source data. The fusion rule is:
[0130]
[0131] in, Indicates data conflict items, m t That is, multi-source data that integrates the trust information of different data sources.
[0132] The multi-source heterogeneous data processing method provided in the embodiment of the present application uses an adaptive noise reduction processing method to perform noise reduction on the multi-source data of the new power system, which can not only effectively retain useful signals, but also filter out most of the noise, thereby effectively improving the accuracy of the data; the denoised multi-source data is first subjected to a Yeo-Johnson transformation to improve the non-normality problem of the data, and then subjected to standard score processing to improve the accuracy of standardization; for the standardized multi-source data, the conflict problem that occurs in the data fusion process of the DS inference algorithm is optimized based on the exponentially weighted moving average method to ensure the effective fusion of data sources from different sources, which can improve the accuracy and reliability of the fused data. On this basis, the new power system can perform subsequent prediction, analysis and other processing based on the fused multi-source data to meet different application requirements.
[0133] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.
[0134] It should be noted that the above is a description of a specific embodiment of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0135] like Figure 5As shown, the embodiment of the present application also provides a multi-source heterogeneous data processing device, including:
[0136] An acquisition module, used to acquire multi-source data of the new power system;
[0137] A denoising module is used to perform denoising on multi-source data to obtain denoised multi-source data;
[0138] A standardization module, used for performing standardization processing on the denoised multi-source data to obtain standardized multi-source data;
[0139] The fusion module is used to perform fusion processing on the standardized multi-source data using a preset fusion method to obtain fused multi-source data.
[0140] For the convenience of description, the above devices are described in terms of functions divided into various modules. Of course, when implementing the embodiments of the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.
[0141] The apparatus of the above-mentioned embodiment is used to implement the corresponding method in the above-mentioned embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.
[0142] Figure 6 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.
[0143] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.
[0144] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.
[0145] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.
[0146] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).
[0147] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).
[0148] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.
[0149] The electronic device of the above embodiment is used to implement the corresponding method in the above embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.
[0150] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.
[0151] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.
[0152] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present disclosure, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered illustrative rather than restrictive.
[0153] Although the present disclosure has been described in conjunction with specific embodiments of the present disclosure, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.
[0154] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present disclosure.
Claims
1. A method for processing multi-source heterogeneous data, characterized in that: include: Obtain multi-source data for new power systems; Performing noise reduction processing on the multi-source data to obtain noise-reduced multi-source data; Performing standardization processing on the denoised multi-source data to obtain standardized multi-source data; The standardized multi-source data is fused using a preset fusion method to obtain fused multi-source data.
2. The method according to claim 1, characterized in that The denoising process is performed on the multi-source data to obtain the denoised multi-source data, including: Performing wavelet transformation on any original data in the multi-source data to obtain wavelet coefficients of each layer corresponding to the original data; According to the wavelet coefficients of each layer, determine the wavelet coefficient peak ratio corresponding to each layer; According to the wavelet coefficient peak ratio of each layer and the data length of each layer, the peak and ratio adjustment factor corresponding to each layer is determined; Determine the threshold value corresponding to each layer according to the peak and ratio adjustment factor of each layer, the noise estimation, the total length of the original data and the energy adjustment factor; wherein the energy adjustment factor is determined according to the energy value of each layer, and the energy value is determined according to the wavelet coefficient and the data length of the corresponding layer; According to the threshold value of each layer, the wavelet coefficients of the corresponding layer are subjected to denoising to obtain the wavelet coefficients of each layer after the noise is removed; Based on the wavelet coefficients of each layer after noise removal, the original data after noise reduction is reconstructed.
3. The method according to claim 2, characterized in that The method for determining the energy regulation factor comprises: According to the wavelet coefficients of each layer and the data length of the corresponding layer, the energy value corresponding to each layer is determined; According to the energy value of each layer, the critical decomposition layer corresponding to the energy extreme value is determined; For layers smaller than the critical decomposition layer, the energy adjustment factor is set to a first energy adjustment factor; for layers greater than or equal to the critical decomposition layer, the energy adjustment factor is set to a second energy adjustment factor, and the second energy adjustment factor is greater than the first energy adjustment factor.
4. The method according to claim 3, characterized in that According to the peak and ratio adjustment factor of each layer, the noise estimation, the total length of the original data and the energy adjustment factor, the threshold corresponding to each layer is determined as follows: Among them, λ j is the threshold of the jth layer, σ is the noise estimation, N is the total length of the original data, r is the energy adjustment factor, F j is the peak-to-peak ratio adjustment factor of the j-th layer.
5. The method according to claim 3 or 4, characterized in that: The value range of the first energy adjustment factor is -0.5-0.3, and the value range of the second energy adjustment factor is 0.1-0.
3.
6. The method according to claim 1, characterized in that The denoised multi-source data is subjected to standardization processing to obtain standardized multi-source data, including: Performing Yeo-Johnson transformation on the denoised multi-source data to obtain transformed multi-source data; Performing standard score processing on the transformed multi-source data to obtain multi-source data after standard preprocessing; Performing magnitude unification processing on the multi-source data after the standard preprocessing to obtain multi-source data with uniform magnitude; The multi-source data with uniform magnitude is subjected to time consistency processing to obtain multi-source data with consistent time.
7. The method according to claim 1, characterized in that The standardized multi-source data is fused using a preset fusion method, including: The standardized multi-source data are fused using the DS inference method.
8. The method according to claim 1, characterized in that The method of acquiring multi-source data of the new power system includes: According to the preset target requirements, obtain the corresponding target multi-source data; The obtained fused multi-source data is: obtained fused target multi-source data.
9. The method according to claim 8, characterized in that Also includes: Based on the fused target multi-source data, a prediction result corresponding to the target demand is determined using a preset prediction model.
10. A multi-source heterogeneous data processing device, characterized in that: include: An acquisition module, used to acquire multi-source data of the new power system; A denoising module is used to perform denoising on multi-source data to obtain denoised multi-source data; A standardization module, used for performing standardization processing on the denoised multi-source data to obtain standardized multi-source data; The fusion module is used to perform fusion processing on the standardized multi-source data using a preset fusion method to obtain fused multi-source data.
Citation Information
Cited By
Intelligent control medium-voltage power distribution method and system
CN119602487A
An intelligent control method and system for medium-voltage power distribution
CN119602487B
