Data processing method and device, electronic equipment, storage medium and program product

Abstract

The invention provides a data processing method and apparatus, an electronic device, a storage medium and a program product. The method comprises the steps of obtaining difference sequence data of a device; the difference sequence data is determined based on a difference value sequence of the normal sequence data and the abnormal sequence data; processing the difference sequence data by using a pre-trained agent to obtain a cleaning result obtained by the agent based on the target cleaning action; wherein; the intelligent agent is trained based on various difference sequence samples; the difference sequence sample is virtual twin data. In conclusion, the technical scheme provided by the invention has wide adaptability, and meanwhile, the data cleaning efficiency and precision can be enhanced.

Description

Technical Field

[0001] The present disclosure relates to the field of communication technologies, and in particular to a data processing method, device, electronic device, storage medium, and program product. Background Art

[0002] At present, when cleaning a large amount of data of different types, models, and time periods in complex scenarios such as intelligent operation and maintenance, remote decision-making, and digital twins, there are problems such as insufficient basis and loss of effective information in the field of time series cleaning. This will result in the guiding "addition" and "deletion" of the original time series generated in real equipment. Cleaning actions such as are not accurate enough, have limitations, and are inefficient. Summary of the Invention

[0003] The present disclosure provides a data processing method, apparatus, electronic device, storage medium and program product, which are used to, to a certain extent, solve the problems of insufficient basis and loss of effective information in the field of time series cleaning when performing data cleaning. This will result in the cleaning actions such as guided "addition" and "deletion" of the original time series generated in real devices being inaccurate, limited and inefficient.

[0004] According to one aspect of the present disclosure, a data processing method is provided, which includes: obtaining difference sequence data of a device; determining the difference sequence data based on a difference sequence between normal sequence data and abnormal sequence data; processing the difference sequence data using a pre-trained intelligent agent to obtain a cleaning result obtained by the intelligent agent based on a target cleaning action; wherein; the intelligent agent is trained based on multiple difference sequence samples; and the difference sequence samples are virtual twin data.

[0005] In addition, according to a method in one aspect of the present disclosure, the method also includes: for any device, performing isomorphic processing on the original sequence data of the device to obtain isomorphic sequence data; based on the isomorphic sequence data and at least one simulator, obtaining the first simulation parameters when the results are the same; inputting the first simulation parameters into the second model to obtain a virtual twin; the second model is a three-dimensional modeling model; based on the virtual twin, adjusting the first simulation parameters to obtain normal virtual twin data and multiple abnormal virtual twin data; based on the normal virtual twin data and multiple abnormal virtual twin data, determining multiple difference sequence samples; and using multiple difference sequence samples to train the intelligent agent.

[0006] In addition, according to a method of one aspect of the present disclosure, an intelligent agent is trained using multiple difference sequence samples, including: determining a first cleaning action for any difference sequence sample; determining a reward function value corresponding to the difference sequence sample based on the first cleaning action; updating the reward function value based on the correlation between the reward function value and the timing error, and updating the first cleaning action until the reward function value reaches a maximum value, and determining the first cleaning action corresponding to the maximum value of the reward function as the target cleaning action of the difference sequence sample.

[0007] In addition, according to a method according to one aspect of the present disclosure, the reward function value is determined by weighting the spectral density, kurtosis, and margin of the new sequence data after the first cleaning action and the original sequence data.

[0008] In addition, according to a method of one aspect of the present disclosure, based on isomorphic sequence data and at least one simulator, obtaining the first simulation parameters when the results are the same includes: inputting the isomorphic sequence data into at least one simulator to obtain at least one simulation result; for any simulation result, when the simulation result does not meet the first threshold, updating the second simulation parameters corresponding to the simulation result until the simulation result meets the first threshold; when all simulation results meet the first threshold, obtaining the timing waveform characteristics of all simulation results; when all timing waveform characteristics meet the second threshold, determining that all simulation results are the same; obtaining the first simulation parameters corresponding to the same results.

[0009] In addition, according to one aspect of the method of the present disclosure, the method also includes: when any timing waveform feature does not meet the second threshold, determining that there are differences in simulation results; adjusting the timestamps or speeds of simulators with different simulation results until all simulation results are the same.

[0010] In addition, according to a method according to one aspect of the present disclosure, the timing waveform feature includes at least one of the following: root mean square, peak-to-peak value, kurtosis, and margin.

[0011] In addition, according to a method in one aspect of the present disclosure, when the simulator simulates the temperature of the motor, the first simulation parameter includes at least one of the following: density, stiffness, moment of inertia, wear, damping, and heat transfer coefficient.

[0012] In addition, according to a method in one aspect of the present disclosure, the isomorphism processing includes: type isomorphism and time sequence isomorphism.

[0013] In addition, according to a method in one aspect of the present disclosure, for any device, the original sequence data of the device is subjected to isomorphic processing to obtain isomorphic sequence data, including: obtaining historical processing data of the device; and using the ratio relationship between the historical processing data and the original sequence data to correct the original sequence data to obtain isomorphic sequence data.

[0014] In addition, according to a method in one aspect of the present disclosure, when the isomorphism processing is type isomorphism, the historical processing data includes at least one of the following: amplitude effective value, amplitude peak, amplitude variance, prediction length and constraint complexity; when the isomorphism processing is time series isomorphism, the historical processing data includes at least one of the following: protocol bandwidth, processing delay, data frame length, test calculation delay.

[0015] In addition, according to a method in one aspect of the present disclosure, obtaining difference sequence data of a device includes: monitoring and collecting normal sequence data and abnormal sequence data of the device; obtaining a difference sequence between the normal sequence data and the abnormal sequence data; performing comprehensive processing on the difference sequence to obtain difference sequence data; the comprehensive processing includes at least one of the following: waveform quantization processing, peak quantization processing, pulse quantization processing, root mean square processing, peak-to-peak processing, and variance processing.

[0016] In addition, according to the method of one aspect of the present disclosure, the cleaning action includes at least one of the following: a filter type, a filter function.

[0017] According to another aspect of the present disclosure, a data processing device is provided, which includes: an acquisition unit for acquiring difference sequence data of a device; the difference sequence data is determined based on a difference sequence between normal sequence data and abnormal sequence data; a processing unit for processing the difference sequence data using a pre-trained intelligent agent to obtain a cleaning result obtained by the intelligent agent based on a target cleaning action; wherein; the intelligent agent is trained based on multiple difference sequence samples; and the difference sequence samples are virtual twin data.

[0018] According to yet another aspect of the present disclosure, an electronic device is provided, comprising: a memory for storing computer-readable instructions; and a processor for executing the computer-readable instructions so that the electronic device executes the method according to any one embodiment of the one aspect.

[0019] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium is provided for storing computer-readable instructions. When the computer-readable instructions are executed by a processor, the processor is caused to execute the method as described in any embodiment of one aspect.

[0020] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the method according to any one embodiment of the first aspect is implemented.

[0021] The present disclosure provides a data processing method, apparatus, electronic device, storage medium, and program product. The present disclosure obtains differential sequence data from a device; the differential sequence data is determined based on a sequence of differences between normal sequence data and abnormal sequence data; and a pre-trained intelligent agent processes the differential sequence data to obtain a cleaning result based on a target cleaning action. The intelligent agent is trained based on multiple differential sequence samples, and the differential sequence samples are virtual twin data. This method constructs a digital twin that fully corresponds to the physical device, maps the device's operating status in real time, and generates a large number of virtual differential sequence samples, thus overcoming the traditional data cleaning reliance on real abnormal data. By training the intelligent agent with a large number of differential sequence samples, any abnormality in the real device can be directly input into the intelligent agent to obtain the target cleaning action. Furthermore, compared to abnormal or normal data, differential sequence data can better highlight the changing trends of the device's operating status, the subtle differences in abnormal characteristics, and the transition between normal and abnormal states. This allows the intelligent agent to more deeply understand the type and severity of different abnormal data, and thus develop more targeted and accurate cleaning strategies based on specific differential sequence data, further enhancing the accuracy of cleaning action determination. Compared with existing manual labeling or manual testing methods, the method disclosed in the present invention has wide adaptability and can enhance the efficiency and accuracy of data cleaning.

[0022] It is to be understood that both the foregoing general description and the following detailed description are exemplary, and are intended to provide further explanation of the technology as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above and other purposes, features, and advantages of the present disclosure will become more apparent through a more detailed description of the embodiments of the present disclosure in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and are not intended to limit the present disclosure. In the drawings, the same reference numerals generally represent the same components or steps.

[0024] Figure 1 A schematic diagram of a data processing system provided in accordance with an embodiment of the present disclosure;

[0025] Figure 2 A flowchart of a data processing method provided in an embodiment of the present disclosure;

[0026] Figure 3 A schematic diagram of a process for establishing a digital twin of a motor through temperature simulation according to an embodiment of the present disclosure;

[0027] Figure 4 A structural block diagram of a data processing device provided in an embodiment of the present disclosure;

[0028] Figure 5 A hardware block diagram of an electronic device provided in an embodiment of the present disclosure;

[0029] Figure 6 A schematic diagram of a computer-readable storage medium provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the present disclosure more apparent, the following will describe in detail exemplary embodiments of the present disclosure with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0031] At present, when cleaning a large amount of data of different types, models, and time periods in complex scenarios such as intelligent operation and maintenance, remote decision-making, and digital twins, there are problems such as insufficient basis and loss of effective information in the field of time series cleaning. Due to the lack of sufficient samples, the guiding "supplementation" and "deletion" of the original time series generated in the real equipment will not be accurate enough. The anomaly removal and feature optimization of real equipment mostly rely on manual labeling or manual testing, which has limitations and low efficiency and accuracy. Therefore, in response to the above-mentioned problems, this application provides a data processing method that uses digital twin data for autonomous model training, which can improve the efficiency and accuracy of data cleaning and has wide adaptability. First, please refer to Figure 1 , Figure 1 Schematic diagram of a data processing system provided by an embodiment of the present disclosure. Figure 1 It can be seen that the heterogeneous cluster data cleaning system mainly includes software layer, hardware layer, digital twin cluster, and reinforcement learning model.

[0032] The software layer can be understood as the component responsible for data processing logic and algorithm management within a heterogeneous cluster data cleaning system. The software layer can address at least one of the following issues, including but not limited to: alignment of read and write formats, time state alignment, and calibration parameter alignment across multiple devices. Specifically, the software layer can include a sample case library, a device parameter library, a scenario case library, a device model library, an algorithm model library, and a node data distribution strategy. These components collaborate with each other. Specifically, the sample case library provides successful historical data processing examples, the device parameter library records detailed device parameters, the scenario case library provides customized data processing solutions for different application scenarios, the device model library constructs virtual models of devices to assist in analyzing device data characteristics, and the algorithm model library integrates multiple data processing algorithms to provide technical support for data cleaning. The node data distribution strategy is responsible for rationally allocating data processing tasks to each node, improving data processing efficiency.

[0033] The hardware layer can be understood as the basic support part responsible for the underlying data transmission and physical device interaction in the heterogeneous cluster data cleaning system. The hardware layer can solve at least one of the following problems, including but not limited to: data alignment of multiple device communication protocols, and sampling alignment of multiple device communication protocols. Specifically, the hardware layer may include: a protocol case library, a terminal performance library, a data frame case library, a delay case library, and a node data fusion interface. Specifically, the protocol case library can store relevant information of different device communication protocols; the terminal performance library can record the performance indicators of the terminal device; the data frame case library is used to standardize and process the data frame format during data transmission; the delay case library can analyze data transmission delay problems and optimize the data transmission process; the node data fusion interface can realize data integration and interaction between different nodes to ensure data consistency and integrity.

[0034] A digital twin cluster can be understood as a virtual mapping entity built from data at the software and hardware layers. It deeply integrates virtual models, case libraries, and algorithmic models of devices in the software layer with communication protocols, performance metrics, and other data from the hardware layer. A digital twin cluster can accurately simulate the operating status of physical devices and systems.

[0035] Reinforcement learning models are intelligent models that can autonomously learn and optimize based on environmental feedback. In heterogeneous cluster data cleaning systems, reinforcement learning models are based on the virtual environment simulated by digital twin clusters. By continuously interacting with the virtual environment, they try different data cleaning strategies and operations, and evaluate the effectiveness of these strategies and operations based on reward signals from the environment. This will be described in detail later in conjunction with examples.

[0036] This disclosure provides a data processing method. Figure 2 , Figure 2 A flow chart of a data processing method provided by an embodiment of the present disclosure. Figure 2 As shown, the method includes:

[0037] In step S201, the difference sequence data of the device is obtained; the difference sequence data is determined based on the difference sequence of the normal sequence data and the abnormal sequence data.

[0038] In the disclosed embodiments, the difference sequence data can be understood as a data set that can accurately reflect the characteristics of the changes in the operating state of the equipment. It can be obtained by subtracting the abnormal sequence data generated when an abnormal condition occurs from the normal sequence data generated when the equipment is in normal operation. The normal sequence data can be understood as a data sequence composed of various operating parameters of the equipment under stable, fault-free standard working conditions, arranged in time or a specific logical order. The abnormal sequence data can be understood as a sequence composed of various types of data collected at the corresponding time or in the same logical order when the equipment is in abnormal conditions such as failure, performance degradation, and parameter deviation from the normal range.

[0039] Specifically, differential sequence data can be obtained by utilizing monitoring equipment or systems to collect equipment operating data in real time. Specifically, during normal equipment operation, normal sequence data is comprehensively collected and organized according to at least one of the established sampling frequency and data format. When an equipment anomaly occurs, abnormal sequence data is acquired using the same collection specifications. Subsequently, a specially designed algorithm is used to calculate the point-by-point difference between the abnormal sequence data and the corresponding values of the normal sequence data, thereby generating a differential sequence. This differential sequence is then subjected to necessary preprocessing operations such as screening, denoising, and normalization to ultimately determine the differential sequence data.

[0040] In step S202, the difference sequence data is processed using a pre-trained intelligent agent to obtain a cleaning result obtained by the intelligent agent based on the target cleaning action; wherein; the intelligent agent is trained based on a variety of difference sequence samples; the difference sequence samples are virtual twin data.

[0041] In the present disclosure, the intelligent agent can be understood as an intelligent algorithm model with autonomous decision-making and learning capabilities. It can process and automatically generate and execute corresponding target cleaning actions based on the input differential sequence data to achieve efficient cleaning and optimization of the data. It should be noted that the intelligent agent of the present disclosure is trained with a large number of rich and diverse differential sequence samples. These differential sequence samples are derived from virtual twin data, that is, by constructing a virtual model that is highly similar to the actual equipment operating environment, simulating the operating status of the equipment under different working conditions and different fault scenarios, thereby obtaining a large amount of virtual abnormal sequence data and generating differential sequence samples. During the training process, the intelligent agent can select a cleaning action, determine the reward value, and change the cleaning action by continuously adjusting the reward value until the cleaning result meets the standard. In this way, after repeated training and optimization, the intelligent agent gradually has the ability to accurately and efficiently clean various complex differential sequence data, and can quickly and accurately process the input differential sequence data in practical applications and output high-quality cleaning results.

[0042] The following specifically describes how the present disclosure trains an intelligent agent. The method includes:

[0043] For any device, the original sequence data of the device is homogenized to obtain homogeneous sequence data;

[0044] Based on the isomorphic sequence data and at least one simulator, obtaining a first simulation parameter when the results are the same;

[0045] Inputting the first simulation parameters into the second model to obtain a virtual twin; the second model is a three-dimensional model;

[0046] Based on the virtual twin, adjusting the first simulation parameter to obtain normal virtual twin data and a plurality of abnormal virtual twin data;

[0047] Based on normal virtual twin data and multiple abnormal virtual twin data, multiple difference sequence samples are determined;

[0048] Train the agent using a variety of difference sequence samples.

[0049] In the present disclosure, first, the training agent can perform isomorphic processing on the original sequence data to obtain isomorphic sequence data. Among them, the original sequence data can be understood as a series of real, unprocessed data generated by the device during actual operation, which may contain different types and different time series. Isomorphic sequence data can be understood as data that has been isomorphically processed and has consistency in data type and data time series. Isomorphic processing can make the original sequence data reach a unified standard in type and time series, eliminate the influence caused by at least one factor such as device differences and different data collection methods, thereby improving the comparability and usability of the data, and providing a better data foundation for subsequent simulation and modeling work.

[0050] Then, the homogeneous sequence data processed by the device can be input into at least one simulator to obtain at least one simulation result. When the results are the same, the first simulation parameter is determined. Among them, the first simulation parameter can be understood as the parameter corresponding to when the simulator simulates the homogeneous sequence data so that different simulators obtain the same simulation results. The first simulation parameter can specifically include data such as the operating status and environmental parameters of the device, and the specific results and embodiments are described in detail later. Obtaining the first simulation parameter provides an accurate data basis for the subsequent construction of a virtual twin that is highly matched with the actual device, so that the virtual twin can more realistically reflect the operating status of the device.

[0051] Then, the first simulation parameters with the same results are input into the second model to obtain a virtual twin of the device. The second model can be understood as a three-dimensional modeling model, which can provide detailed static physical state and accurate three-dimensional structural information of the device. The second model can specifically include but is not limited to at least one of the following: geometric modeling, physical property modeling, etc. In this way, combined with the dynamic parameters of the first simulation parameters, a virtual twin can be constructed that is highly similar to the actual device in structure and behavior. This virtual twin can not only intuitively display the appearance and internal structure of the device, but also simulate the operating status of the device under different working conditions.

[0052] Furthermore, for the virtual twin corresponding to the device, normal virtual twin data and various abnormal virtual twin data can be obtained by adjusting the first simulation parameters. Specifically, this can be achieved by changing the operating parameters of the device (such as increasing the temperature, changing the density, reducing the speed, etc.) and simulating abnormal changes in the external environment (such as increasing wear, changing the heat transfer coefficient, etc.). In this way, the operation of the device in normal operation and various abnormal working conditions can be simulated, providing data support for the subsequent analysis of the characteristic differences of the device in different states.

[0053] Furthermore, after obtaining normal virtual twin data and various abnormal virtual twin data, the team subtracted the abnormal virtual twin data from the normal operating state to identify the differences between the two, thereby determining various difference sequence samples. These samples can highlight the characteristic changes of the equipment under abnormal conditions, providing valuable information for the training of the intelligent agent.

[0054] Finally, the agent is trained using a variety of difference sequence samples. By inputting these samples into the agent's training model, the agent learns the characteristics and patterns of the device under normal and abnormal conditions, thereby improving its ability to identify and handle abnormal device conditions.

[0055] The following describes in detail how to use multiple difference sequence samples to train an agent. The method includes:

[0056] For any differential sequence sample, determining a first cleaning action;

[0057] Based on the first cleaning action, determining the reward function value corresponding to the difference sequence sample;

[0058] Based on the correlation between the reward function value and the timing error, the reward function value is updated, and the first cleaning action is updated until the reward function value reaches a maximum value, and the first cleaning action corresponding to the maximum value of the reward function is determined as the target cleaning action of the difference sequence sample.

[0059] In the present disclosure, when training an intelligent agent using difference sequence samples, a first cleaning action can be selected. The first cleaning action can be understood as a cleaning process for the difference sequence samples. After selecting the first cleaning action, it is applied to the corresponding difference sequence sample to perform preliminary processing and optimization on the sample data. Next, a reward function value corresponding to the difference sequence sample is determined based on the first cleaning action. The reward function measures the optimization effect of the first cleaning action on the difference sequence sample, and its value reflects the effectiveness of the action in cleaning the data. In this way, the effectiveness of the first cleaning action can be quantitatively evaluated using the reward function value, providing a clear reference for subsequent action adjustments and optimization. Furthermore, since the reward function value is correlated with the timing error, this relationship can be used to update the reward function value. The timing error generally reflects the accuracy and consistency of the sample data in the time series. When the first cleaning action affects the timing error, the reward function value will also change accordingly. Based on this correlation between the reward function value and the timing error, the reward function value and the first cleaning action can be updated. During each update, the agent can adjust and improve the first cleaning action based on the changes in the reward function value, attempting to find a more optimal cleaning strategy. This process is repeated until the reward function value reaches its maximum value. When the reward function value reaches its maximum value, it means that the first cleaning action at that time has achieved the optimal cleaning effect for the difference sequence sample under the current conditions. Therefore, the first cleaning action corresponding to the maximum reward function value is designated as the target cleaning action for the difference sequence sample. In this way, the agent can gradually find the optimal cleaning strategy for different difference sequence samples through continuous learning and adjustment, thereby improving the efficiency and accuracy of data cleaning.

[0060] Exemplarily, the preferred process of training an agent using multiple difference sample sequences is as follows:

[0061] Preparatory action: Initialize the reward function value table and define the reward function value of the difference sequence data and the cleaning action for any difference sequence data.

[0062] Step 1: Select a cleaning action: A greedy (ε-greedy) strategy can be used to select a cleaning action from the action space. That is, based on the current reward function table, the current optimal cleaning action is selected for the input sequence segment. The selection process operates according to the principle of probabilistic jumping, that is, there is a probability of ε to select a suboptimal or even inferior cleaning action.

[0063] Step 2: Execute the action and observe the result: In the simulator, according to the discretized abnormal feature array, by adjusting the deviation degree of the simulator parameters, a physical sequence whose abnormal deviation degree sequence meets the conditions is obtained.

[0064] Step 3: Calculate the reward function value under the current cleaning action.

[0065] Step 4: Update the reward function value using the correlation between the reward function value and the timing error. Specifically, first, calculate the untrimmed temporal difference error, which can satisfy the following formula:

[0066] δ=R(s,a)+γmaxQ(s′,a′)-Q(s,a)

[0067] Q(s,a)=∑R(s,a)

[0068] Where δ represents the temporal difference error, R(s,a) represents the reward function value for taking cleaning action a under the difference sequence sample s, γ represents the discount factor, maxQ(s′,a′) represents the sum of the reward function values obtained for cleaning action a′ in the next period of time, and Q(s,a) represents the sum of the reward function values obtained for cleaning action a in the next period of time.

[0069] Then the timing difference error is trimmed to meet the following formula:

[0070] δ clipped =min(max(δ,-C),C)

[0071] Among them, δ clipped It is represented as the temporal difference error after clipping, and C is the clipping threshold.

[0072] Finally, the reward function value is updated using the clipped temporal difference error:

[0073] Q(s,a)←Q(s,a)+αδ clipped

[0074] Among them, α represents the learning rate.

[0075] Step 5: Update the cleaning action based on the updated reward function value: By adjusting the simulator parameters, a virtual physical signal that conforms to the abnormal form can be output.

[0076] Step 6: Repeat steps 1 to 6 until the number of iterations is reached or the reward function value converges. For a given state, select the action with the maximum reward function value as the cleaning strategy, which can satisfy the following formula:

[0077] π(s)=argmaxQ(s,a)

[0078] Among them, π(s) represents the cleaning strategy under the difference sequence sample s.

[0079] The following describes in detail how the reward function of the present disclosure is determined:

[0080] The reward function value of the present disclosure is determined based on the spectral density, kurtosis, and margin weighting of the difference sequence samples.

[0081] In this disclosure, the reward function can be understood as an important indicator used to quantitatively evaluate the effectiveness of cleaning actions taken on difference sequence samples. The reward function value is determined based on a weighted process of the spectral density, kurtosis, and margin of the new sequence data and the original sequence data after the first cleaning action. Specifically, the spectral density reflects the energy distribution in the frequency domain of the new sequence data and the original sequence data after the cleaning action, reflecting the relative strength of different frequency components in the signal. For the new sequence data and the original sequence data, appropriate spectral density characteristics help more accurately identify signal patterns and characteristics, thereby providing more valuable information for subsequent data analysis and processing. Kurtosis describes the sharpness or flatness of the distribution of the new sequence data and the original sequence data after the cleaning action. By analyzing the kurtosis, anomalies in the sample data can be detected and the effectiveness of the cleaning action in improving data anomalies can be evaluated. Margin is an important indicator for measuring the reliability and stability of the new sequence data and the original sequence data after the cleaning action. It reflects the degree of fluctuation and interference resistance of the sample data within a certain range.

[0082] The reward function disclosed herein can comprehensively consider the impact of spectral density, kurtosis, and margin on the new sequence data and original sequence data after the cleaning action, and adopt a weighted processing method to determine the reward function value. Specifically, corresponding weights are assigned to the spectral density, kurtosis, and margin, and the size of these weights can reflect the relative importance of each indicator in evaluating the effectiveness of the cleaning action. By multiplying the spectral density, kurtosis, and margin by the corresponding weights and adding the results, the final reward function value can be obtained. In this way, the reward function can comprehensively and objectively reflect the comprehensive optimization effect of the cleaning action on the difference sequence samples in many aspects, and provide accurate guidance for the intelligent agent to select the optimal cleaning action during the training process.

[0083] For example, the calculation of the reward function can satisfy the following formula:

[0084] R(s,a)=α spectral R spectral +α kurtosis R kurtosis +α margin R margin

[0085] Among them, α spectral Expressed as the weight coefficient of spectral density, R spectral represents the reward value of the spectral density, α kurtosis Expressed as the weight coefficient of kurtosis, R kurtosis Expressed as the reward value of kurtosis, αmargin Expressed as the weight coefficient of margin, R margin The reward value expressed as margin.

[0086] Specifically, α spectral , α kurtosis , α margin The three weight coefficients can be adjusted flexibly according to actual conditions, and R spectral 、R kurtosis 、R margin The three reward values are calculated as follows:

[0087] R spectral The calculation can satisfy the following formula:

[0088]

[0089] Among them, S(f) represents the spectral density of the difference sequence sample, f noise Expressed as the frequency range of the noise, f signal Expressed as the frequency range of the useful signal.

[0090] R kurtosis The calculation can satisfy the following formula:

[0091] R kurtosis =-|kurtosis(s′)|

[0092] Here, kurtosis(s′) represents the kurtosis of the difference sequence after cleaning.

[0093] R margin The calculation can satisfy the following formula:

[0094] R margin =margin(s′)-margin(s)

[0095] Wherein, margin(s′) represents the margin of the difference sequence after cleaning, and margin(s) represents the margin of the difference sequence.

[0096] The following will specifically describe how the present disclosure obtains the first simulation parameter. In this case, the method includes:

[0097] Inputting the homogeneous sequence data into at least one simulator to obtain at least one simulation result;

[0098] For any simulation result, when the simulation result does not meet the first threshold, updating the second simulation parameter corresponding to the simulation result until the simulation result meets the first threshold;

[0099] When all simulation results meet the first threshold, obtaining timing waveform features of all simulation results;

[0100] When all the timing waveform characteristics meet the second threshold, determining that all the simulation results are the same;

[0101] The first simulation parameters corresponding to the same results are obtained.

[0102] In the present disclosure, first, isomorphic sequence data can be input into at least one simulator. Isomorphic sequence data is data that has consistency in data type and data timing after isomorphism processing. Different simulators can simulate and analyze the input isomorphic sequence data based on their own algorithms and models, thereby obtaining at least one simulation result. After obtaining the simulation results, the consistency of the timing waveform characteristics of all simulation results can be determined to determine whether the simulation results are the same. Among them, the timing waveform characteristics can reflect the changing patterns and characteristics of the simulation results in the time series, can contain a large amount of information about the simulation process and results, and can more comprehensively and accurately understand the characteristics and performance of the simulation results.

[0103] Next, for any simulation result, it can be compared with a first threshold. The first threshold can be a pre-set standard or limit used to measure whether the simulation result meets the expected requirements or goals. This threshold can be determined based on the specific simulation objectives and needs, and its unit can be a percentage deviation, without limitation. If a simulation result does not meet the first threshold, the second simulation parameter corresponding to the simulation result can be updated until all simulation results meet the first threshold. The second simulation parameter can be a variable or factor closely related to the simulation process. By adjusting the simulation conditions and process, different simulation results can be obtained. This will be described in detail later in conjunction with the embodiments. Then, when all simulation results meet the first threshold, the timing waveform characteristics of all simulation results are obtained, and then all timing waveform characteristics can be compared with a second threshold. The second threshold can be at least one pre-set standard based on actual needs or historical experience, used to determine whether the simulation results meet the requirements. When all timing waveform characteristics meet the second threshold, this means that the time series variation patterns and characteristics of the simulation results obtained by each simulator are highly consistent, meaning that all simulation results are determined to be identical. Once all simulation results are determined to be identical, the first simulation parameter can be obtained.

[0104] When the timing waveform feature of any simulation result does not meet the second threshold, it can be adjusted to meet the second threshold. The method includes:

[0105] When any one of the timing waveform characteristics does not meet the second threshold, determining that the simulation results are different;

[0106] Adjust the timestamps or speeds of simulators with different simulation results until all simulation results are the same.

[0107] In the present disclosure, when any one of the timing waveform characteristics does not meet the second threshold, it means that there are deviations in the changing patterns and characteristics of the simulation results of different simulators in the time series, and it is impossible to directly determine that all simulation results are the same. At this time, it is necessary to adjust the simulators with different simulation results, and by adjusting their timestamps or speeds, change the performance of the simulator output results in the time dimension, so that the timing waveform characteristics of its output can be close to other simulation results, thereby meeting the requirements of meeting the second threshold. In this way, by adjusting the simulation results that do not meet the second threshold, it can be ensured that all simulation results have a high consistency in time series characteristics, and finally successfully obtain accurate and reliable first simulation parameters. This process ensures the rigor and accuracy of the entire simulation and modeling process, so that the virtual twin constructed based on the first simulation parameters can more realistically reflect the operation of the actual equipment, and provide a solid foundation for subsequent intelligent body training and equipment management.

[0108] The following describes the specific features of the timing waveforms disclosed herein:

[0109] The timing waveform characteristics disclosed herein may include but are not limited to at least one of the following: root mean square, peak-to-peak value, kurtosis, and margin.

[0110] In the present disclosure, the timing waveform features can display the information of the simulation results in the time series from multiple dimensions. Among them, the root mean square can be calculated by taking the square root of the average value of the square of the instantaneous value of the signal at each moment, which can effectively characterize the average energy level of the signal. The peak-to-peak value is the difference between the maximum and minimum values of the signal in one cycle. It can clearly define the range of signal fluctuations and is an important indicator for judging the severity of signal changes. Kurtosis can be used as a statistic to describe the degree to which the data distribution deviates from the normal distribution. In the timing waveform features, it is mainly used to reveal whether there are extreme values or sudden shocks in the data. Margin can reflect the fluctuation characteristics and anti-interference ability of the signal within a certain range. A larger margin means that the signal has stronger robustness and can maintain relatively stable characteristics even in complex and changing working environments. These timing waveform features work together to provide a rich perspective for a comprehensive and in-depth understanding of the simulation results, helping to accurately judge the operating status of the equipment and the effectiveness of the simulation.

[0111] The specific content of the first simulation parameter of the present disclosure will be described below:

[0112] When the simulator simulates the temperature of the motor, the first simulation parameter of the present disclosure may include but is not limited to at least one of the following: density, stiffness, moment of inertia, wear, damping, and heat transfer coefficient.

[0113] In the present disclosure, the first simulation parameter can comprehensively characterize the operating state of the device. When the motor temperature is within the specified range, the simulation parameters may include, but are not limited to, at least one of the following: density, stiffness, moment of inertia, wear, damping, and heat transfer coefficient. Density, representing the mass per unit volume of a substance, plays a fundamental role in device simulation. For example, in different scenarios, density can directly impact the overall performance and stability of an engine. Accurately setting the density parameter is crucial for accurately simulating engine operation. Stiffness measures the device structure's ability to resist deformation. For example, in bridge design and simulation, the stiffness parameters of each bridge component determine its degree of deformation under external forces such as vehicle and pedestrian loads and natural wind. Appropriate stiffness parameters ensure that the virtual simulation accurately reflects the bridge's actual load-bearing capacity and safety. The moment of inertia, representing the rotational inertia of an object, is a core parameter in device simulation involving rotational motion. The motor rotor's moment of inertia affects the dynamic processes of motor starting, stopping, and speed regulation. By accurately setting the moment of inertia parameter, the virtual simulation can realistically reproduce the motor's operating characteristics under different operating conditions. Wear is a critical parameter used to simulate the loss of surface material due to friction, corrosion, and other factors during the long-term use of simulated equipment. For example, in the simulation of the piston and cylinder wall of an automobile engine, considering the wear parameters between the piston and cylinder wall can effectively predict the engine's power performance degradation at different mileages, providing a strong basis for engine maintenance and life assessment. Damping is primarily used to describe the energy dissipation characteristics of a vibration system and is indispensable in the simulation of various vibration equipment. For example, in the simulation of the seismic design of building structures, the appropriate selection of damping parameters can simulate the attenuation of the structure's vibration under the action of seismic waves, helping designers optimize the structure's seismic performance. The heat transfer coefficient plays a key role in the simulation of equipment involving heat transfer. For example, in the simulation of the condenser of an air conditioning system, the heat transfer coefficient determines the rate at which heat is transferred from the refrigerant to the external environment. Accurately setting this parameter allows the simulation to accurately simulate the heat dissipation performance of the condenser, providing data support for optimizing the energy efficiency of the air conditioning system.

[0114] These first simulation parameters comprehensively reflect the equipment characteristics from multiple fields such as mechanics and thermals, providing indispensable basic operating status data for building a highly realistic virtual twin.

[0115] As an example, the following describes how to build a digital twin:

[0116] The steps to establish a digital twin system are divided into two stages:

[0117] Phase 1: During the simulator optimization phase, the relationships between sensors are used as a reference to adjust simulator parameters such as density, stiffness, moment of inertia, wear, damping, and heat transfer coefficient using a unique multi-step gradient descent algorithm. The parameter adjustment process includes the following steps:

[0118] a. Input parameter vectors and change each vector by a small increment dx i , record the simulation accuracy change dy at this time i , and use this to calculate the gradient vector:

[0119]

[0120] b. Change all parameters by percentage, record the simulation accuracy change dy at this time, and update each parameter using the following formula:

[0121]

[0122] c. By repeatedly adjusting each simulator, an accurate mapping of sensor variables is ultimately achieved, indicating that the physical parameters of the device have been accurately calibrated.

[0123] Phase 2: Compare the waveforms of the calculation results of similar devices in the cluster. By comparing the four key parameters of the real-time simulation waveforms output by different simulators, it can be determined whether the delay of the simulation process is synchronized with the rest of the simulators. If they are not synchronized, modify the time stamp delay of the simulator output data. If modifying the timestamp does not meet the time alignment requirements, further modify the parameters related to the calculation speed in the simulator to time-align the output results of multiple simulators. Finally, all parameters are loaded into the property file of the 3D model. After connecting to the motion data and sensor data, a spatiotemporally synchronized digital twin cluster is formed in the simulation space. Its characteristics are that the real-time simulation process clocks of the simulators of all devices are synchronized and the physical signal simulation is accurate.

[0124] Among them, the calculation of the four key parameters of the analog waveform can be:

[0125]

[0126] Where RMS stands for root mean square, N stands for the total number of data points, and x i Represents the value of the i-th data point.

[0127] PP=x max -x min

[0128] Where PP represents peak-to-peak value, x max Indicates the maximum value in the data, xmin Indicates the minimum value in the data.

[0129]

[0130] Where GM represents the margin and α represents the margin-related parameter.

[0131]

[0132] Among them, K urtosis ‘ represents the kurtosis, and b represents the standard deviation of the data.

[0133] Figure 3 A schematic diagram of the process of establishing a digital twin for the preferred motor temperature simulation provided by the embodiment of the present disclosure. Figure 3 As can be seen, the following steps are as follows: Step 1: Load the physical data of device X in the cluster. Step 2: Set the material parameters and solve the calibration parameters of interest using the set parameters and the simulator. For example, the spindle temperature inside the motor can be calculated based on the measured housing temperature and air temperature. Step 3: Use the temperature sensor inside the motor to compare the actual spindle temperature with the simulated spindle temperature. Step 4: Determine whether the percentage deviation value exceeds the threshold. If so, modify the simulator parameters. If not, proceed to Step 5. Step 5: Output the simulation parameters and use them as the twin configuration. Step 6: Compare the output of the simulator with the output of other devices: RMS, peak-to-peak value, kurtosis, and margin. The specific calculation method is described above and will not be repeated here. Step 7: Determine whether the percentage deviation of the four spatial parameters is less than the threshold. If so, proceed to Step 8. If not, modify the data's time stamp or amplitude coefficient. Step 8: Add the device twin to the cluster field policy, bind the motion data, and enable the sensor data interface. All temperature data is automatically input to the simulator, and the simulated spindle temperature is continuously output.

[0134] The following will specifically explain how to perform isomorphism processing, including the following methods:

[0135] The isomorphism processing disclosed in the present invention includes: type isomorphism and time series isomorphism.

[0136] In this disclosure, type isomorphism can be understood as the standardized processing of different types of data in the device's raw sequence data, so that the different types of data reach consistency in terms of data format, data type expression, etc. For example, temperature data collected by different devices, some expressed as floating-point numbers in Celsius, and some expressed as integers in Fahrenheit, can be uniformly converted to a standard floating-point format in Celsius through type isomorphism, allowing subsequent analysis and processing within the same data framework.

[0137] Time series isomorphism can be understood as regularizing raw sequence data along the time series dimension, ensuring a uniform timestamp format, consistent time intervals, and chronological continuity. For example, some devices may collect data at irregular intervals, with some collecting data every minute, while others collect data based on specific events. Through time series isomorphism, all data is resampled or interpolated at a uniform time interval, resulting in a regular and continuous distribution across the time series.

[0138] More specific methods for isomorphism processing include:

[0139] Get historical processing data of the device;

[0140] The original sequence data is corrected by using the ratio relationship between the historical processed data and the original sequence data to obtain the isomorphic sequence data.

[0141] In the present disclosure, when performing isomorphism processing (type isomorphism, time series isomorphism), it is necessary to obtain the historical processing data of the device. Among them, the historical processing data can be understood as a data set obtained based on a historical template, which has completed the type and time series regularization through the previous same type of processing process. These historical processing data can include various types of labeled processing information under the previous equipment operation state, which conform to the standard format in terms of data type and time series. By obtaining these historical processing data, the ratio relationship between them and the original sequence data can be used to correct the original sequence data. The specific correction method can be adjusted according to different isomorphisms, which will be described in detail in conjunction with the embodiments below. In this way, by combining the ratio relationship between the historical processing data and the original sequence data for correction operations, not only can the original data be optimized by utilizing the empirical information contained in the historical data, but also the quality and reliability of the isomorphic sequence data can be further improved on the basis of type isomorphism and time series isomorphism, providing a more solid data foundation for subsequent simulation, modeling, and intelligent agent training based on the data.

[0142] When the classification of isomorphic processing is different, the historical processing data will also be different, as follows:

[0143] When the isomorphism processing is type isomorphism, the historical processing data may include but is not limited to at least one of the following: amplitude effective value, amplitude peak value, amplitude variance, prediction length and constraint complexity;

[0144] When the isomorphism processing is time series isomorphism, the historical processing data may include but is not limited to at least one of the following: protocol bandwidth, processing delay, data frame length, and test calculation delay.

[0145] In the present disclosure, when the isomorphism process is type isomorphism, the amplitude effective value in the historical processed data can reflect the average effective level of the device's signal amplitude during past operation and can be used to compare the signal amplitude effective value in the current raw sequence data to see if it is within the normal range. The amplitude peak value records the maximum amplitude value ever reached by the device signal, which is crucial for determining whether the signal peak in the raw sequence data is abnormal. The amplitude variance reflects the degree of dispersion of the past signal amplitude relative to the average value. The larger the variance, the more severe the signal amplitude fluctuation. By comparing it with the amplitude variance of the raw sequence data, the stability of the current signal amplitude fluctuation can be identified. The prediction length represents the time span or amount of data involved in the previous prediction of the device's operating status. It helps to rationally process and adjust the raw sequence data based on historical prediction experience during the type isomorphism process to meet unified data type and format requirements. The constraint complexity reflects the complexity of the historical data in meeting various constraints. These constraints may involve the physical characteristics of the device, operating rules, etc. By referring to this complexity, the raw sequence data can be better incorporated into the existing data type specification system.

[0146] For example, when the isomorphism processing is type isomorphism, the calculation corrected by using historical processing data may satisfy the following formula:

[0147]

[0148] The five correction factors are as follows:

[0149]

[0150]

[0151] Among them, F(t) represents the data of the original sequence data after type isomorphism, f(t) represents the original sequence data, K1 represents the ratio of the amplitude effective values of the historical processed data and the original sequence data, K2 represents the ratio of the amplitude peak values of the historical sequence data and the original sequence data, K3 represents the ratio of the amplitude variances of the historical sequence data and the original sequence data, K4 represents the ratio of the predicted lengths of the historical sequence data and the original sequence data, and K5 represents the ratio of the constraint complexity of the historical sequence data and the original sequence data.

[0152] When isomorphism is performed on time series data, the protocol bandwidth in historical processing data refers to the information transmission rate range specified by the protocol followed by the device during past communications. It determines the data transmission volume and cadence along the time axis. When performing time series isomorphism on raw sequence data, the protocol bandwidth can be used to determine whether the current data transmission meets historical standards, thereby adjusting the data transmission interval. Processing delay records the time it took the device to process data in the past. This is important for ensuring that the processing time of the current raw sequence data is consistent with historical data or conforms to expected patterns. Data frame length refers to the amount of information contained in each data frame in historical data transmission. It can affect the data segmentation and transmission method in the time series. By referring to the historical data frame length, the raw sequence data can be appropriately framed and reassembled to achieve time series isomorphism. Test and computation delay reflects the time consumed by the device during historical testing and computation tasks. When processing raw sequence data, test and computation delay can be used to rationally allocate computing resources and time windows, ensuring the accuracy and consistency of data processing in the time sequence, ultimately completing the time series isomorphism operation.

[0153] For example, when the isomorphism processing is time series isomorphism, the calculation corrected by using historical processing data may satisfy the following formula:

[0154] T=L1·L4·(t-t2-t4)

[0155]

[0156] t2 = standard terminal processing delay

[0157]

[0158] t4 = calculated delay from clock synchronization test

[0159] Among them, T represents the data of the original sequence data after time series isomorphism, L1 represents the ratio of the protocol bandwidth of the historical sequence data to the original sequence data, t2 represents the standard terminal processing delay of the original sequence data, L3 represents the ratio of the data frames of the historical sequence data to the original sequence data, and t4 represents the measurement and calculation delay of the original sequence data after clock synchronization.

[0160] The following describes how to obtain the differential sequence data of a device. The method includes:

[0161] Monitor and collect normal sequence data and abnormal sequence data of equipment;

[0162] Obtain the difference sequence between normal sequence data and abnormal sequence data;

[0163] The difference sequence is comprehensively processed to obtain difference sequence data; the comprehensive processing includes at least one of the following: waveform quantization processing, peak quantization processing, pulse quantization processing, root mean square processing, peak-to-peak processing, and variance processing.

[0164] In the present disclosure, special monitoring equipment and acquisition systems can be used to continuously and comprehensively monitor and collect data generated by the equipment during normal operating conditions and when abnormal conditions occur. Normal sequence data represents the operating data of the equipment in a stable and fault-free state. Abnormal sequence data corresponds to relevant data when the equipment fails or the operating state deviates from the normal track. After successfully obtaining the normal sequence data and the abnormal sequence data, the difference sequence between the two can be further calculated. The purpose of this step is to highlight the changes in data when the equipment changes from a normal state to an abnormal state. By subtracting each data point in the abnormal sequence data from the corresponding time point data in the normal sequence data, the difference is obtained, and these differences constitute a difference sequence. The difference sequence intuitively shows the degree of deviation and change trend of each parameter of the equipment relative to the normal state when an abnormality occurs. Subsequently, the difference sequence is comprehensively processed to obtain difference sequence data that is more valuable and representative for analysis.

[0165] Comprehensive processing can include, but is not limited to, at least one of the following: waveform quantization, which discretizes a continuous waveform signal into a finite sequence of quantization levels according to specific quantization rules. This process converts complex waveform information into a data format that is convenient for computer storage, transmission, and analysis, while preserving the waveform's key characteristics as much as possible. Peak quantization, which quantizes the peak information in the signal. Specifically, it maps the maximum or minimum peak amplitude of the signal to a corresponding quantized value based on a pre-defined quantization standard. Pulse quantization, which converts various pulse parameters such as pulse amplitude, width, and interval into discrete values according to a specific quantization method for signals with pulse characteristics. This effectively simplifies the representation of pulse signals and facilitates subsequent statistical analysis, identification, and classification of pulse signals. Root mean square (RMS) processing, which performs an RMS operation on the data in the difference sequence. The RMS value effectively reflects the average energy level of the data and can highlight the average intensity of the data fluctuations in the difference sequence. Peak processing focuses on extracting the maximum and minimum values in the difference sequence, that is, the peaks. These peaks can intuitively reflect the extreme conditions of parameter changes in the equipment under abnormal conditions, which helps to quickly identify possible serious fault hazards. Variance processing is used to measure the degree of discreteness of the difference sequence data. The larger the variance, the more drastic the fluctuation of the data, reflecting the higher the instability of the equipment operation status. Frequency domain conversion processing, with the help of methods such as Fourier transform, can convert the difference sequence from the time domain to the frequency domain, revealing the distribution of data in different frequency components. Different equipment failures often produce characteristic changes in specific frequency bands. Through frequency domain conversion processing, the frequency characteristics hidden in the time domain data can be excavated, providing a strong basis for accurately diagnosing the type of equipment failure.

[0166] Exemplarily, a preferred method for obtaining differential sequence data may include:

[0167] The basic characteristics of the difference sequence between abnormal data and normal data are used as the state vector:

[0168] n(t)=n notmal (t)-n snmotmal (t)

[0169] Among them, n normal (t) represents normal data, n abnormal (t) represents abnormal data.

[0170] Then the following index of the difference sequence n(t) is calculated as the state (ie, the difference sequence sample of the present disclosure):

[0171] [Waveform index, peak index, pulse index, RMS, peak-to-peak value, variance].

[0172] The following describes the content of the differential sequence samples disclosed herein:

[0173] The cleaning action disclosed herein may include but is not limited to at least one of the following: filter type, filter function.

[0174] Filter types can be understood as different ways of filtering and processing signals. Specifically, they include: low-pass filters, which allow low-frequency signals to pass while attenuating high-frequency signals and are often used to remove high-frequency noise from signals; high-pass filters, which, in contrast to low-pass filters, only allow high-frequency signals to pass while suppressing low-frequency signals and can be used to enhance signal detail; band-pass filters, which allow signals within a specific frequency range to pass while blocking signals outside of that range and are often used to select signals in specific frequency bands in communication systems; and band-stop filters, which act in the opposite way of band-pass filters, blocking signals within a specific frequency range while retaining signals at other frequencies and can be used to eliminate interference at specific frequencies. A filter function can be understood as a mathematical expression that defines the filter's specific operation and characteristics, accurately describing how the filter processes the input signal. Specifically, they include: finite impulse response filter functions, whose output depends only on the current and past input values; and infinite impulse response filter functions, whose output depends not only on the current and past input values but also on past output values, enabling the implementation of more complex filtering characteristics with fewer coefficients.

[0175] Exemplary, preferred complete process of training an agent is as follows:

[0176] Step 1: Initialize the reward function value table to store the reward function value of each state (i.e., the difference sequence data disclosed in the present invention)-action (i.e., the cleaning action disclosed in the present invention) pair.

[0177] Step 2: Define the state space (i.e., the multiple difference sequence data disclosed in this disclosure). This includes normal and abnormal time series segments of the device and their corresponding discrete state vectors. The sequences can be stored in a database. Since the processing target is abnormal data, the basic features of the difference sequence between abnormal data and normal data are used as the state vector, and then the indicators of the difference sequence are calculated as the state. By calculating the difference sequence between normal data and abnormal data, the state space directly reflects the difference between the two, which helps to highlight abnormal features and thus improve the accuracy of anomaly detection.

[0178] Step 3: Define the action space (i.e., the cleaning actions disclosed in this paper), including completion, deletion, and replacement operations. The main action of data cleaning in this solution is to select the types and parameters of various filters, which are represented by the following filter attribute dictionary:

[0179] {filter type: [param1, param2, param3, ...]}.

[0180] Since the types and parameters of filters that can perform completion, deletion, and replacement operations are limited, all filter types and parameter optional ranges constitute the action space. The core goal of the reinforcement learning method adopted in this scheme is to intelligently design filter parameters based on different abnormal sequence types.

[0181] Step 4: Set the learning rate, discount factor, clipping threshold C, and the weight coefficient of the reward function.

[0182] Step 5: Randomly modify certain parameters to abnormal values in the simulator cluster, and then generate various virtual signals as the output state s.

[0183] Step 6: Loop through the following steps until the stop condition is met:

[0184] (1) Select a cleaning action; (2) Execute the action and observe the result; (3) Calculate the reward; (4) Calculate the temporal difference error. (5) Apply the clipping mechanism to clip the temporal difference error. (6) Update the Q value and use the clipped temporal difference error to update the Q table. (7) Update the cleaning action until maximum convergence is achieved.

[0185] For example, consider a production line composed of 30 industrial robots, each capable of collecting data on motor temperature, current, speed, and angle. Due to sensor failures or communication delays, these signals may sometimes be disconnected or misaligned. To ensure stable decision-making, these signals are fed into a reinforcement learning agent. Because the agent is trained on a large number of virtual abnormal signals output by a simulator that have been corrected using normal synchronous data, and its reward function is designed using margin, kurtosis, and spectral density, the agent selects filter parameters to correct the signal, aiming to suppress drastic changes in these metrics. Ultimately, the time series data for these 30 industrial robots in the database remains continuous, stable, and clear, effectively improving relevant analytical tasks.

[0186] The present disclosure also provides a data processing device. Figure 4 A structural block diagram of a data processing device provided in an embodiment of the present disclosure is shown in FIG. Figure 4 As shown, the data processing device 400 includes:

[0187] An acquisition unit 401 is configured to acquire differential sequence data of a device; the differential sequence data is determined based on a difference sequence between normal sequence data and abnormal sequence data;

[0188] The processing unit 402 is used to process the difference sequence data using a pre-trained intelligent agent to obtain a cleaning result obtained by the intelligent agent based on a target cleaning action; wherein; the intelligent agent is trained based on a variety of difference sequence samples; the difference sequence samples are virtual twin data.

[0189] In an exemplary embodiment, the processing unit 402 is specifically used to: for any device, perform isomorphic processing on the original sequence data of the device to obtain isomorphic sequence data; based on the isomorphic sequence data and at least one simulator, obtain the first simulation parameter when the simulation result is the same; input the first simulation parameter into the second model to obtain a virtual twin; the second model is a three-dimensional modeling model; based on the virtual twin, adjust the first simulation parameter to obtain normal virtual twin data and multiple abnormal virtual twin data; based on the normal virtual twin data and multiple abnormal virtual twin data, determine multiple difference sequence samples; and use multiple difference sequence samples to train the intelligent agent.

[0190] In an exemplary embodiment, the processing unit 402 is specifically used to: determine a first cleaning action for any difference sequence sample; determine a reward function value corresponding to the difference sequence sample based on the first cleaning action; update the reward function value based on the correlation between the reward function value and the timing error, and update the first cleaning action until the reward function value reaches a maximum value, and determine the first cleaning action corresponding to the maximum value of the reward function as the target cleaning action for the difference sequence sample.

[0191] In an exemplary embodiment, the processing unit 402 is specifically configured to determine the reward function value based on a weighted process of the spectral density, kurtosis, and margin of the difference sequence samples.

[0192] In an exemplary embodiment, the processing unit 402 is specifically configured to: input the homogeneous sequence data into at least one simulator to obtain at least one simulation result; obtain timing waveform characteristics of all simulation results; and determine that all simulation results are identical when all timing waveform characteristics meet a preset threshold. Obtain a first simulation parameter corresponding to the identical simulation results.

[0193] In an exemplary embodiment, the processing unit 402 is specifically used to: determine that there are differences in simulation results when any timing waveform feature does not meet a preset threshold; and adjust the timestamps or speeds of simulators with different simulation results until all simulation results are the same.

[0194] In an exemplary embodiment, the processing unit 402 is specifically configured to: the timing waveform characteristics include at least one of the following: root mean square, peak-to-peak value, kurtosis, and margin.

[0195] In an exemplary embodiment, the processing unit 402 is specifically configured to: the first simulation parameter includes at least one of the following: density, stiffness, moment of inertia, wear, damping, and heat transfer coefficient.

[0196] In an exemplary embodiment, the processing unit 402 is specifically configured to: the isomorphism processing includes: type isomorphism and time sequence isomorphism.

[0197] In an exemplary embodiment, the processing unit 402 is specifically configured to: obtain historical processing data of the device; and modify the original sequence data using a ratio between the historical processing data and the original sequence data to obtain isomorphic sequence data.

[0198] In an exemplary embodiment, the processing unit 402 is specifically used to: when the isomorphism processing is type isomorphism, the historical processing data includes at least one of the following: amplitude effective value, amplitude peak, amplitude variance, prediction length and constraint complexity; when the isomorphism processing is time series isomorphism, the historical processing data includes at least one of the following: protocol bandwidth, processing delay, data frame length, test calculation delay.

[0199] In an exemplary embodiment, the processing unit 402 is specifically used to: monitor and collect normal sequence data and abnormal sequence data of the device; obtain a difference sequence between the normal sequence data and the abnormal sequence data; perform comprehensive processing on the difference sequence to obtain difference sequence data; the comprehensive processing includes at least one of the following: root mean square processing, peak processing, variance processing, and frequency domain conversion processing.

[0200] In an exemplary embodiment, the processing unit 402 is specifically configured to: the cleaning action includes at least one of the following: a filter type and a filter function.

[0201] Figure 5 This is a hardware block diagram of an electronic device provided in an embodiment of the present disclosure. The electronic device 500 according to an embodiment of the present disclosure includes at least a processor and a memory for storing computer-readable instructions. When the computer-readable instructions are loaded and executed by the processor, the processor executes the data processing method described in any of the preceding embodiments of the present disclosure.

[0202] Figure 5The electronic device 500 shown specifically includes: a central processing unit (CPU) 501, a graphics processing unit (GPU) 502, and a memory 503. These units are interconnected via a bus 504. The central processing unit (CPU) 501 and / or the graphics processing unit (GPU) 502 can be used as the above-mentioned processor, and the memory 503 can be used as the above-mentioned memory for storing computer-readable instructions. In addition, the electronic device 500 may also include a communication unit 505, a storage unit 506, an output unit 507, an input unit 508, and an external device 509, which are also connected to the bus 504.

[0203] Figure 6 A schematic diagram of a computer-readable storage medium provided in an embodiment of the present disclosure. Figure 6 As shown, a computer-readable storage medium 600 according to an embodiment of the present disclosure has computer-readable instructions 601 stored thereon. When the computer-readable instructions 601 are executed by a processor, the data processing method described with reference to the above figures according to any of the above embodiments of the present disclosure is executed. Computer-readable storage media include, but are not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, optical disk, magnetic disk, etc.

[0204] The present disclosure further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the data processing method described in any of the foregoing embodiments of the present disclosure.

[0205] The present disclosure provides a data processing method, apparatus, electronic device, storage medium, and program product. The present disclosure obtains differential sequence data from a device; the differential sequence data is determined based on a sequence of differences between normal and abnormal sequence data; and a pre-trained intelligent agent processes the differential sequence data to obtain a cleaning result based on a target cleaning action. The intelligent agent is trained based on multiple differential sequence samples, and the differential sequence samples are virtual twin data. This method, by constructing a digital twin that fully corresponds to the physical device, maps the device's operating status in real time and generates a large number of virtual differential sequence samples, thus overcoming the traditional data cleaning reliance on real abnormal data. By training the intelligent agent with a large number of differential sequence samples, any abnormality in the real device can be directly input into the intelligent agent to obtain the target cleaning action. Furthermore, compared to abnormal or normal data, differential sequence data can better highlight the changing trends of the device's operating status, the subtle differences in abnormal characteristics, and the transition between normal and abnormal states. This allows the intelligent agent to more deeply understand the type and severity of different abnormal data, thereby providing more targeted and accurate cleaning strategies for specific differential sequence data, further enhancing the accuracy of cleaning action determination. Compared with existing manual labeling or manual testing methods, the method disclosed in the present invention has wide adaptability and can enhance the efficiency and accuracy of data cleaning.

[0206] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0207] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this disclosure are merely illustrative and not restrictive, and should not be construed as necessarily possessed by each embodiment of the present disclosure. Furthermore, the specific details disclosed above are provided for illustrative purposes and to facilitate understanding, rather than as limitations. These details do not limit the present disclosure to necessarily being implemented using these specific details.

[0208] The block diagrams of the devices, devices, equipment, and systems involved in this disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "include," "comprise," "have," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0209] Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.

[0210] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0211] Various changes, substitutions, and modifications may be made to the technology described herein without departing from the teachings defined by the appended claims. Moreover, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of things, means, methods, and actions described above. Currently existing or later developed processes, machines, manufactures, compositions of things, means, methods, or actions that perform substantially the same function or achieve substantially the same results as the corresponding aspects described herein may be utilized. Accordingly, the appended claims include within their scope such processes, machines, manufactures, compositions of things, means, methods, or actions.

[0212] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0213] The above description has been provided for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A data processing method, characterized in that: The method comprises: Acquiring differential sequence data of the device; the differential sequence data is determined based on a difference sequence between the normal sequence data and the abnormal sequence data; The difference sequence data is processed using a pre-trained intelligent agent to obtain a cleaning result obtained by the intelligent agent based on a target cleaning action; wherein; the intelligent agent is trained based on a variety of difference sequence samples; and the difference sequence samples are virtual twin data.

2. The method according to claim 1, characterized in that The method further comprises: For any device, the original sequence data of the device is homogenized to obtain homogeneous sequence data; Based on the isomorphic sequence data and at least one simulator, obtaining a first simulation parameter when the results are the same; Inputting the first simulation parameters into the second model to obtain a virtual twin; the second model is a three-dimensional model; Based on the virtual twin, adjusting the first simulation parameter to obtain normal virtual twin data and multiple abnormal virtual twin data; Determining a plurality of difference sequence samples based on the normal virtual twin data and the plurality of abnormal virtual twin data; The intelligent agent is trained using the multiple difference sequence samples.

3. The method according to claim 2, characterized in that The method of training the intelligent agent by using the plurality of difference sequence samples comprises: For any differential sequence sample, determining a first cleaning action; Determining a reward function value corresponding to the difference sequence sample based on the first cleaning action; Based on the correlation between the reward function value and the timing error, the reward function value is updated, and the first cleaning action is updated until the reward function value reaches a maximum value, and the first cleaning action corresponding to the maximum value of the reward function is determined as the target cleaning action for the difference sequence sample.

4. The method according to claim 3, characterized in that The reward function value is determined based on a weighted process of spectral density, kurtosis, and margin of the new sequence data after the first cleaning action and the original sequence data.

5. The method according to claim 2, characterized in that The step of obtaining a first simulation parameter when the results of the isomorphic sequence data and at least one simulator are the same includes: Inputting the isomorphic sequence data into at least one simulator to obtain at least one simulation result; For any one of the simulation results, when the simulation result does not meet a first threshold, updating a second simulation parameter corresponding to the simulation result until the simulation result meets the first threshold; When all the simulation results meet the first threshold, obtaining timing waveform features of all the simulation results; When all of the timing waveform characteristics meet a second threshold, determining that all of the simulation results are the same; The first simulation parameters corresponding to the same results are obtained.

6. The method according to claim 5, characterized in that The method further comprises: When any one of the timing waveform characteristics does not meet the second threshold, determining that the simulation results are different; The timestamps or speeds of the simulators with different simulation results are adjusted until all the simulation results are the same.

7. The method according to claim 5 or 6, characterized in that The timing waveform characteristics include at least one of the following: root mean square, peak-to-peak value, kurtosis, and margin.

8. The method according to claim 2, characterized in that When the simulator simulates the temperature of a motor, the first simulation parameter includes at least one of the following: density, stiffness, moment of inertia, wear, damping, and heat transfer coefficient.

9. The method according to claim 2, characterized in that The isomorphism processing includes: type isomorphism and time sequence isomorphism.

10. The method according to claim 2 or 9, characterized in that For any device, the original sequence data of the device is subjected to isomorphism processing to obtain isomorphic sequence data, including: Get historical processing data of the device; The original sequence data is corrected using the ratio relationship between the historical processed data and the original sequence data to obtain the isomorphic sequence data.

11. The method according to claim 9, characterized in that When the isomorphism processing is the type isomorphism, the historical processing data includes at least one of the following: amplitude effective value, amplitude peak value, amplitude variance, prediction length and constraint complexity; When the isomorphism processing is the time series isomorphism processing, the historical processing data includes at least one of the following: protocol bandwidth, processing delay, data frame length, and test calculation delay.

12. The method according to claim 1, characterized in that The acquiring device difference sequence data includes: Monitoring and collecting the normal sequence data and the abnormal sequence data of the device; Obtaining a difference sequence between the normal sequence data and the abnormal sequence data; The difference sequence is subjected to comprehensive processing to obtain the difference sequence data; the comprehensive processing includes at least one of the following: waveform quantization processing, peak quantization processing, pulse quantization processing, root mean square processing, peak-to-peak processing, and variance processing.

13. The method according to claim 3, characterized in that The cleaning action includes at least one of the following: a filter type and a filter function.

14. A data processing device, characterized in that: The device comprises: An acquiring unit, configured to acquire differential sequence data of a device; the differential sequence data is determined based on a difference sequence between normal sequence data and abnormal sequence data; A processing unit is used to process the difference sequence data using a pre-trained intelligent agent to obtain a cleaning result obtained by the intelligent agent based on a target cleaning action; wherein; the intelligent agent is trained based on a variety of difference sequence samples; and the difference sequence samples are virtual twin data.

15. An electronic device, characterized in that: include: a memory for storing computer-readable instructions; as well as A processor is configured to execute the computer-readable instructions so that the electronic device performs the method according to any one of claims 1 to 13.

16. A non-transitory computer-readable storage medium for storing computer-readable instructions, characterized in that: When the computer-readable instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 13.

17. A computer program product, characterized in that The method comprises a computer program, which implements the method according to any one of claims 1 to 13 when the computer program is executed by a processor.

Images