Data processing method and device, equipment and medium

Abstract

The embodiment of the invention provides a data processing method and device, equipment and a medium, and relates to the technical field of detection, and the method comprises the steps: collecting the data of a target object in real time, and determining effective data from the collected data. And performing setting processing on the valid data to obtain target data, the target data including characterization of the valid data. The valid data and the target data are stored in an associated manner, so that the validity of the stored data is ensured, the waste of storage space is reduced, and the retrieval efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of detection technology, and in particular to a data processing method, device, equipment and medium. Background Art

[0002] With the development of data transmission and detection technology, data recording and playback technology has been applied in more and more scenarios. For example, in energy storage systems, data recording and playback technology is of great significance for system debugging, fault diagnosis, performance evaluation and optimization. Traditional data recording and playback methods usually record a large amount of raw data, resulting in high storage costs, a lot of duplicate data, and low data retrieval efficiency after long-term accumulation and storage. Therefore, how to improve the effectiveness of data recording and playback and improve retrieval efficiency is a problem that needs to be studied. Summary of the invention

[0003] One of the purposes of the present invention includes, for example, providing a data processing method, apparatus, device and medium to at least partially improve data storage effectiveness, reduce storage space waste, and improve retrieval efficiency.

[0004] The embodiments of the present invention can be implemented as follows:

[0005] In a first aspect, an embodiment of the present invention provides a data processing method, including:

[0006] Collect data of target objects in real time;

[0007] Determining valid data from the collected data;

[0008] Performing setting processing on the valid data to obtain target data, wherein the target data includes a representation of the valid data;

[0009] The valid data and the target data are stored in association with each other.

[0010] In an optional implementation manner, determining valid data from the collected data includes:

[0011] Determine valid data based on the data quality of the collected data; or,

[0012] Monitoring the working status of the target object;

[0013] When the target object is in a normal working state, determining the collected data of the target object as valid data and recording it;

[0014] When the target object switches from a normal working state to an abnormal state, the collected data of the target object is determined to be invalid data, and only the timestamp when the data of the target object becomes invalid data is recorded.

[0015] In an optional implementation manner, the setting process of the valid data to obtain the target data includes:

[0016] Based on the valid data, determining the working condition of the target object;

[0017] Based on the working condition, classify the valid data to obtain classification information; and / or,

[0018] Extracting characteristic values ​​of the valid data;

[0019] Using the classification information and / or feature value as target data;

[0020] The storing the valid data and the target data in association with each other includes:

[0021] The classification information and / or feature value are associated with corresponding valid data and stored.

[0022] In an optional implementation manner, extracting the characteristic value of the valid data includes:

[0023] Preprocessing the valid data;

[0024] Initialize the sliding window and fill the preprocessed valid data into the sliding window point by point;

[0025] After the sliding window is filled with preprocessed valid data, the time domain characteristic value is calculated by finding the average, extreme value, variance and standard deviation;

[0026] After the sliding window is filled with pre-processed valid data, the frequency domain eigenvalue is calculated by fast Fourier transform;

[0027] The time domain eigenvalue and the frequency domain eigenvalue are used as the eigenvalues ​​of the valid data.

[0028] In an optional implementation manner, the storing the valid data and the target data in association with each other includes:

[0029] In combination with the timestamp, the valid data and the target data are stored in association;

[0030] When new valid data is obtained, it is determined whether the new valid data is different from the valid data stored at the previous moment;

[0031] If so, the valid data and target data with differences are stored in association with each other in combination with the timestamp;

[0032] If not, the storage of the new valid data is abandoned.

[0033] In a second aspect, an embodiment of the present invention provides a data processing method, including:

[0034] Obtain the working condition to be retrieved of the target object;

[0035] According to the valid data and target data associated and stored in the data processing method described in any one of the first aspects, the response data corresponding to the working condition to be retrieved is retrieved.

[0036] In an optional embodiment, the method further comprises:

[0037] When the working condition to be retrieved is a working condition that needs to be optimized, identifying the problem bottleneck and the corresponding key parameters according to the response data corresponding to the working condition to be retrieved;

[0038] Determine and implement the optimization solution of the target object based on the problem bottleneck and the corresponding key parameters, so as to iteratively obtain the optimal solution of the working condition to be retrieved; and / or,

[0039] When the working condition to be retrieved is a working condition that needs to be replayed, replaying is performed according to the response data corresponding to the working condition to be retrieved;

[0040] Using the replayed data as input to a test case, executing the test case and analyzing it; and / or,

[0041] When the working condition to be retrieved is a fault working condition, identifying key parameters affecting the fault and performing data analysis according to the response data corresponding to the working condition to be retrieved;

[0042] Formulate an early warning plan based on the results of the data analysis;

[0043] The early warning scheme is verified and the optimal scheme is iterated.

[0044] In a third aspect, an embodiment of the present invention provides a data processing device, including:

[0045] An information acquisition module is used to collect data of the target object in real time;

[0046] The information processing module is used to determine valid data from the collected data; perform setting processing on the valid data to obtain target data, wherein the target data includes a representation of the valid data; and associate and store the valid data and the target data.

[0047] In a fourth aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the data processing method described in any one of the aforementioned implementation modes when executing the program.

[0048] In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a computer program, and when the computer program is executed, the electronic device where the computer-readable storage medium is located is controlled to execute the data processing method described in any one of the aforementioned implementation modes.

[0049] The beneficial effects of the embodiments of the present invention include, for example, determining and storing valid data for the target object data collected in real time, thereby ensuring the validity of the stored data and reducing storage costs. By setting and processing the valid data, target data that characterizes the valid data is obtained, and the valid data and the target data are associated and stored, so that fast and reliable retrieval of the valid data can be achieved based on the characterizing target data, thereby improving retrieval efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0051] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present invention is shown.

[0052] Figure 2 A flow chart of a data processing method provided by an embodiment of the present invention is shown.

[0053] Figure 3 A schematic diagram of a sub-step flow chart of S120 provided in an embodiment of the present invention is shown.

[0054] Figure 4 A schematic diagram of a sub-step flow chart of S130 provided in an embodiment of the present invention is shown.

[0055] Figure 5 A schematic diagram of the sub-step flow of S133 provided in an embodiment of the present invention is shown.

[0056] Figure 6 A schematic diagram of a sub-step flow chart of S140 provided in an embodiment of the present invention is shown.

[0057] Figure 7 The figure shows an overall flow chart of a data processing method provided by an embodiment of the present invention.

[0058] Figure 8 Another flow chart of a data processing method provided by an embodiment of the present invention is shown.

[0059] Fig. 9 An exemplary structural block diagram of a data processing device provided by an embodiment of the present invention is shown.

[0060] Icon: 100 - electronic device; 110 - memory; 120 - processor; 130 - communication module; 140 - data processing device; 141 - information acquisition module; 142 - information processing module. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0062] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] It should be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0064] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0065] It should be noted that, in the absence of conflict, the features in the embodiments of the present invention may be combined with each other.

[0066] Please refer to Figure 1, is a block diagram of an electronic device 100 provided in this embodiment. The electronic device 100 in this embodiment may be a server, a processing device, a processing platform, etc. capable of data interaction and processing. For example, the electronic device 100 may be a device for data collection, analysis and processing in an energy storage system. The electronic device 100 includes a memory 110, a processor 120 and a communication module 130. The memory 110, the processor 120 and the communication module 130 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components may be electrically connected to each other via one or more communication buses or signal lines.

[0067] The memory 110 is used to store programs or data. The memory 110 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc.

[0068] The processor 120 is used to read / write data or programs stored in the memory 110 and execute corresponding functions.

[0069] The communication module 130 is used to establish a communication connection between the electronic device 100 and other communication terminals through the network, and to send and receive data through the network.

[0070] It should be understood that Figure 1 The structure shown is only a schematic diagram of the structure of the electronic device 100. The electronic device 100 may also include Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0071] Please refer to Figure 2 , which is a flow chart of a data processing method provided by an embodiment of the present invention, and is used to implement data storage (recording and broadcasting), which can be Figure 1 The electronic device 100 executes, for example, the processor 120 in the electronic device 100. The data processing method includes S110, S120, S130 and S140.

[0072] S110, collecting data of the target object in real time.

[0073] S120, determining valid data from the collected data.

[0074] S130, performing setting processing on the valid data to obtain target data, wherein the target data includes a representation of the valid data.

[0075] S140, storing the valid data and target data in association with each other.

[0076] By determining the valid data, the storage cost waste caused by storing all data, such as invalid data, is avoided. By associating the valid data with the target data, the valid data can be quickly and reliably retrieved based on the target data that characterizes the valid data, thereby improving the retrieval efficiency.

[0077] In this embodiment, the solution of performing data processing based on S110 to S140 to realize data storage can be applied to multiple scenarios, for example, it can be applied to detection systems, monitoring systems, storage systems, etc. In this embodiment, the corresponding process is illustrated by taking application to an energy storage system as an example.

[0078] In the case of storing data for an energy storage system, the target object in S110 may be an important device or equipment in the energy storage system that needs to collect data. For example, the target object may be a battery, a power conversion system (PCS), etc. in the energy storage system. Accordingly, the data collected in S110 is the data of the battery, PCS, etc.

[0079] For different target objects, the data collection method can be flexibly selected to ensure the reliability of data collection. For example, the data of various devices, sensors and other devices in the energy storage system can be collected in real time through serial communication protocols such as Modbus, Generic Object Oriented Substation Event (GOOSE), and IEC-104.

[0080] For example, data that changes slowly (e.g., the speed of change is lower than a certain threshold) in the energy storage system, such as battery data, can be collected through Modbus. Data that changes quickly (e.g., the speed of change is higher than a certain threshold), such as PCS data, can be collected through the GOOSE protocol.

[0081] After the data of the target object is collected based on S110, the method of determining valid data from the collected data in S120 can be flexibly selected.

[0082] For example, the quality of the data can be identified, and based on the quality of the data, it can be determined whether the currently collected data is valid.

[0083] For example, data integrity checks can be performed through missing value detection, outlier detection, duplicate value detection, etc. Data consistency checks can be performed through field consistency detection, time consistency detection, logical consistency detection, etc. Data accuracy checks can be performed through checksum verification, reference data comparison, calibration, etc. Data distribution checks can be performed through statistical analysis, visual analysis, etc. Data real-time checks can be performed through timestamp detection, heartbeat detection, etc. In this way, the quality of the collected data can be effectively identified and verified, and whether the data is valid can be determined based on the quality of the data. For example, valid data screening conditions can be set to store only valid data to improve data reliability and availability, providing a solid foundation for subsequent data analysis and application.

[0084] For another example, the validity of the collected data can be determined based on the working status of the target object. Figure 3 , determining valid data from the collected data can be achieved through S121, S122 and S123.

[0085] S121, monitoring the working status of the target object.

[0086] S122, when the target object is in a normal working state, determining the collected data of the target object as valid data and recording it.

[0087] S123, when the target object switches from a normal working state to an abnormal state, the collected data of the target object is determined as invalid data, and only the timestamp of the target object data becoming invalid data is recorded.

[0088] For example, when the target object includes a sensor, if the sensor works normally, the data of the sensor is determined to be valid data and recorded. If the communication is interrupted due to sensor failure, timeout, etc., the collected data is determined to be invalid, and the invalid data is not stored, and only the timestamp of the communication interruption and the data becoming invalid data is recorded.

[0089] After valid data is determined based on S120, in S130, the valid data is set and processed, and the method of obtaining the target data can be flexibly selected. In one implementation, please refer to Figure 4 , which can be achieved through S131, S132 and S133.

[0090] S131, determining the working condition of the target object based on the valid data.

[0091] S132: Based on the working condition, classify the valid data to obtain classification information.

[0092] S133, extracting characteristic values ​​of the valid data.

[0093] For example, the working conditions of the target object can be identified based on the valid data, and the valid data can be labeled and classified based on the working conditions. By labeling and classifying the collected valid data, during subsequent data retrieval and playback, the required data can be quickly retrieved through label and classification screening, thereby accelerating playback.

[0094] In S131, the operating condition identification can be implemented flexibly. For example, the definition of the collected data can be identified, and the operating condition of the energy storage system can be identified based on the value of the data.

[0095] For example, the energy storage system operating conditions can be identified through state points, such as standby, startup, operation, fault and other operating conditions. Each operating condition can be further subdivided, such as the operating condition can be divided into charging and discharging. Through the combination of single or multiple collected data, the specific operating condition is determined, and then the collected data is labeled and classified.

[0096] It is also possible to determine in real time which state the energy storage system is currently in, such as standby or running, based on a combination of data such as the system status and PCS status. After determining the current unique operating condition, the data can be classified in advance based on the operating condition information before storage to facilitate quick retrieval later.

[0097] For another example, in order to simply and effectively represent the essence of the collected data and provide support for subsequent data processing and analysis, such as data retrieval and playback, data feature extraction can be performed on valid data, wherein the feature values ​​of the valid data may include time domain feature values, frequency domain feature values, and the like.

[0098] The time domain eigenvalues ​​can include the average value, extreme value (maximum and minimum value), variance and standard deviation, etc. The frequency domain eigenvalues ​​can obtain the frequency characteristics of the data through fast Fourier transform and record the most important frequencies. Taking the calculation of the average value as an example, a sliding window can be designed. When the valid data fills the sliding window, the average value is calculated. Before the sliding window is filled, the average value is invalid.

[0099] See also Figure 5 In S133, extracting the characteristic value of valid data can be achieved through S1331 to S1335.

[0100] S1331, pre-processing the valid data.

[0101] S1332, initialize the sliding window, and fill the pre-processed valid data into the sliding window point by point.

[0102] S1333, after the sliding window is filled with pre-processed valid data, the time domain feature value is calculated by finding the average, extreme value, variance and standard deviation.

[0103] S1334, after the sliding window is filled with pre-processed valid data, frequency domain eigenvalues ​​are calculated by fast Fourier transform.

[0104] S1335: Use the time domain eigenvalue and the frequency domain eigenvalue as the eigenvalues ​​of the valid data.

[0105] After the eigenvalues ​​of the data are calculated based on the above process, the eigenvalues ​​are associated with the data and stored to facilitate subsequent playback and analysis.

[0106] For example, based on time domain analysis, the time series characteristics of the voltage and current signals in the energy storage system during the charging and discharging process, such as mean, variance, waveform shape, peak value, etc., can be captured, so as to identify the stability and volatility of the battery charging and discharging process, as well as possible abnormal behaviors such as overcharging and over-discharging.

[0107] For example, based on frequency domain analysis, time domain signals can be converted into frequency domain to obtain characteristics such as frequency distribution, bandwidth, harmonic components, etc. of the signal. This helps to analyze the periodic fluctuations and frequency response of the energy storage system under different load conditions, and identify the resonant frequency, noise source and potential failure mode in the energy storage system.

[0108] In summary, feature extraction provides a powerful tool for subsequent state monitoring, fault diagnosis, performance evaluation, etc. of the energy storage system, which helps to ensure the safe and stable operation of the energy storage system.

[0109] When the classification information and characteristic values ​​of the valid data are obtained, in S140, the valid data and the target data are stored in association with each other, including: the classification information and the characteristic values ​​are stored in association with the corresponding valid data, so that data retrieval and playback can be achieved quickly and accurately based on the classification information and the characteristic values.

[0110] The method of associating and storing the classification information and feature values ​​with the corresponding valid data can be flexibly selected. For example, it can be stored locally. For another example, it can also be stored on a cloud platform.

[0111] On the basis of storing only valid data to avoid wasting storage space due to storing invalid data, in order to further reduce the space occupied by data storage and improve the efficiency of subsequent data retrieval and playback, in this embodiment, only the valid data collected for the first time and the valid data that has changed later can be stored, and a timestamp can be carried.

[0112] Accordingly, see Figure 6In S140, the valid data and the target data are stored in association with each other, which can be achieved through S141, S142, S143 and S144.

[0113] S141, storing the valid data and target data in association with each other in combination with a timestamp.

[0114] S142, when new valid data is obtained, determine whether the new valid data is different from the valid data stored at the last moment. If so, execute S143. If not, execute S144.

[0115] S143, in combination with the timestamp, the valid data and the target data with differences are associated and stored.

[0116] S144, abandoning the storage of the new valid data.

[0117] See also Figure 7 , provides an overall implementation process for data storage, through data collection, effective data screening, only storing the same effective data once, data classification, feature value extraction, and by storing the collected effective data on a local storage device or cloud platform, with a timestamp when recording the data, the integrity and reliability of the stored data are ensured. On the basis of the stored effective data, only the data that changes subsequently is stored, and the data that does not change is not stored, which further reduces the amount of stored data, thereby reducing the storage space, and thus ensuring the efficiency of subsequent data retrieval and playback.

[0118] Based on the implementation of data storage, see Figure 8 This embodiment also provides a data processing method for realizing data retrieval and playback based on stored data. Figure 1 The electronic device 100 executes, for example, the processor 120 in the electronic device 100. The data processing method includes S210 and S220.

[0119] S210, obtaining the working condition to be retrieved of the target object.

[0120] S220, according to the above-mentioned associated stored valid data and target data, retrieve the response data corresponding to the working condition to be retrieved.

[0121] The response data may be corresponding valid data and characteristic values.

[0122] When the target data includes classification information and feature values, data retrieval and playback can be achieved quickly and accurately based on the classification information and feature values ​​during retrieval.

[0123] When the corresponding data is retrieved, it can be used in a variety of ways based on the working conditions to be retrieved, for example, it can be applied to working condition optimization, test analysis, fault detection and processing, etc.

[0124] For example, when the working condition to be retrieved is a working condition that needs to be optimized for performance, the problem bottleneck and the corresponding key parameters, such as one or more key parameters, can be identified based on the response data corresponding to the working condition to be retrieved through working principles and data analysis. Thus, the optimization solution of the target object is determined and implemented based on the problem bottleneck and the corresponding key parameters, so as to iteratively obtain the optimal solution for the working condition to be retrieved.

[0125] For example, according to the working conditions that need to be analyzed, the recorded data is retrieved and played back. According to industry standards or historical performance, inefficient operating modes are identified, and the characteristic values ​​of key parameters such as charging and discharging efficiency and reaction time are extracted. The characteristic values ​​are analyzed to determine the performance bottleneck. An optimization plan is formulated, and the plan is tracked and iterated to achieve performance optimization of the working conditions.

[0126] For another example, when the working condition to be retrieved is a working condition that needs to be replayed in software testing, the working condition to be replayed can be selected according to the test case, and the response data corresponding to the working condition to be retrieved can be retrieved for replay. The replayed data is used as the input of the test case, and the test case is executed and analyzed.

[0127] For example, retrieve the data of the working condition to be tested, replay the data as the input of the test case, and analyze the output results of the test case. For example, by recording and replaying the startup working condition, the startup process of the device can be tested. By recording and replaying the fault working condition, the fault handling of the device can be tested.

[0128] For another example, during the pre-maintenance process, when the working condition to be retrieved is a fault working condition, such as a subdivided fault working condition when the equipment that needs pre-maintenance fails, the key parameters that affect the fault can be identified and data analysis can be performed based on the response data corresponding to the working condition to be retrieved. In order to identify the key parameters, a comparative analysis of multiple stored data may be involved. An early warning plan is formulated based on the results of the data analysis. The early warning plan is verified and the optimal plan is iterated.

[0129] For example, during equipment pre-maintenance, historical data of equipment failures are replayed and key parameters are identified. The characteristic values ​​of key parameters are analyzed to identify, predict, and classify failures. For example, for battery failures, key battery parameters such as current and temperature can be monitored. Failure types may include aging, capacity attenuation, overcharge, or over-discharge. Pre-maintenance can be used to warn equipment before it fails, so that the equipment can be repaired and replaced in advance to reduce losses.

[0130] In this embodiment, based on the data processing method that can realize recording storage, retrieval and playback, real-time monitoring, testing, performance optimization, preventive maintenance, alarm, etc. of target objects, such as equipment and components in the energy storage system, can be achieved based on data recording and playback, thereby ensuring the operational reliability of the equipment and components.

[0131] In order to execute the corresponding steps in the above embodiments and various possible methods, a data processing device implementation method is provided below. Fig. 9 , Fig. 9 A functional module diagram of a data processing device 140 provided in an embodiment of the present invention, the data processing device 140 can be applied to Figure 1 The electronic device 100 is shown. It should be noted that the basic principle and technical effect of the data processing device 140 provided in this embodiment are the same as those of the above method embodiment. For the sake of brief description, for the parts not mentioned in this embodiment, reference can be made to the corresponding contents in the above method embodiment. The data processing device 140 includes an information acquisition module 141 and an information processing module 142.

[0132] The information acquisition module 141 is used to collect data of the target object in real time.

[0133] The information processing module 142 is used to determine valid data from the collected data; perform setting processing on the valid data to obtain target data, wherein the target data includes a representation of the valid data; and associate and store the valid data and the target data.

[0134] Based on the above, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium includes a computer program, and when the computer program is executed, the electronic device where the computer-readable storage medium is located is controlled to execute the above data processing method.

[0135] By adopting the above scheme in the embodiment of the present invention, by determining the valid data, only the valid data and the target data such as classification information and characteristic values ​​are associated and stored, thereby avoiding the waste of storage space caused by storing all data, such as invalid data. Based on the target data that characterizes the valid data, the valid data can be quickly and reliably retrieved, thereby improving the retrieval efficiency. Through fast and reliable data recording and playback, there can be a variety of applications, such as analysis, debugging, verification, etc., and the working reliability of the target object in the scene can be improved through fault diagnosis, performance optimization, etc.

[0136] For example, by analyzing the data of the energy storage system, the status of the energy storage system can be monitored in real time, and abnormal conditions can be detected and alarmed in time. By analyzing the operating data of the energy storage system and evaluating the performance of the energy storage system, optimization plans can be proposed to improve efficiency and extend the life of the equipment. Based on the analysis results of the energy storage data, the remaining life of the equipment can be predicted and maintenance strategies can be planned to avoid sudden failures. By replaying and analyzing the characteristics of historical data, faults in the energy storage system can be diagnosed and future failure modes can be predicted. When developing new application software, the corresponding working conditions can be retrieved according to the test cases, and then the data can be quickly retrieved for recording and playback, so that unit testing or integration testing can be implemented to meet the corresponding needs.

[0137] In several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0138] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0139] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0140] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data processing method, characterized in that: include: Collect data of target objects in real time; Determining valid data from the collected data; Performing setting processing on the valid data to obtain target data, wherein the target data includes a representation of the valid data; The valid data and the target data are stored in association with each other.

2. The data processing method according to claim 1, characterized in that: Determining valid data from the collected data includes: Determine valid data based on the data quality of the collected data; or, Monitoring the working status of the target object; When the target object is in a normal working state, determining the collected data of the target object as valid data and recording it; When the target object switches from a normal working state to an abnormal state, the collected data of the target object is determined to be invalid data, and only the timestamp when the data of the target object becomes invalid data is recorded.

3. The data processing method according to claim 1, characterized in that: The setting process of the valid data to obtain target data includes: Based on the valid data, determining the working condition of the target object; Based on the working condition, classify the valid data to obtain classification information; and / or, Extracting characteristic values ​​of the valid data; Using the classification information and / or feature value as target data; The storing the valid data and the target data in association with each other includes: The classification information and / or feature value are associated with corresponding valid data and stored.

4. The data processing method according to claim 3, characterized in that: The step of extracting the characteristic value of the valid data comprises: Preprocessing the valid data; Initialize the sliding window and fill the preprocessed valid data into the sliding window point by point; After the sliding window is filled with preprocessed valid data, the time domain characteristic value is calculated by finding the average, extreme value, variance and standard deviation; After the sliding window is filled with pre-processed valid data, the frequency domain eigenvalue is calculated by fast Fourier transform; The time domain eigenvalue and the frequency domain eigenvalue are used as the eigenvalues ​​of the valid data.

5. The data processing method according to any one of claims 1 to 4, characterized in that: The storing the valid data and the target data in association with each other includes: In combination with the timestamp, the valid data and the target data are stored in association; When new valid data is obtained, it is determined whether the new valid data is different from the valid data stored at the previous moment; If so, the valid data and target data with differences are stored in association with each other in combination with the timestamp; If not, the storage of the new valid data is abandoned.

6. A data processing method, characterized in that: include: Obtain the working condition to be retrieved of the target object; According to the valid data and target data associated with the stored data in the data processing method according to any one of claims 1 to 5, the response data corresponding to the working condition to be retrieved is retrieved.

7. The data processing method according to claim 6, characterized in that: The method further comprises: When the working condition to be retrieved is a working condition that needs to be optimized, identifying the problem bottleneck and the corresponding key parameters according to the response data corresponding to the working condition to be retrieved; Determine and implement the optimization solution of the target object based on the problem bottleneck and the corresponding key parameters, so as to iteratively obtain the optimal solution of the working condition to be retrieved; and / or, When the working condition to be retrieved is a working condition that needs to be replayed, replaying is performed according to the response data corresponding to the working condition to be retrieved; Using the replayed data as input to a test case, executing the test case and analyzing it; and / or, When the working condition to be retrieved is a fault working condition, identifying key parameters affecting the fault and performing data analysis according to the response data corresponding to the working condition to be retrieved; Formulate an early warning plan based on the results of the data analysis; The early warning scheme is verified and the optimal scheme is iterated.

8. A data processing device, characterized in that: include: An information acquisition module is used to collect data of the target object in real time; An information processing module, used to determine valid data from the collected data; The valid data is set and processed to obtain target data, wherein the target data includes a representation of the valid data; and the valid data and the target data are associated and stored.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the data processing method according to any one of claims 1 to 7 when executing the program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a computer program, and when the computer program is executed, the electronic device where the computer-readable storage medium is located is controlled to execute the data processing method according to any one of claims 1 to 7.

Images