Acquired data preprocessing method and system of engineering machinery and medium

Abstract

The invention relates to the technical field of data preprocessing, and discloses a preprocessing method and system for collected data of engineering machinery and a medium, the method comprises the following steps: establishing a database, dividing a corresponding data group for each target data collection unit of the engineering machinery in the database, and establishing a data index linked list; storing the data acquired by each target data acquisition unit into a corresponding data index linked list; performing first verification through the same type of data collected according to the time sequence in the same data group, and determining to-be-verified items and item time generated by data change in the data group; performing cross-data-group second verification to verify whether the to-be-verified items and the corresponding item time in each data group are correct or not; and if the to-be-verified item in the data group and the corresponding item time are wrong, performing abnormal data marking in the attribute unit to identify abnormal data. According to the invention, the technical problems of data noise, many abnormal values and many data samples in the collected data of the engineering machinery can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of data preprocessing, and in particular to a method for preprocessing collected data of an engineering machinery, a data preprocessing system and a readable storage medium. Background Art

[0002] Due to the complexity of its working conditions and the difficulty of construction, construction machinery often stops working and fails. The shutdown of construction machinery will lead to a series of losses, such as equipment limitation losses, project progress delays, and high emergency maintenance and repair costs. In addition, resuming work after emergency maintenance may result in further damage to the equipment if the failure is not completely eliminated.

[0003] Construction machinery (such as excavators, cranes, road rollers, etc.) collects a large amount of raw data due to the complexity of their structure, components and construction environment. The data types include mechanical system data, hydraulic system data, electrical and control system data, operating condition data, and environmental data.

[0004] By analyzing and monitoring the above data, it is possible to monitor the equipment operation status of construction machinery and the health status of each component, so that manufacturers can track the status of the equipment in a timely manner and perform after-sales maintenance in a timely manner to reduce construction machinery downtime and failures.

[0005] However, there are multiple difficulties in data collection for construction machinery. For example, construction machinery often works under complex working conditions. The harsh environment (for example, high temperature, high humidity, and dust) makes the sensors susceptible to interference, and the coupling of multiple physical fields may cause the mechanical, hydraulic, and electrical data to affect each other (for example, engine vibration interferes with the hydraulic pressure signal). In addition, construction machinery often has the characteristics of data heterogeneity and multi-source data fusion, which requires the simultaneous processing of time series data, image data, and discrete events.

[0006] At the same time, engineering machinery has high requirements for data real-time collection, and the on-board hardware edge computing resources are limited. Engineering machinery also collects large amounts of data (for example, a full sensor sampling of an excavator can generate dozens of GB of data per day), resulting in high long-term storage costs.

[0007] Therefore, the collected data of construction machinery have technical problems such as data noise, many outliers, and many data samples, which cause great difficulties in the analysis process of directly using the original data for data analysis, resulting in inaccurate data analysis results. Summary of the invention

[0008] The main purpose of the present invention is to provide a method, system and medium for preprocessing collected data of engineering machinery, aiming to solve the problem in the prior art that the collected data of engineering machinery has a lot of data noise and outliers, and a large number of data samples, which creates great difficulties for the analysis process of directly using the original data for data analysis, resulting in inaccurate data analysis results.

[0009] To achieve the above object, the present invention provides a method for preprocessing collected data of an engineering machinery, comprising the following steps: The steps include: Establishing a database, dividing a corresponding data group for each target data acquisition unit of the engineering machinery in the database, and establishing a data index linked list for each data group, wherein the data index linked list includes a data unit for storing each original data, a time sequence unit for marking the time sequence of each original data, and an attribute unit for marking the attribute of each original data; Acquire the data collected by each target data collection unit, and store the data collected by each target data collection unit into a corresponding data index linked list; By using the same type of data collected in the same data group in a time series, a first verification is performed on the data in the same data group, so as to determine the items to be verified generated by the data change in the data group and the item time corresponding to the items to be verified according to the change of the same type of data in the same data group; Through the items to be verified and the item time in each data group, a second cross-data group verification is performed to verify whether the items to be verified and the corresponding item time in each data group are correct; If the items to be verified and the corresponding item time in the data group are wrong, the wrong data corresponding to the items to be verified and the corresponding item time in the data group are marked as abnormal data in the attribute unit to identify the abnormal data; The cloud server obtains abnormal data from the database to perform fault analysis on the construction machinery.

[0010] Optionally, the method further includes: The data that does not generate items to be verified in the first verification is temporarily marked as data in the attribute unit; If the items to be verified and the corresponding item time in the data group are correct, the correct data corresponding to the items to be verified and the corresponding item time in the data group will be marked as normal data in the attribute unit.

[0011] Optionally, before the step of performing a first verification on the data in the same data group by using the same type of data collected in time series in the same data group to determine the items to be verified generated by the data changes in the data group and the item time corresponding to the items to be verified according to the changes of the same type of data in the same data group, the following steps are included: According to the current working conditions, multiple data groups with set strong association relationships are divided into the same association group; one data group is selected from the association group as the trust group, and the other data groups are selected as the follow-up groups; Calculate the matching of the follow-up group and the trust group, generate a corresponding trust factor for the follow-up group whose matching with the trust group reaches a preset condition, and generate a corresponding penalty factor for the follow-up group whose matching with the trust group reaches a preset deviation condition; According to the penalty factor, the data of the data group corresponding to the penalty factor is corrected.

[0012] Optionally, the step of acquiring the data collected by each target data collection unit and storing the data collected by each target data collection unit into a corresponding data index linked list includes: Acquire newly collected data from each target data collection unit; Generate data units, timing units and attribute units corresponding to the newly collected data in the data index linked list; The newly collected data is stored in the newly generated data unit, and the time sequence corresponding to the newly collected data is stored in the newly generated time sequence unit.

[0013] Optionally, the step of performing a first verification on the data in the same data group by using the same type of data collected in time series in the same data group to determine the items to be verified generated by the data changes in the data group and the item time corresponding to the items to be verified according to the changes of the same type of data in the same data group includes: Obtaining a preset interval set for the same data group, wherein each interval set includes at least one data interval, each data interval is used to represent different types of items to be verified, and the data intervals included in the preset interval set for the same data group are different; For the data in the same data group, each data interval is used for verification, so that the data in the same data interval in the continuous time period are combined into the same item to be verified in the data group; The collection time corresponding to the data of the same item to be verified that is merged into the data group is merged to determine the item time corresponding to the item to be verified.

[0014] Optionally, the step of performing a second verification across data groups through the items to be verified and the item time in each data group to verify whether the items to be verified and the corresponding item time in each data group are correct includes: Calculate the overlap of the items to be verified and the item time in each data group, and determine whether the overlap is lower than a preset value; If yes, the time of the items to be verified and the corresponding items in the data group are wrong; If not, the items to be verified in the data group and the corresponding item time are correct.

[0015] Optionally, the following method is used to determine the items to be verified caused by the data changes in the data group and the item time corresponding to the items to be verified: Get the i A set of preset intervals for each data set ;in, Indicates i The first j Data intervals, , J For the i The number of data intervals in the interval set of data groups, ; Get the i The data sequence formed by continuous sampling of data groups ,in, For the i The first of the data sequences of consecutive samples of the data group m Sample data, , M For the i The number of data in a data sequence of consecutive samples of a data group; Calculate the i Each sampled data in the data sequence of continuous samples of data groups The label of the data interval to which it belongs ,Right now Indicates i The interval label corresponding to the mth data of a data group: ; The first i The data sequence formed by continuous sampling of data groups is converted into an interval label sequence ; Scan the interval label sequence and find the continuous subsequence with the same label; The items to be verified preset in the data interval corresponding to the continuous subsequences with the same label in the data group are obtained, and the data sampling time interval corresponding to the continuous subsequences with the same label in the data group is obtained as the item time corresponding to the items to be verified.

[0016] Optionally, refer to the following method to verify whether the items to be verified and the corresponding item time in each data group are correct: For items to be verified of the same type across data groups, the same interval number is preset; According to the overlap of interval labels in the interval label sequences of different data groups, verify whether the items to be verified and the corresponding item time in each data group are correct, specifically: ; in, Indicates i The interval label and the k The overlap of the interval labels of the data groups; Indicates k The first m The interval label corresponding to the data is Indicates i The first m The interval number and k The first m The indicator function of the interval labels, When the indicator function is established, the result is 1. When it is not true, the result of the indicator function is 0; ; in, ,and , , N is the number of data sets; like , it means that the items to be verified in the data group and the corresponding item time are wrong; like , it means that the items to be verified and the corresponding item time in the data group are correct; among them, A is the preset coincidence parameter.

[0017] To achieve the above-mentioned purpose, the present invention further proposes a data preprocessing system, wherein the data preprocessing system adopts the collected data preprocessing method of the engineering machinery to preprocess the data collected by the engineering machinery.

[0018] To achieve the above-mentioned purpose, the present invention further proposes a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for preprocessing collected data of the engineering machinery are implemented.

[0019] The technical solution of the present invention is conducive to solving the problem that the data collected by engineering machinery in the prior art has many data noises and outliers, and many data samples, which makes it very difficult to directly analyze the data using the original data, resulting in inaccurate data analysis results. The specific analysis is as follows: a data group is established for each target data collection unit in the database, and a data index linked list is established for each data group. Among them, in each data index linked list, the real-time data newly collected by the corresponding target data collection unit is stored in the data unit, and the time sequence corresponding to the real-time data is recorded in the time sequence unit. Therefore, in the database structure, the time sequence and original data of each target data collection unit collecting real-time data are retained, so that the real-time data is recorded in an orderly manner in the data group. Further, by means of a first check, the data change situation is checked from the unified data group, and it is determined whether there are items to be checked in the data change according to the data change situation, and the time of the items to be checked is obtained. At the same time, by performing a second check across data groups, it is judged whether the items to be checked and the time of the items in each data group are correct. Therefore, through the data changes in the same data group, the operation and construction conditions of the engineering machinery, as well as the corresponding time of the operation and construction conditions can be found. Through the second cross-group verification, it can be found whether other data groups also feedback the same operation and construction conditions of the engineering machinery, and through cross-data group verification, the operation and construction conditions between different data groups can be mutually verified. Data that cannot be verified with other data groups will be identified as erroneous data, and the attributes of abnormal data will be marked in the data index linked list. Therefore, through data groups and data index linked lists, the original data of the engineering machinery can be fully recorded, and the abnormal data can be marked. The cloud server only needs to obtain abnormal data for fault analysis, which can not only ensure the storage of complete time series data within a short period of time, but also ensure that only abnormal data is analyzed separately when analyzing data. The original data is stored according to different functions. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic flow chart of the method for preprocessing collected data of engineering machinery in the first embodiment of the present invention; Figure 2 A schematic diagram of establishing a data index linked list for each data group in a database in the present invention; Figure 3 In the present invention, i Schematic diagram of converting a data sequence formed by continuous sampling of data groups into an interval label sequence; Figure 4 A schematic diagram of calculating the overlap of interval labels in interval label sequences of different data groups in the present invention; Figure 5This is a schematic diagram of the target data collection unit of the engineering machinery in the present invention adding the collected data to the database.

[0021] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0022] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0023] In the following description, suffixes such as "unit", "component" or "unit" used to represent elements are only used to facilitate the description of the present invention, and have no specific meanings. Therefore, "unit", "component" or "unit" can be used in a mixed manner.

[0024] See also Figures 1 to 5 In a first embodiment of the present invention, a method for preprocessing collected data of an engineering machinery is provided, comprising the following steps: Step S10, establishing a database, dividing a corresponding data group for each target data acquisition unit of the engineering machinery in the database, and establishing a data index linked list for each data group, wherein the data index linked list includes a data unit for storing each original data, a time sequence unit for marking the time sequence of each original data, and an attribute unit for marking the attribute of each original data; Step S20, acquiring the data collected by each target data collection unit, and storing the data collected by each target data collection unit into a corresponding data index linked table; Step S30, performing a first check on the data in the same data group by using the same type of data collected in time series in the same data group, so as to determine the items to be checked generated by the data changes in the data group and the item time corresponding to the items to be checked according to the changes of the same type of data in the same data group; Step S40, performing a second cross-data group verification through the items to be verified and the item time in each data group, to verify whether the items to be verified and the corresponding item time in each data group are correct; Step S50, if the items to be verified and the corresponding item time in the data group are wrong, the wrong data corresponding to the items to be verified and the corresponding item time in the data group are marked as abnormal data in the attribute unit to identify the abnormal data; In step S60, the cloud server obtains abnormal data from the database to perform fault analysis on the construction machinery.

[0025] The technical solution of the present invention is conducive to solving the problem that the data collected by engineering machinery in the prior art has many data noises and outliers, and many data samples, which makes it very difficult to directly analyze the data using the original data, resulting in inaccurate data analysis results. The specific analysis is as follows: a data group is established for each target data collection unit in the database, and a data index linked list is established for each data group. Among them, in each data index linked list, the real-time data newly collected by the corresponding target data collection unit is stored in the data unit, and the time sequence corresponding to the real-time data is recorded in the time sequence unit. Therefore, in the database structure, the time sequence and original data of each target data collection unit collecting real-time data are retained, so that the real-time data is recorded in an orderly manner in the data group. Further, by means of a first check, the data change situation is checked from the unified data group, and it is determined whether there are items to be checked in the data change according to the data change situation, and the time of the items to be checked is obtained. At the same time, by performing a second check across data groups, it is judged whether the items to be checked and the time of the items in each data group are correct. Therefore, through the data changes in the same data group, the operation and construction conditions of the engineering machinery, as well as the corresponding time of the operation and construction conditions can be found. Through the second cross-group verification, it can be found whether other data groups also feedback the same operation and construction conditions of the engineering machinery, and through cross-data group verification, the operation and construction conditions between different data groups can be mutually verified. Data that cannot be verified with other data groups will be identified as erroneous data, and the attributes of abnormal data will be marked in the data index linked list. Therefore, through data groups and data index linked lists, the original data of the engineering machinery can be fully recorded, and the abnormal data can be marked. The cloud server only needs to obtain abnormal data for fault analysis, which can not only ensure the storage of complete time series data within a short period of time, but also ensure that only abnormal data is analyzed separately when analyzing data. The original data is stored according to different functions.

[0026] The data collected in real time by construction machinery (such as excavators, cranes, loaders, etc.), especially the data related to fault monitoring, usually covers the key parameters of mechanical, hydraulic, electrical, thermal and other systems.

[0027] Therefore, the target data acquisition unit in the present application refers to a component used to collect key working parameters of engineering machinery.

[0028] For example: the target data acquisition unit can be an acceleration sensor for collecting vibration velocity or acceleration; the target data acquisition unit can also be a pressure sensor for detecting the pressure of hydraulic components; or, the target data acquisition unit can be a flow meter for detecting oil flow; for another example, the target data acquisition unit can be a temperature sensor for detecting motor temperature, etc.

[0029] Meanwhile, the target data acquisition unit may be a component for detecting environment and operation data, for example, a detection component for GPS data, posture data, handle signal, and pedal signal.

[0030] In the real-time monitoring data of construction machinery, different types of work data are strongly related to their working status and the operations performed. Through multi-sensor data fusion and pattern recognition, the real-time status of the machinery (such as normal operation, no load, overload, failure, etc.) and specific operations (such as excavation, lifting, rotation, etc.) can be accurately judged.

[0031] And based on the data changes in the same data group, the corresponding work items can also be identified.

[0032] For example, when the construction machinery is an excavator, one of the target data acquisition units is a pressure sensor that monitors changes in the main pump pressure (the main pump pressure here takes the outlet pressure as an example), and the real-time data of the main pump pressure is stored in the data index linked list of the corresponding data group, the operating status of the excavator can be determined by the changes in the main pump pressure.

[0033] For example, when the main pump pressure is [0MPa, 5MPa], the corresponding items to be checked may be: no-load standby operation; When the main pump pressure is (5MPa, 15MPa], the corresponding items to be checked may be: light load operation; When the main pump pressure is (15MPa, 25MPa], the corresponding items to be checked may be: normal operating load; When the main pump pressure is (25MPa, 30MPa], the corresponding items to be checked may be: heavy load operation; When the main pump pressure exceeds 30MPa, the corresponding items to be checked may be: overpressure state; When the pressure fluctuation rate of the main pump reaches the set rate, the corresponding items to be checked may be: sudden load change or pump failure.

[0034] For example, when the construction machinery is an excavator, and the other target data acquisition unit is a flow meter that monitors the changes in the main pump flow, when the real-time data of the main pump flow is stored in the data index linked list of the corresponding data group, the excavator operating status can also be determined by the changes in the main pump flow.

[0035] For example, when the main pump flow rate is (20L / min, 50 L / min], the corresponding items to be checked may be: no-load standby operation; When the main pump flow is (50 L / min, 120 L / min], the corresponding items to be checked may be: light load operation; When the main pump flow is (120L / min, 180L / min], the corresponding items to be checked may be: normal operating load; When the main pump flow is (180L / min, 200 L / min], the corresponding items to be checked may be: heavy-load operation; When the main pump flow is (0 L / min, 20 L / min], the corresponding items to be checked may be: system blockage; When the flow fluctuation rate of the main pump reaches the set rate, the corresponding items to be checked may be: sudden load change or pump failure.

[0036] The above data are only examples. Specifically, the corresponding relationship between the parameter range and the operating status in the actual operation of each construction machinery can be measured, and the data ranges of each data group can be determined according to the measured results.

[0037] Therefore, through the first verification, the items to be verified caused by the data changes in the same data group and the item time corresponding to the items to be verified can be determined according to the data changes in the same data group.

[0038] Furthermore, the second verification is a verification across data groups. That is, the various items to be verified and the time of the items obtained in each data group are matched with the various items to be verified and the time of the items obtained in the remaining other data groups to determine whether the various items to be verified and the time of the items obtained in the data group are correct. If correct, it usually means that the system is operating normally. If wrong, it means that the corresponding target data acquisition unit is faulty, or the target element used to monitor by the target data acquisition unit is faulty, and the wrong data needs to be analyzed for faults.

[0039] Based on the first embodiment of the method for preprocessing collected data of an engineering machine of the present invention, in a second embodiment of the method for preprocessing collected data of an engineering machine of the present invention, the method further includes: Step S70, temporarily marking the data that does not generate items to be verified in the first verification in the attribute unit; Step S80, if the items to be verified and the corresponding item time in the data group are correct, the correct data corresponding to the items to be verified and the corresponding item time in the data group are marked as normal data in the attribute unit.

[0040] In the first embodiment, various data that generate items to be verified are listed. In actual engineering applications, that is, in data analysis, it is not necessary to set corresponding items to be verified for each interval of monitoring data (for example, light load operation may not be used as an item to be verified), thereby reducing the data processing pressure of the system.

[0041] For example, when one of the data groups is the main pump pressure, only the following items can be set for the main pump pressure to be 25-30MPa, the main pump pressure to be more than 30MPa, and the main pump pressure fluctuation rate to reach the set rate, respectively.

[0042] Therefore, data that is not in the above range is data that does not generate items to be verified, and temporary data marking is performed in the attribute unit corresponding to the data that does not generate items to be verified.

[0043] Temporary data mark means that these data are unimportant data set by after-sales personnel or manufacturers. The data can be cleared according to the set temporary data clearing cycle to retain the system's storage capacity.

[0044] At the same time, the items to be verified and the time of the items obtained in each data group are matched with the items to be verified and the time of the items obtained in the remaining other data groups to determine that the items to be verified and the time of the items obtained in the data group are correct. This usually means that the system is operating normally during these data collection cycles, or that the monitoring object corresponding to the data group is working normally. At this time, these data are marked as normal data in the corresponding attribute unit. Normal data makes it convenient for after-sales personnel to retrieve data in a timely manner and have a real-time understanding of the equipment operation status of the construction machinery. Similarly, data can also be cleared according to the set normal data cleaning cycle to retain the system's storage capacity.

[0045] At the same time, according to the attributes in each data group marked as normal and abnormal, it is also possible to understand the normal operation time cycle and abnormal operation cycle of the equipment. In addition, the items to be verified and the time of the items, as well as the normal operation cycle and the abnormal operation cycle, can also be used to generate data analysis results on which items to be verified are prone to abnormal operation status of the equipment, so as to facilitate equipment failure cause analysis and equipment failure tracking.

[0046] Based on the first embodiment of the method for preprocessing collected data of an engineering machine of the present invention, in a third embodiment of the method for preprocessing collected data of an engineering machine of the present invention, before step S30, the following steps are included: Step S90, according to the current working condition, multiple data groups with a set strong association relationship are divided into the same association group; one data group is selected from the association group as a trust group, and the other data groups are selected as follow-up groups; Step S100, calculating the matching situation between the follower group and the trust group, generating a corresponding trust factor for the follower group whose matching situation with the trust group reaches a preset condition, and generating a corresponding penalty factor for the follower group whose matching situation with the trust group reaches a preset deviation condition; Step S110: correcting the data of the data group corresponding to the penalty factor according to the penalty factor.

[0047] Specifically, the multiple data groups having a set strong correlation relationship refer to the strong correlation in which the physical parameters have a calculation relationship.

[0048] For example, pump power = main pump inlet and outlet pressure difference × main pump flow rate, that is, , P is the pump power, The pressure difference between the inlet and outlet of the main pump, Main pump flow.

[0049] At this time, the main pump inlet pressure data group and the main pump outlet pressure data group can be used as a trust group (of course, a group of data groups can be at least one data group, here there are two data groups), and the main pump inlet and outlet pressure difference data can be calculated, because it directly reflects the load and there is a theoretical calculation formula with the main pump flow.

[0050] Specifically, the trust factor and penalty factor are calculated as follows: (1) Main pump inlet pressure data group And the main pump outlet pressure data group For the trust group, the main pump flow Q is the follow-up group; the constraints of the theoretical strong correlation are: × Q ≤ rated power.

[0051] (2) Calculate the real-time theoretical main pump flow : ; (3) Calculate the actual main pump flow Theoretical main pump flow The matching degree : ; The trust group and follow group include M Sample data, m Represents the sequence number of the sampled data, , To follow the group m The actual main pump flow rate, According to the trust group m The first in the follower group is obtained by calculating the pressure difference between the inlet and outlet pressures of the first main pump m Theoretical main pump flow.

[0052] When the matching situation with the trust group reaches the preset condition, a mapping relationship table between the preset matching degree and the trust factor is prepared, and the value of the trust factor is determined according to the specific value of the matching degree; When the matching situation with the trust group does not meet the preset conditions, a mapping relationship table between the preset matching degree and the penalty factor is prepared, and the value of the penalty factor is determined according to the specific value of the matching degree.

[0053] Specifically, when the penalty factor belongs to the set value range, the attribute unit corresponding to the corresponding follow-up group data can be marked as abnormal data. In this case, the first check and the second check are not required for the data group.

[0054] Based on the first embodiment of the method for preprocessing collected data of an engineering machine of the present invention, in a fourth embodiment of the method for preprocessing collected data of an engineering machine of the present invention, the step S20 includes: Step S21, acquiring the newly collected data of each target data collection unit; Step S22, generating data units, time sequence units and attribute units corresponding to the newly collected data in the data index linked list; Step S23, storing the newly collected data into the newly generated data unit, and storing the time sequence corresponding to the newly collected data into the newly generated time sequence unit.

[0055] Specifically, when the data index linked list generates newly collected data, it generates data units, time sequence units and attribute units for the newly collected data, thereby achieving automatic matching between the length of the data index linked list and the amount of collected data.

[0056] The timing unit is used to ensure that the order of data is recorded correctly. The attribute unit is used to mark the data type to achieve automatic sorting and automatic classification marking of data. This allows the cloud server to call the required type of data for analysis, and also facilitates subsequent data classification storage.

[0057] Based on the first embodiment of the method for preprocessing collected data of an engineering machine of the present invention, in a fifth embodiment of the method for preprocessing collected data of an engineering machine of the present invention, the step S30 includes: Step S31, obtaining a preset interval set for the same data group, wherein each interval set includes at least one data interval, each data interval is used to represent different types of items to be verified, and the data intervals included in the preset interval set for the same data group are different; Step S32, verifying the data in the same data group using each data interval, so as to combine the data in the same data interval within a continuous period into the same item to be verified in the data group; Step S33, merging the collection time corresponding to the data of the same item to be verified that are merged into the data group to determine the item time corresponding to the item to be verified.

[0058] Specifically, for the same data group, each data interval in the interval set corresponds to an operating state of the construction machinery. The interval set of the same data group may contain one or more data intervals, and the number of data intervals in the interval set of the same data group is related to the preset monitored working state. For example, when the background only needs to obtain real-time data of heavy-load operations, the interval set may include only one data interval, and this data interval can correspond to the data interval of heavy-load operations. For another example, when the background needs to obtain real-time data of no-load standby operations and heavy-load operations, the interval set includes two data intervals, one of which corresponds to the data interval of no-load standby operations, and the other corresponds to the data interval of heavy-load operations.

[0059] Based on the fifth embodiment of the method for preprocessing collected data of an engineering machine of the present invention, in a sixth embodiment of the method for preprocessing collected data of an engineering machine of the present invention, the step S40 includes: Step S41, calculating the overlap of the item types and item times across the data groups for the items to be checked and the item times in each data group, and determining whether the overlap is lower than a preset value; If yes, execute step S42: the items to be verified in the data group and the corresponding item time are wrong; If not, execute step S43: the items to be verified in the data group and the corresponding item time are correct.

[0060] Based on the sixth embodiment of the method for preprocessing collected data of an engineering machine of the present invention, in the seventh embodiment of the method for preprocessing collected data of an engineering machine of the present invention, the following method is adopted to determine the items to be verified generated by the data change in the data group and the item time corresponding to the items to be verified: Get the i A set of preset intervals for each data set ;in, Indicates i The first j Data intervals, , J For the i The number of data intervals in the interval set of data groups, ; Get the i The data sequence formed by continuous sampling of data groups ,in, For the i The first of the data sequences of consecutive samples of the data group m Sample data, , M For the iThe number of data in a data sequence of consecutive samples of a data group; Calculate the i Each sampled data in the data sequence of continuous samples of data groups The label of the data interval to which it belongs ,Right now Indicates i The interval label corresponding to the mth data of a data group: ; The first i The data sequence formed by continuous sampling of data groups is converted into an interval label sequence ; Scan the interval label sequence and find the continuous subsequence with the same label; The items to be verified preset in the data interval corresponding to the continuous subsequences with the same label in the data group are obtained, and the data sampling time interval corresponding to the continuous subsequences with the same label in the data group is obtained as the item time corresponding to the items to be verified.

[0061] Based on the seventh embodiment of the method for preprocessing collected data of an engineering machine of the present invention, in the eighth embodiment of the method for preprocessing collected data of an engineering machine of the present invention, the following method is used to verify whether the items to be verified and the corresponding item time in each data group are correct: For items to be verified of the same type across data groups, the same interval number is preset; According to the overlap of interval labels in the interval label sequences of different data groups, verify whether the items to be verified and the corresponding item time in each data group are correct, specifically: ; in, Indicates i The interval label and the k The overlap of the interval labels of the data groups; Indicates k The data set m The interval label corresponding to the data is Indicates i The data set m The interval number and k The data set m The indicator function of the interval labels, When the indicator function is established, the result is 1. If it is not true, the result of the indicator function is 0; ; in, ,and , ,N is the number of data sets; like , it means that the items to be verified in the data group and the corresponding item time are wrong; like , it means that the items to be verified and the corresponding item time in the data group are correct; among them, A is the preset coincidence parameter.

[0062] It should be noted that A Represents the minimum allowed overlap. If it is equal to or lower than the minimum overlap, it is determined that the data changes of this data group cannot be mutually verified with the data changes of other data groups. When the minimum overlap can be reached, it is within the overlap range allowed by the system.

[0063] Furthermore, as another extended embodiment, the normal data may be further marked: Specifically, set another standard overlap C , standard overlap C Greater than A ; , then it means i The data groups have good overlap with other data groups, and good data are further marked in the corresponding attribute units.

[0064] , then it means i The overlap between each data group and other data groups is normal, and general data marking is further performed in the corresponding attribute units.

[0065] This allows good data to be distinguished and marked from general data.

[0066] To achieve the above-mentioned purpose, the present invention further proposes a data preprocessing system, wherein the data preprocessing system adopts the collected data preprocessing method of the engineering machinery to preprocess the data collected by the engineering machinery.

[0067] To achieve the above-mentioned purpose, the present invention further proposes a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for preprocessing collected data of the engineering machinery are implemented.

[0068] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, or by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device to enter the method described in each embodiment of the present invention.

[0069] In the description of this specification, the description with reference to the terms "an embodiment", "another embodiment", "other embodiments", or "first embodiment to Xth embodiment" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, method steps or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0070] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0071] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0072] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for preprocessing collected data of engineering machinery, characterized in that: The steps include: Establishing a database, dividing a corresponding data group for each target data acquisition unit of the engineering machinery in the database, and establishing a data index linked list for each data group, wherein the data index linked list includes a data unit for storing each original data, a time sequence unit for marking the time sequence of each original data, and an attribute unit for marking the attribute of each original data; Acquire the data collected by each target data collection unit, and store the data collected by each target data collection unit into a corresponding data index linked list; By using the same type of data collected in the same data group in a time series, a first verification is performed on the data in the same data group, so as to determine the items to be verified generated by the data change in the data group and the item time corresponding to the items to be verified according to the change of the same type of data in the same data group; Through the items to be verified and the item time in each data group, a second cross-data group verification is performed to verify whether the items to be verified and the corresponding item time in each data group are correct; If the items to be verified and the corresponding item time in the data group are wrong, the wrong data corresponding to the items to be verified and the corresponding item time in the data group are marked as abnormal data in the attribute unit to identify the abnormal data; The cloud server obtains abnormal data from the database to perform fault analysis on the construction machinery.

2. The method for preprocessing collected data of engineering machinery according to claim 1, characterized in that: The method further comprises: The data that does not generate items to be verified in the first verification is temporarily marked as data in the attribute unit; If the items to be verified and the corresponding item time in the data group are correct, the correct data corresponding to the items to be verified and the corresponding item time in the data group will be marked as normal data in the attribute unit.

3. The method for preprocessing collected data of engineering machinery according to claim 1, characterized in that: Before the step of performing a first verification on the data in the same data group by using the same type of data collected in time series in the same data group to determine the items to be verified caused by the data changes in the data group and the item time corresponding to the items to be verified according to the changes of the same type of data in the same data group, the following steps are included: According to the current working conditions, multiple data groups with set strong association relationships are divided into the same association group; one data group is selected from the association group as the trust group, and the other data groups are selected as the follow-up groups; Calculate the matching of the follow-up group and the trust group, generate a corresponding trust factor for the follow-up group whose matching with the trust group reaches a preset condition, and generate a corresponding penalty factor for the follow-up group whose matching with the trust group reaches a preset deviation condition; According to the penalty factor, the data of the data group corresponding to the penalty factor is corrected.

4. The method for preprocessing collected data of engineering machinery according to claim 1, characterized in that: The step of acquiring the data collected by each target data collection unit and storing the data collected by each target data collection unit into a corresponding data index linked table includes: Acquire newly collected data from each target data collection unit; Generate data units, timing units and attribute units corresponding to the newly collected data in the data index linked list; The newly collected data is stored in the newly generated data unit, and the time sequence corresponding to the newly collected data is stored in the newly generated time sequence unit.

5. The method for preprocessing collected data of engineering machinery according to claim 1, characterized in that: The step of performing a first verification on the data in the same data group by using the same type of data collected in time series in the same data group to determine the items to be verified caused by the data change in the data group and the item time corresponding to the items to be verified according to the change of the same type of data in the same data group, includes: Obtaining a preset interval set for the same data group, wherein each interval set includes at least one data interval, each data interval is used to represent different types of items to be verified, and the data intervals included in the preset interval set for the same data group are different; For the data in the same data group, each data interval is used for verification, so that the data in the same data interval in the continuous time period are combined into the same item to be verified in the data group; The collection time corresponding to the data of the same item to be verified that is merged into the data group is merged to determine the item time corresponding to the item to be verified.

6. The method for preprocessing collected data of engineering machinery according to claim 5, characterized in that: The step of performing a second verification across data groups by using the items to be verified and the item time in each data group to verify whether the items to be verified and the corresponding item time in each data group are correct includes: Calculate the overlap of the items to be verified and the item time in each data group, and determine whether the overlap is lower than a preset value; If yes, the time of the items to be verified and the corresponding items in the data group are wrong; If not, the items to be verified in the data group and the corresponding item time are correct.

7. The method for preprocessing collected data of engineering machinery according to claim 6, characterized in that: The following method is used to determine the items to be verified caused by the data changes in the data group and the item time corresponding to the items to be verified: Get the i A set of preset intervals for each data set ;in, Indicates i The first j Data intervals, , J For the i The number of data intervals in the interval set of data groups, ; Get the i The data sequence formed by continuous sampling of data groups ,in, For the i The first of the data sequences of consecutive samples of the data group m Sample data, , M For the i The number of data in a data sequence of consecutive samples of a data group; Calculate the i Each sampled data in the data sequence of continuous samples of data groups The label of the data interval to which it belongs ,Right now Indicates i The interval label corresponding to the mth data of a data group: ； The first i The data sequence formed by continuous sampling of data groups is converted into an interval label sequence ; Scan the interval label sequence and find the continuous subsequence with the same label; The items to be verified preset in the data interval corresponding to the continuous subsequences with the same label in the data group are obtained, and the data sampling time interval corresponding to the continuous subsequences with the same label in the data group is obtained as the item time corresponding to the items to be verified.

8. The method for preprocessing collected data of engineering machinery according to claim 7, characterized in that: Refer to the following method to verify whether the items to be verified and the corresponding item time in each data group are correct: For items to be verified of the same type across data groups, the same interval number is preset; According to the overlap of interval labels in the interval label sequences of different data groups, verify whether the items to be verified and the corresponding item time in each data group are correct, specifically: ； in, Indicates i The interval label and the k The overlap of the interval labels of the data groups; Indicates k The first m The interval label corresponding to the data is Indicates i The data set m The interval number and k The first m The indicator function of the interval labels, When the indicator function is established, the result is 1. When it is not true, the result of the indicator function is 0; ； in, ,and , , N is the number of data sets; like , it means that the items to be verified in the data group and the corresponding item time are wrong; like , it means that the items to be verified and the corresponding item time in the data group are correct; among them, A is the preset coincidence parameter.

9. A data preprocessing system, characterized in that: The data preprocessing system adopts the engineering machinery collected data preprocessing method according to any one of claims 1 to 8 to preprocess the data collected by the engineering machinery.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for preprocessing collected data of an engineering machinery according to any one of claims 1 to 8 are implemented.

Images