Engineering machinery multi-source data processing method, device and system and storage medium
By using a unique external time source for pulsed time synchronization, the problem of time alignment of multi-source sensors in engineering machinery is solved, the consistency of sensor timestamps is achieved, and the accuracy and efficiency of data processing are improved.
Patent Information
- Application Number
- CN202510139387.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-08
- Publication Date
- 2025-05-16
AI Technical Summary
In the field of construction machinery, time alignment of multi-source sensors is difficult to achieve, resulting in time drift problems and limiting the development of unmanned construction.
The unique external time source is used to unify the initial time of various sensors and perform time synchronization through pulse mode to ensure the consistency of sensor timestamps.
It effectively solves the sensor time drift problem, keeps the timestamps of multi-source sensors consistent, and improves the accuracy and efficiency of data processing.
Smart Images
Figure CN120011727A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of data processing, and in particular to a method, device, system, and storage medium for processing multi-source data of engineering machinery. Background Art
[0002] In the field of engineering machinery construction, due to the particularity of the working environment, the rapid development of unmanned technology has solved the problem of difficulty in recruiting workers in the construction field. However, the number of sensors required for unmanned operation has also increased with the complexity of the working scene. How to solve the time alignment of multi-source sensors has become a major problem restricting the development of unmanned engineering machinery. Summary of the invention
[0003] The inventors have found through research that the related multi-source sensor alignment method uses a unified time source for time synchronization. However, this method has poor adaptability when there are many sensors, high precision requirements, and complex industrial environments, and it is difficult to ignore the problem of time-consuming sensor acquisition.
[0004] In view of at least one of the above technical problems, the present disclosure provides a method, device, system, and storage medium for multi-source data processing of engineering machinery, which use a unique external time source to unify the initial times of various sensors and align them in a pulse manner.
[0005] According to one aspect of the present disclosure, a method for processing multi-source data of an engineering machinery is provided, comprising:
[0006] Acquire multi-source data from multi-source sensors of construction machinery;
[0007] The external time source is controlled to perform time synchronization on the multi-source data in a pulse manner.
[0008] In some embodiments of the present disclosure, controlling the external time source to synchronize the multi-source data in a pulse manner includes:
[0009] Control each sensor in the multi-source sensor, synchronize the time from the only external hardware time source, obtain the current accurate timestamp to replace the local timestamp of each sensor;
[0010] The external hardware time source is controlled to send a timing signal to each sensor at a predetermined time interval to update a local time stamp of each sensor.
[0011] In some embodiments of the present disclosure, the method for processing multi-source data of engineering machinery further includes:
[0012] Frequency alignment is performed on data of different frequencies from multiple source sensors.
[0013] In some embodiments of the present disclosure, the frequency alignment of different frequency data of multi-source sensors includes:
[0014] After each sensor in the multi-source sensor is time synchronized, one sensor is used as a reference sensor;
[0015] The first sensor data is mapped to a reference timestamp of the reference sensor to obtain first sensor calibration data of the first sensor at the reference timestamp, wherein the first sensor is a sensor other than the reference sensor in the multi-source sensor.
[0016] In some embodiments of the present disclosure, mapping the data of the first sensor to the reference timestamp of the reference sensor to obtain the first sensor calibration data of the first sensor at the reference timestamp includes:
[0017] According to the reference sensor data and the first sensor data, a predetermined model is adopted based on an artificial intelligence algorithm to determine first sensor calibration data of the first sensor at the reference timestamp.
[0018] In some embodiments of the present disclosure, mapping the data of the first sensor to the reference timestamp of the reference sensor to obtain the first sensor calibration data of the first sensor at the reference timestamp includes:
[0019] According to the reference sensor data and the first sensor data, first sensor calibration data of the first sensor at the reference timestamp is determined by adopting at least one of interpolation, extrapolation and redundancy removal.
[0020] In some embodiments of the present disclosure, the engineering machinery multi-source data processing method further includes: when the collected data of one sensor of the multi-source sensors is missing a frame, performing data frame filling on the collected data of the sensor.
[0021] In some embodiments of the present disclosure, when the collected data of one sensor of the multi-source sensor is missing a frame, performing frame supplementation on the collected data of the sensor includes:
[0022] Comparing the collected data of the sensor with the reference frequency of the sensor to determine the frame missing timestamp of the sensor;
[0023] The data corresponding to the frame missing timestamp is determined according to all the data within two time periods of the sensor.
[0024] In some embodiments of the present disclosure, the method for processing multi-source data of engineering machinery further includes: performing data cleaning on the multi-source data to obtain clean data, wherein the data cleaning includes at least one of data conversion, data filtering and data correction.
[0025] In some embodiments of the present disclosure, performing data cleaning on the multi-source data to obtain clean data includes at least one of the following steps:
[0026] Setting a threshold range of acceptable data for different sensors, and when the collected data is greater than an acceptable upper threshold, correcting the data exceeding the upper threshold according to the upper threshold;
[0027] A device coordinate system is set as a standard coordinate system, and data in different robot coordinate systems are uniformly converted to the standard coordinate system;
[0028] The multi-source data is filtered according to a preset range of the sensor.
[0029] In some embodiments of the present disclosure, the engineering machinery multi-source data processing method further includes:
[0030] Transmit the processed data to the data bus through the network;
[0031] The data storage server is controlled to obtain the processed data from a specific channel.
[0032] According to another aspect of the present disclosure, there is provided a multi-source data processing device for engineering machinery, comprising:
[0033] A multi-source data acquisition module is configured to acquire multi-source data of multi-source sensors of the engineering machinery;
[0034] The time synchronization module is configured to control an external time source to perform time synchronization on the multi-source data in a pulse manner.
[0035] According to another aspect of the present disclosure, there is provided a multi-source data processing device for engineering machinery, comprising:
[0036] a memory configured to store instructions; and
[0037] The processor is configured to execute the instructions so that the engineering machinery multi-source data processing device implements the engineering machinery multi-source data processing method as described in any of the above embodiments.
[0038] According to another aspect of the present disclosure, a multi-source data processing system for engineering machinery is provided, comprising a multi-source sensor, an external time source, and the multi-source data processing device for engineering machinery as described in any one of the above embodiments.
[0039] According to another aspect of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the method for processing multi-source data of engineering machinery as described in any of the above embodiments is implemented.
[0040] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the method for processing multi-source data of an engineering machinery as described in any of the above embodiments is implemented.
[0041] The present disclosure can use a unique external time source to unify the initial times of various sensors and align them in a pulse manner, thereby solving the problem of sensor time drift and making the sensor timestamps consistent. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0043] Figure 1 Schematic diagram of some embodiments of the multi-source data processing method for engineering machinery disclosed in the present invention.
[0044] Figure 2 This is a schematic diagram before time synchronization in some embodiments of the present disclosure.
[0045] Figure 3 This is a schematic diagram after time synchronization in some embodiments of the present disclosure.
[0046] Figure 4 Schematic diagrams of other embodiments of the multi-source data processing method for engineering machinery disclosed in the present invention.
[0047] Figure 5 In some embodiments of the present disclosure, interpolation is used to align data of different frequencies.
[0048] Figure 6 In some embodiments of the present disclosure, extrapolation is used to align data of different frequencies.
[0049] Figure 7 A schematic diagram of missing frame data alignment in some embodiments of the present disclosure.
[0050] Figure 8 The figure is a schematic diagram of data flow in some embodiments of the present disclosure.
[0051] Fig. 9 It is a schematic diagram of the structure of some embodiments of the multi-source data processing device for engineering machinery disclosed in the present invention.
[0052] Fig.10 Schematic diagram of the structure of other embodiments of the multi-source data processing device for engineering machinery disclosed in the present invention. DETAILED DESCRIPTION
[0053] The technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present disclosure and its application or use. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.
[0054] Unless specifically stated otherwise, the relative arrangement of components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present disclosure.
[0055] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.
[0056] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.
[0057] In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.
[0058] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.
[0059] The inventors also found through research that how to solve the time alignment of multi-source sensors and how to accurately and quickly align multi-source data has become a major problem restricting the development of unmanned construction machinery. The data collection frequency of different sensors in the related technology is inconsistent, and the data needs to be interpolated or de-redundant. Currently, interpolation and extrapolation methods are mostly used, based on the least squares method or averaging within a time period.
[0060] Related technology time synchronization uses an external clock source to unify the reference time of various sensors, but the inconsistent sampling frequencies, transmission delays, and relative position movements of various sensors are difficult to solve using an external clock source.
[0061] The classical interpolation and extrapolation method of related technologies uses a polynomial fitting method to align data from different sensors. For scenarios where the sensor is missing frames or has a low frequency, the polynomial fitting method is used, which has obvious errors.
[0062] In view of at least one of the above technical problems, the present disclosure provides a method, device, system, and storage medium for processing multi-source data of engineering machinery, which are described below through embodiments.
[0063] Figure 1 Schematic diagram of some embodiments of the multi-source data processing method for engineering machinery disclosed in the present invention. Figure 1 The embodiment can be executed by the engineering machinery multi-source data processing device or the engineering machinery multi-source data processing system of the present disclosure. Figure 1 As shown, Figure 1 The method of the embodiment may include at least one of steps 11 and 12.
[0064] Step 11, obtaining multi-source data of multi-source sensors of the construction machinery.
[0065] Step 12: Control the external time source to perform time synchronization on the multi-source data in a pulse manner.
[0066] In some embodiments of the present disclosure, step 12 may include at least one of steps 121 to 122 .
[0067] Step 121 , controlling each sensor in the multi-source sensor to perform time synchronization from a unique external hardware time source, and obtaining a current accurate timestamp to replace the local timestamp of each sensor.
[0068] In some embodiments of the present disclosure, step 121 may include: based on the unique external hardware time synchronization, each sensor obtains a current accurate timestamp from the hardware device to replace the local timestamp.
[0069] Step 122: Control the external hardware time source to send a timing signal to each sensor at a predetermined time interval to update a local time stamp of each sensor.
[0070] In some embodiments of the present disclosure, the problem of time drift may occur during the continuous operation of each sensor, that is, the time of different sensors is inconsistent. Therefore, the hardware device needs to send a timing signal to each sensor regularly to update the local timestamp of the sensor to ensure that the time of different sensors is synchronized at the millisecond level. Figure 2 and Figure 3 shown.
[0071] Figure 2 This is a schematic diagram before time synchronization in some embodiments of the present disclosure. Figure 3 This is a schematic diagram after time synchronization in some embodiments of the present disclosure. Figure 2 and Figure 3 In an embodiment, the multi-source sensor disclosed in the present invention includes a camera, a radar, and an IMU (Inertial Measurement Unit).
[0072] In some embodiments of the present disclosure, step 12 may include: using an external device as the only time source to align the timestamps of each sensor at the initial moment, using the PPS (Pulse Per Second) generator built into the GNSS (Global Navigation Satellite System), each sensor captures the PPS signal, and when two pulses are received consecutively, adding one second to the last pulse time obtained as the timestamp of the sensor, thereby achieving time unification of each sensor and ensuring that there is no drift in the time difference of each sensor.
[0073] Figure 4 Schematic diagrams of other embodiments of the multi-source data processing method for engineering machinery disclosed in the present invention. Figure 4 The embodiment can be executed by the engineering machinery multi-source data processing device or the engineering machinery multi-source data processing system of the present disclosure. Figure 4 As shown, Figure 4 The method of the embodiment may include at least one of steps 1 to 5, and the order of steps 2 to 5 may be adjusted.
[0074] Step 1, acquiring multi-source data from multi-source sensors of engineering machinery; and controlling an external time source to perform time synchronization on the multi-source data in a pulse manner.
[0075] In some embodiments of the present disclosure, step 1 may include at least one step from step 11 to step 122 .
[0076] In some embodiments of the present disclosure, step 1 may include Figures 1 to 3 The method steps described in at least one embodiment.
[0077] Step 2: performing data cleaning on the multi-source data to obtain clean data, wherein the data cleaning includes at least one of data conversion, data filtering and data correction.
[0078] In some embodiments of the present disclosure, step 2 may include: data preprocessing.
[0079] In some embodiments of the present disclosure, step 2 may include: preprocessing the collected multi-source data to obtain clean data, wherein the preprocessing may include data conversion, abnormal data filtering, etc.
[0080] In some embodiments of the present disclosure, step 2 may include: receiving multi-sensor data, and performing pre-processing according to preset rules, wherein the pre-processing may include at least one of data correction, filtering, and noise reduction.
[0081] In some embodiments of the present disclosure, step 2 may include: data cleaning.
[0082] In some embodiments of the present disclosure, step 2 may include at least one of steps 21 to 23.
[0083] Step 21, setting a threshold range of acceptable data for different sensors, when the collected data is greater than an acceptable upper threshold limit, correcting the data exceeding the upper threshold limit according to the upper threshold limit.
[0084] In some embodiments of the present disclosure, step 21 may include: setting a threshold range for data that can be accepted by different sensors, and correcting data that exceeds the threshold according to the maximum value.
[0085] Step 22, setting a device coordinate system as a standard coordinate system, and uniformly converting data in different robot coordinate systems to the standard coordinate system.
[0086] In some embodiments of the present disclosure, step 22 may include: uniformly converting data in different robot coordinate systems into a certain device coordinate system such as a radar coordinate system.
[0087] Step 23: Filter the multi-source data according to the preset range of the sensor.
[0088] In some embodiments of the present disclosure, step 23 may include: filtering unnecessary data according to a preset range of the sensor, and retaining only specific data.
[0089] Step 3: align the frequencies of the data with different frequencies from the multi-source sensors.
[0090] In some embodiments of the present disclosure, step 3 may include at least one of steps 31 to 32.
[0091] Step 31: After each sensor in the multi-source sensor is time synchronized, one sensor is used as a reference sensor.
[0092] In some embodiments of the present disclosure, the reference sensor may be a radar.
[0093] Step 32: Map the first sensor data to the reference timestamp of the reference sensor to obtain first sensor calibration data of the first sensor at the reference timestamp, wherein the first sensor is a sensor other than the reference sensor in the multi-source sensor.
[0094] In some embodiments of the present disclosure, the reference sensor may be a radar; and the first sensor may be an IMU and a camera.
[0095] In some embodiments of the present disclosure, step 32 may include: determining first sensor calibration data of the first sensor at the reference timestamp based on the reference sensor data and the first sensor data and using a predetermined model based on an AI (Artificial Intelligence) algorithm.
[0096] In some embodiments of the present disclosure, the predetermined model may be obtained by training based on pre-collected data and LSTM (Long Short-Term Memory) or Transform method.
[0097] In some embodiments of the present disclosure, step 32 may include: after the time alignment of each sensor, a certain sensor (such as radar) is used as a reference, and other types of sensors are mapped to the timestamp of the sensor, and the reference sensor and other sensor data (IMU, camera) are input. The AI-based algorithm uses a preset model and outputs the data of other sensors at the timestamp, such as Figure 5 and Figure 6 shown.
[0098] In some embodiments of the present disclosure, step 32 may include: determining the first sensor calibration data of the first sensor at the reference timestamp by using at least one of interpolation, extrapolation and redundancy removal according to the reference sensor data and the first sensor data, such as Figure 5 and Figure 6 shown.
[0099] In some embodiments of the present disclosure, step 32 may include: based on a predetermined model, taking a certain sensor (such as a laser radar) as a reference sensor, interpolating and removing redundancy from the multi-source data, such as Figure 5 and Figure 6 shown.
[0100] Figure 5 It is a schematic diagram of aligning data of different frequencies in some embodiments of the present disclosure. Figure 6 This is a schematic diagram of aligning data of different frequencies in some embodiments of the present disclosure. Figure 5 and Figure 6 In the figure, the first to third rows are the data after the time synchronization of the camera, radar and IMU, and the fourth row is the data after the IMU frequency data is aligned with the radar data. The radar is the reference sensor, and the frequency of the radar and the camera is the same, while the frequency of the IMU and radar data is different. Figure 5 In some embodiments of the present disclosure, interpolation is used to align data of different frequencies. Figure 6 Some embodiments of the present disclosure use extrapolation to align data of different frequencies.
[0101] In the above embodiments of the present disclosure, due to the different data collection moments and exposure times, it is necessary to align the multi-source data. Based on the radar time as a reference, other sensor data and radar data are interpolated based on a predetermined model to complete the time alignment.
[0102] Step 4: When a frame of collected data of one of the multi-source sensors is missing, frame filling is performed on the collected data of the sensor.
[0103] In some embodiments of the present disclosure, step 4 may include at least one of steps 41 to 42.
[0104] Step 41 : Compare the collected data of the sensor with the reference frequency of the sensor to determine the frame missing timestamp of the sensor.
[0105] Step 42: Determine the data corresponding to the frame missing timestamp based on all the data within two time periods of the sensor.
[0106] In some embodiments of the present disclosure, if the data is not missing frames, data alignment can be inferred through other methods, such as polynomial fitting, least squares method, etc. to infer data at a specific timestamp. When the environmental impact is large, such as in a construction environment with high cold, high temperature, and high vibration, data missing frames or abnormal situations are difficult to infer in a fixed manner.
[0107] In some embodiments of the present disclosure, step 4 may include: if the sensor collects data frames due to network transmission problems or other abnormalities, based on the reference sensor frequency and the frame-missing data timestamp, all data within two time periods of the reference sensor are collected as input, and the data corresponding to the frame-missing timestamp is output, such as Figure 7 shown. Figure 7 A schematic diagram of missing frame data alignment in some embodiments of the present disclosure. Figure 7 The shorter medium-grey lines represent interpolated frame data.
[0108] The Disclosure Figure 2 and Figure 3 The embodiment is time synchronization, the present disclosure Figures 5 to 7 Any embodiment is data alignment. In the above embodiments of the present disclosure, multi-source sensor time synchronization and data alignment are one of the necessary conditions for data fusion processing.
[0109] In the above-mentioned embodiments of the present disclosure, the engineering machinery uses sensors to collect data, which can be synchronized with external hardware time, and a preset model is used to align data for abnormal, frame-missing and other data.
[0110] The above embodiments of the present disclosure can implement abnormal data frame filling. The sensor collects mechanical operation data as input, such as the position and angle of the operation joint, and based on the preset range, the sensor exceeds the preset range and processes the abnormal data based on the preset rules to avoid abnormal interruption of equipment operation due to data abnormality.
[0111] The above-mentioned embodiments of the present disclosure can realize the processing of missing frame data. The sensor may occasionally have loose wire harnesses due to network transmission or high vibration in the working environment. After interpolating the missing frame sensor data based on the preset model and aligning it based on the specific sensor frequency, the output is saved to an external storage device.
[0112] Step 5: Data storage.
[0113] In some embodiments of the present disclosure, step 5 may include: transmitting the processed data to a data bus via a network; controlling a data storage server to obtain the processed data from a specific channel, such as Figure 8 shown.
[0114] In some embodiments of the present disclosure, step 5 may include: the processed radar, IMU and camera data are transmitted to the data bus through the network; the data storage server obtains the radar, IMU and camera data from a specific channel to facilitate the connection and storage upload of personal computers, servers, cloud devices, etc. Figure 8 shown. Figure 8 The figure is a schematic diagram of data flow in some embodiments of the present disclosure.
[0115] In some embodiments of the present disclosure, step 5 may include: the processed data is input into other business models or stored in a PC (personal computer), server or cloud server at a specified frequency.
[0116] The above-mentioned embodiments of the present disclosure can use a unique external time source to unify the initial time of various sensors and align them in a pulse manner to solve the problem of sensor time drift, thereby solving the problem of inconsistent sensor timestamps.
[0117] The above-mentioned embodiments of the present disclosure are based on the AI preset model, which avoids the problem of error accumulation in polynomial fitting or least squares method when it comes to data missing frames.
[0118] Fig. 9 Schematic diagram of the structure of some embodiments of the multi-source data processing device for engineering machinery disclosed in the present invention. Fig. 9 As shown, the multi-source data processing device for engineering machinery disclosed in the present invention may include a multi-source data acquisition module 91 and a time synchronization module 92 .
[0119] The multi-source data acquisition module 91 is configured to acquire multi-source data from multi-source sensors of the engineering machinery.
[0120] The time synchronization module 92 is configured to control an external time source to perform time synchronization on the multi-source data in a pulse manner.
[0121] In some embodiments of the present disclosure, the time synchronization module 92 can be configured to control each sensor in the multi-source sensor, perform time synchronization from a unique external hardware time source, obtain a current accurate timestamp to replace the local timestamp of each sensor; control the external hardware time source to send a timing signal to each sensor at a predetermined time interval to update the local timestamp of each sensor.
[0122] In some embodiments of the present disclosure, the multi-source data processing device for engineering machinery of the present disclosure may also be configured to perform frequency alignment on data of different frequencies from multiple source sensors.
[0123] In some embodiments of the present disclosure, when frequency alignment is performed on different frequency data of multi-source sensors, the multi-source data processing device of the engineering machinery of the present disclosure can be configured to use one sensor as a reference sensor after time synchronization of each sensor in the multi-source sensors; map the first sensor data to the reference timestamp of the reference sensor to obtain the first sensor calibration data of the first sensor at the reference timestamp, wherein the first sensor is a sensor other than the reference sensor in the multi-source sensors.
[0124] In some embodiments of the present disclosure, the multi-source data processing device of the engineering machinery of the present disclosure can be configured to determine the first sensor calibration data of the first sensor at the reference timestamp based on the reference sensor data and the first sensor data by adopting a predetermined model based on an artificial intelligence algorithm.
[0125] In some embodiments of the present disclosure, the multi-source data processing device of the engineering machinery of the present disclosure can be configured to determine the first sensor calibration data of the first sensor at the reference timestamp based on the reference sensor data and the first sensor data by using at least one of interpolation, extrapolation and de-redundancy.
[0126] In some embodiments of the present disclosure, the multi-source data processing device for engineering machinery of the present disclosure may also be configured to perform frame complementation on the collected data of a sensor when a frame of collected data of one of the multi-source sensors is missing.
[0127] In some embodiments of the present disclosure, when performing data frame complementation, the multi-source data processing device for engineering machinery of the present disclosure can be configured to compare the collected data of the sensor and the reference frequency of the sensor to determine the frame missing timestamp of the sensor; and determine the data corresponding to the frame missing timestamp based on all data within two time periods of the sensor.
[0128] In some embodiments of the present disclosure, the multi-source data processing device for engineering machinery of the present disclosure may also be configured to perform data cleaning on the multi-source data to obtain clean data, wherein the data cleaning includes at least one of data conversion, data filtering and data correction.
[0129] In some embodiments of the present disclosure, when the multi-source data processing device for engineering machinery of the present disclosure performs data cleaning on the multi-source data to obtain clean data, it can be configured to perform at least one of the following operations: setting a threshold range for acceptable data of different sensors, and when the collected data is greater than an acceptable upper threshold limit, correcting the data exceeding the upper threshold limit according to the upper threshold limit; setting a device coordinate system as a standard coordinate system, and uniformly converting data in different robot coordinate systems to the standard coordinate system; and filtering the multi-source data according to a preset range of the sensor.
[0130] In some embodiments of the present disclosure, the multi-source data processing device for engineering machinery of the present disclosure may also be configured to transmit the processed data to a data bus via a network; and control a data storage server to obtain the processed data from a specific channel.
[0131] In some embodiments of the present disclosure, the multi-source data processing device for engineering machinery of the present disclosure may be configured to implement the multi-source data processing method for engineering machinery as described in any of the above embodiments.
[0132] The above embodiments of the present disclosure solve the time synchronization and data alignment problems of multi-source sensors by ensuring that each sensor is triggered uniformly through hardware devices, thus solving the time drift problem existing in data collection of different sensors. At the same time, the above embodiments of the present disclosure solve the problem of missing frames and missing frames of data by using a preset model to obtain deep-level rules from the data, thus solving the problem of data alignment conveniently and efficiently and improving the accuracy and speed of prediction.
[0133] Fig.10 Schematic diagram of the structure of some other embodiments of the multi-source data processing device for engineering machinery disclosed in the present invention. Fig.10 As shown, the engineering machinery multi-source data processing device disclosed herein may include a memory 101 and a processor 102 .
[0134] The memory 101 is used to store instructions. The processor 102 is coupled to the memory 101 . The processor 102 is configured to execute the engineering machinery multi-source data processing method involved in the above embodiment based on the instructions stored in the memory.
[0135] like Fig.10 As shown, the engineering machinery multi-source data processing device further includes a communication interface 103 for information exchange with other devices. At the same time, the engineering machinery multi-source data processing device further includes a bus 104, through which the processor 102, the communication interface 103, and the memory 101 communicate with each other.
[0136] The memory 101 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory. The memory 101 may also be a memory array. The memory 101 may also be divided into blocks, and the blocks may be combined into virtual volumes according to certain rules.
[0137] In addition, the processor 102 may be a central processing unit (CPU), or may be an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present disclosure.
[0138] According to another aspect of the present disclosure, a multi-source data processing system for engineering machinery is provided, including a multi-source sensor, an external time source, and a system as described in any of the above embodiments (e.g. Fig. 9 or Fig.10 The multi-source data processing device for engineering machinery described in the embodiment).
[0139] The above-mentioned embodiments of the present disclosure collect data based on engineering machinery construction scenarios: hardware time alignment can ensure the consistency of timestamps before data collection, and a preset model is used to align data from different sensors. The above-mentioned embodiments of the present disclosure solve the problem that related technical solutions can only be used for specific scenarios and cannot be migrated, reduce data unavailability caused by missing frames, anomalies, etc., and improve work efficiency.
[0140] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the method for processing multi-source data of an engineering machinery as described in any of the above embodiments is implemented.
[0141] According to another aspect of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the method for processing multi-source data of engineering machinery as described in any of the above embodiments is implemented.
[0142] The computer-readable storage medium of the present disclosure may be implemented as a non-transitory computer-readable storage medium.
[0143] The above-mentioned embodiment of the present disclosure provides a method for multi-sensor time synchronization and multi-source data alignment of engineering machinery, which relates to the field of special data processing, specifically, to the field of multi-source data time alignment and data preprocessing in the field of engineering machinery.
[0144] The above-mentioned embodiments of the present disclosure improve the methods for time alignment and data alignment in the related technologies, and propose a preset model based on the AI algorithm, which solves the problems of time alignment, data alignment, and frame filling for missing frames of data, and improves the efficiency and accuracy of data processing. Through the above-mentioned embodiments of the present disclosure, the application requirements for fast data processing in different operating environments can be met.
[0145] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, devices, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable non-transient storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0146] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram and the combination of processes and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0147] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0148] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0149] The engineering machinery multi-source data processing device, multi-source data acquisition module and time synchronization module described above can be implemented as a general-purpose processor, a programmable logic controller, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component or any appropriate combination thereof for performing the functions described in the present disclosure.
[0150] Those skilled in the art will appreciate that all or part of the steps of the above-described embodiment method of the present disclosure may be accomplished by hardware, and the hardware may be implemented as a general-purpose processor, a programmable logic controller, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, or any appropriate combination thereof for executing the method described in the present disclosure.
[0151] So far, the present disclosure has been described in detail. In order to avoid obscuring the concept of the present disclosure, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solution disclosed here.
[0152] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by instructing the relevant hardware through a program, and the program may be stored in a non-transitory computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.
[0153] The description of the present disclosure is given for the purpose of illustration and description, and is not intended to be exhaustive or to limit the present disclosure to the disclosed form. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present disclosure, and to enable those of ordinary skill in the art to understand the present disclosure and thereby design various embodiments with various modifications suitable for specific uses.
Claims
1. A method for processing multi-source data of construction machinery, comprising: Acquire multi-source data from multi-source sensors of construction machinery; The external time source is controlled to perform time synchronization on the multi-source data in a pulse manner.
2. The method for processing multi-source data of engineering machinery according to claim 1, wherein: The controlling the external time source to synchronize the multi-source data in a pulse manner comprises: Control each sensor in the multi-source sensor, synchronize the time from the only external hardware time source, obtain the current accurate timestamp to replace the local timestamp of each sensor; The external hardware time source is controlled to send a timing signal to each sensor at a predetermined time interval to update a local time stamp of each sensor.
3. The method for processing engineering machinery multi-source data according to claim 1 or 2, further comprising: Frequency alignment is performed on data of different frequencies from multiple source sensors.
4. The method for processing multi-source data of engineering machinery according to claim 1 or 2, wherein: The frequency alignment of different frequency data of multi-source sensors includes: After each sensor in the multi-source sensor is time synchronized, one sensor is used as a reference sensor; The first sensor data is mapped to a reference timestamp of the reference sensor to obtain first sensor calibration data of the first sensor at the reference timestamp, wherein the first sensor is another sensor in the multi-source sensor except the reference sensor.
5. The method for processing multi-source data of engineering machinery according to claim 4, wherein: Mapping the data of the first sensor to the reference timestamp of the reference sensor to obtain the first sensor calibration data of the first sensor at the reference timestamp includes: According to the reference sensor data and the first sensor data, a predetermined model is adopted based on an artificial intelligence algorithm to determine first sensor calibration data of the first sensor at the reference timestamp.
6. The method for processing engineering machinery multi-source data according to claim 4, wherein: Mapping the data of the first sensor to the reference timestamp of the reference sensor to obtain the first sensor calibration data of the first sensor at the reference timestamp includes: According to the reference sensor data and the first sensor data, first sensor calibration data of the first sensor at the reference timestamp is determined by adopting at least one of interpolation, extrapolation and redundancy removal.
7. The method for processing engineering machinery multi-source data according to claim 1 or 2, further comprising: When frames of collected data of one sensor of the multi-source sensors are missing, frames of collected data of the sensor are supplemented.
8. The method for processing multi-source data of engineering machinery according to claim 7, wherein: When the collected data of one sensor of the multi-source sensors is missing a frame, supplementing the collected data of the sensor includes: Comparing the collected data of the sensor with the reference frequency of the sensor to determine the frame missing timestamp of the sensor; The data corresponding to the frame missing timestamp is determined according to all the data within two time periods of the sensor.
9. The method for processing engineering machinery multi-source data according to claim 1 or 2, further comprising: The multi-source data is cleaned to obtain clean data, wherein the data cleaning includes at least one of data conversion, data filtering and data correction.
10. The method for processing multi-source data of engineering machinery according to claim 9, wherein: The step of performing data cleaning on the multi-source data to obtain clean data comprises at least one of the following steps: Setting a threshold range of acceptable data for different sensors, and when the collected data is greater than an acceptable upper threshold, correcting the data exceeding the upper threshold according to the upper threshold; A device coordinate system is set as a standard coordinate system, and data in different robot coordinate systems are uniformly converted to the standard coordinate system; The multi-source data is filtered according to a preset range of the sensor.
11. The method for processing engineering machinery multi-source data according to claim 1 or 2, further comprising: Transmit the processed data to the data bus through the network; The data storage server is controlled to obtain the processed data from a specific channel.
12. A multi-source data processing device for engineering machinery, comprising: A multi-source data acquisition module is configured to acquire multi-source data of multi-source sensors of the engineering machinery; The time synchronization module is configured to control an external time source to perform time synchronization on the multi-source data in a pulse manner.
13. A multi-source data processing device for engineering machinery, comprising: a memory configured to store instructions; and The processor is configured to execute the instructions so that the engineering machinery multi-source data processing device implements the engineering machinery multi-source data processing method as described in any one of claims 1 to 11.
14. A multi-source data processing system for engineering machinery, comprising a multi-source sensor, an external time source and the multi-source data processing device for engineering machinery as claimed in claim 12 or 13.
15. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the method for processing multi-source data of engineering machinery according to any one of claims 1 to 11 is implemented.
16. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, the method for processing multi-source data of engineering machinery according to any one of claims 1 to 11 is implemented.