Data processing method, data processing device, readable storage medium and electronic device

Abstract

The present invention proposes a data processing method, a data processing device, a readable storage medium, and an electronic device. The data processing method includes: obtaining an operating condition parameter set of engineering equipment; preprocessing the operating condition parameter set to determine multiple first elastic queues with different acquisition frequencies; determining a first wide table object based on the first elastic queue to determine multiple first wide table objects with different acquisition frequencies; determining a second wide table object based on the multiple first wide table objects; wherein the first wide table object is used to indicate a parameter set of operating condition parameters collected at the same acquisition frequency but different communication addresses with aligned data acquisition times, and the second wide table object is used to indicate a parameter set of operating condition parameters collected at different acquisition frequencies with aligned data acquisition times. In this way, users can perform multi-dimensional analysis of the operating conditions of engineering equipment based on the second wide table object to fully understand the operating status of the engineering equipment.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data processing method, a data processing device, a readable storage medium, and an electronic device. Background Art

[0002] In existing technology, due to the different types, installation locations, communication addresses, and accuracy requirements of multiple sensors on construction machinery, the collected CAN data has different acquisition frequencies, and the time of data collection for different acquisition frequencies is inconsistent. Analyzing the operating status of construction machinery typically requires multi-dimensional analysis of data at the same moment, but data with different sampling frequencies and times cannot meet this requirement. Therefore, providing a technical solution to align the time of data with different frequencies to facilitate multi-dimensional analysis of the construction machinery's operating status based on real-time CAN data has become an urgent problem. Summary of the Invention

[0003] The present invention aims to solve at least one of the technical problems existing in the prior art or related art.

[0004] To this end, a first aspect of the present invention is to propose a data processing method.

[0005] A second aspect of the present invention is to provide a data processing device.

[0006] A third aspect of the present invention is to provide a readable storage medium.

[0007] A fourth aspect of the present invention is to provide an electronic device.

[0008] In view of this, according to one aspect of the present invention, a data processing method is proposed, including: obtaining an operating condition parameter set of engineering equipment; preprocessing the operating condition parameter set to determine a plurality of first elastic queues with different acquisition frequencies; determining a first wide table object based on the first elastic queue to determine a plurality of first wide table objects with different acquisition frequencies; determining a second wide table object based on the plurality of first wide table objects; wherein the first wide table object is used to indicate a parameter set of operating condition parameters collected at different communication addresses with the same acquisition frequency that have aligned data acquisition time, and the second wide table object is used to indicate a parameter set of operating condition parameters with different acquisition frequencies that have aligned data acquisition time.

[0009] It should be noted that the execution subject of the data processing method proposed in the present invention can be a data processing device. In order to more clearly illustrate the data processing method proposed in the present invention, the following technical solution is exemplified by taking the execution subject of the data processing method as the data processing device.

[0010] In this technical solution, the above-mentioned first elastic queue is used to indicate a set of operating condition parameters of the same acquisition frequency category; the above-mentioned first wide table object is used to indicate a parameter set of parsed operating condition parameters of different communication addresses with the same acquisition frequency that have aligned data acquisition time; the second wide table object is used to indicate a parameter set of operating condition parameters of different acquisition frequencies that have aligned data acquisition time.

[0011] Specifically, the data processing device first obtains the operating parameters of the engineering equipment. Specifically, the engineering equipment is equipped with multiple sensors, which will send the collected operating parameters to the Kafka message queue (distributed message queue) through the data acquisition device according to their corresponding acquisition frequencies. The data processing device obtains the above-mentioned operating parameter set by consuming the Kafka message queue. The above-mentioned Kafka message queue is a message queue for storing operating parameters in the data processing system developed by the program. Generally speaking, the operating parameters of the Kafka message queue are in the CAN data format.

[0012] Furthermore, the data processing device preprocesses the operating condition parameter set to determine multiple first elastic queues with different collection frequencies. Specifically, the data processing device sorts the operating condition parameters based on their collection frequency and the communication address from which they were collected, removing any abnormal data. After preprocessing, the data processing device uses the operating condition parameters of the same collection frequency category as a first elastic queue and determines multiple first elastic queues with different collection frequencies.

[0013] Furthermore, the data processing device determines a first wide table object based on the first flexible queue to identify multiple first wide table objects with different acquisition frequencies. Specifically, the data processing device performs time alignment processing on the operating condition parameters in the first flexible queue according to a pre-programmed process to determine the first wide table object. Specifically, the first wide table object is provided with attributes such as the acquisition time, data acquisition device number, pilot pressure, operating pressure, current, engine speed, torque, and temperature of the operating condition parameters. The data processing device determines the first wide table object by filling the time-aligned operating condition parameters into the wide table object according to the corresponding attributes.

[0014] It can be understood that a first wide table object can be determined according to the first elastic queue of each sampling frequency, so the data processing device can determine multiple first wide table objects of different sampling frequencies.

[0015] Furthermore, the data processing device determines a second wide table object based on the plurality of first wide table objects. Specifically, the data processing device performs time alignment processing on the operating condition parameters in the plurality of first wide table objects according to a pre-programmed process to determine a second wide table object having operating condition parameters of different acquisition frequencies aligned with the data acquisition time.

[0016] In this technical solution, the data processing device first obtains the operating condition parameter set of the engineering equipment and preprocesses the operating condition parameter set to determine multiple first elastic queues with different acquisition frequencies. It then performs time alignment processing on the operating condition parameters in the first elastic queue to determine multiple first wide table objects with different acquisition frequencies. After determining the multiple first wide table objects, the data processing device performs time alignment processing on the operating condition parameters in the first wide table object to determine a second wide table object with operating condition parameters of different acquisition frequencies that have aligned data acquisition times. In this way, users can perform multi-dimensional analysis of the operating conditions of the engineering equipment in real time based on the second wide table object to fully understand the operating status of the engineering equipment. At the same time, in the technical solution of the present invention, the acquisition time of the operating condition parameters of the same transmission frequency is first aligned, and then the acquisition time of all operating condition parameters of different transmission frequencies is aligned. This is conducive to improving the efficiency and accuracy of data acquisition time alignment.

[0017] In addition, the data processing method according to the present invention may also have the following additional technical features:

[0018] In the above technical solution, the steps of preprocessing the operating condition parameter set and determining multiple first elastic queues with different acquisition frequencies specifically include: filtering out target parameters in the operating condition parameter set according to a regular expression; diverting the target parameters according to the acquisition frequency; and using the target parameters of the same acquisition frequency as a first elastic queue to determine multiple first elastic queues with different acquisition frequencies.

[0019] In this technical solution, the above-mentioned regular expression is used to indicate pre-set filtering rules, for example, it can be used to filter the operating parameters of the first engineering equipment, or it can be used to remove dirty data in the operating parameter set. Dirty data is used to indicate missing values, or data that is not in CAN format, such as system log data.

[0020] Specifically, the process of preprocessing the operating condition parameter set and determining a plurality of first elastic queues with different acquisition frequencies is as follows: the data processing device filters out target parameters from the operating condition parameter set according to a regular expression.

[0021] Furthermore, the data processing device diverts the target parameters according to the acquisition frequency. Specifically, the sampling frequencies of multiple sensors on the engineering equipment may be different, so the data processing device needs to first divert the screened target parameters according to the acquisition frequency, so as to align the acquisition time of the operating parameters of the same acquisition frequency category.

[0022] Furthermore, target parameters of the same acquisition frequency are used as a first elastic queue, and a plurality of first elastic queues with different acquisition frequencies are determined.

[0023] In this technical solution, before determining the first elastic queue, the data processing device filters target parameters from the operating condition parameter set based on a regular expression. The target parameters are then sorted according to their acquisition frequency. Target parameters within the same acquisition frequency category serve as a first elastic queue, and multiple first elastic queues with different acquisition frequencies are determined. This facilitates time alignment of operating condition parameters within the same acquisition frequency category, ensuring the accuracy of data acquisition time alignment.

[0024] In the above technical solution, the step of using target parameters of the same acquisition frequency as a first elastic queue specifically includes: diverting the target parameters of the same acquisition frequency according to the communication address to determine multiple target parameter subsets; processing abnormal parameters of the multiple target parameter subsets, and determining the first elastic queue based on the processed multiple target parameter subsets.

[0025] In this technical solution, target parameters of the same acquisition frequency category are used as a first elastic queue. The data processing device further distributes the target parameters sorted by acquisition frequency by communication address, determining multiple target parameter subsets. Specifically, a target parameter subset represents a set of operating condition parameters with the same acquisition frequency and communication address.

[0026] Furthermore, the data processing device processes abnormal parameters in the plurality of target parameter subsets. Specifically, abnormal parameters refer to data where the time interval between two consecutive data is much shorter than the data collection period of this type.

[0027] Furthermore, after determining the abnormal data in the target parameter subset, the data processing device deletes the next piece of data from the target parameter subset, and then determines the first elastic queue according to the processed multiple target parameter subsets.

[0028] It is understandable that dividing the target parameters into target parameter subsets of the same communication address makes it easier to confirm abnormal data, which improves the accuracy of abnormal data processing.

[0029] In this technical solution, when determining the first elastic queue based on target parameters of the same acquisition frequency category, the data processing device also needs to separate the target parameters according to their communication addresses to remove abnormal data from them. The first elastic queue is then determined based on the processed subsets of target parameters. This ensures that the first elastic queue contains no abnormal data, thereby improving the accuracy of the time alignment of operating condition parameter acquisition.

[0030] In the above technical solution, the step of determining a first wide table object based on the first elastic queue specifically includes: determining a preset type of operating condition parameter in the first elastic queue as primary data; when the first elastic queue contains two primary data, confirming whether a time parameter of the first-order data in the first elastic queue is less than a first time threshold; when the time parameter is not less than the first time threshold and the first-order data is between positive and negative 1 / 2 cycle time of the first primary data, parsing the first-order data, deleting the first-order data from the first elastic queue, and using the next-order data as the first-order data; and updating the first wide table object based on the parsing result of the first-order data until the time parameter of the first-order data in the first elastic queue is greater than the first time threshold; wherein the time parameter indicates the difference between the acquisition time of the first-order data and the acquisition time of the first primary data, and the first time threshold indicates a positive or negative value of the acquisition time of 1 / 2 cycle time of the primary data.

[0031] In this technical solution, the time parameter is used to indicate the difference between the acquisition time of the first-order data and the time of acquiring the first main data, and the first time threshold is used to indicate the positive or negative value of the time of acquiring 1 / 2 cycle of the main data.

[0032] Specifically, the process of determining the first wide table object based on the first elastic queue is as follows: the data processing device determines a preset type of operating condition parameter in the first elastic queue as primary data. Specifically, the preset type of operating condition parameter is used to indicate a highly important operating condition parameter, such as engine speed.

[0033] Furthermore, when the first elastic queue contains two primary data, the data processing device determines whether a time parameter of the first-order data in the first elastic queue is less than a first time threshold. Specifically, if the first elastic queue contains two primary data, it indicates that the first-order data is suitable for collection time alignment with the first primary data. In this case, the magnitude of the time parameter of the first-order data is determined.

[0034] Furthermore, if the data processing device determines that the time parameter is not less than the first time threshold and the first-order data is within a time range of plus or minus half a cycle of the first master data, it parses the first-order data, deletes the first-order data from the first elastic queue, and uses the next-order data as the first-order data. Specifically, if the time parameter is not less than the first time threshold and the first-order data is within a time range of plus or minus half a cycle of the first master data, it indicates that the first-order data in the first elastic queue has not yet been aligned in terms of collection time, and the collection time of the first data is relatively close to that of the first master data. In this case, the first-order data is parsed to align its collection time with that of the first master data. Simultaneously, the first-order data is deleted from the first elastic queue, the next-order data is used as the new first-order data, and the above steps are repeated.

[0035] Furthermore, the data processing device updates the first wide table object based on the above-mentioned analysis results of the first-order data, that is, after aligning the collection time of the first-order data with the first main data, the analysis results are written into the first wide table object according to the attributes until all the operating condition parameters in the first elastic queue are updated to the first wide table object.

[0036] It should be noted that, when determining that the first elastic queue does not contain two master data, the data processing apparatus performs the step of determining the second wide table object according to multiple first elastic queues that do not contain two master data.

[0037] In this technical solution, the data processing device identifies a preset type of operating condition parameter in a first elastic queue as primary data. Upon confirming that the first elastic queue contains two primary data, that the time parameter of the first-order data in the first elastic queue is not less than a first time threshold, and that the first-order data is within the range of plus or minus 1 / 2 of the cycle time of the first primary data, the data processing device aligns the acquisition time of the first-order data with that of the first primary data, updates the first wide table object, and simultaneously deletes the first-order data from the first elastic queue and replaces the next-order data with the first-order data, until all operating condition parameters in the first elastic queue are updated to the first wide table object. In this way, the data processing device can align the acquisition times of operating condition parameters with the same acquisition frequency but different communication addresses, ensuring the accuracy of the time alignment of all operating condition parameters with different transmission frequencies in subsequent steps.

[0038] In the above technical solution, after confirming whether the time parameter of the first-priority data in the first elastic queue is less than the first time threshold, the data processing method further includes: if the time parameter is not less than the first time threshold and the first-priority data is not between the positive and negative 1 / 2 cycle time of the first main data, deleting the first-priority data from the first elastic queue and using the next-priority data as the first-priority data.

[0039] In this technical solution, if the data processing device determines that the time parameter of the first-priority data is not less than the first time threshold, and the first-priority data is not between the positive and negative 1 / 2 cycle time of the first main data, it indicates that the first-priority data cannot be aligned with the collection time of the first main data. In this case, the first-priority data is deleted from the first elastic queue, the next-priority data is used as the first-priority data, and the above-mentioned judgment process is re-executed.

[0040] In this technical solution, the data processing device determines whether the first-order data is suitable for collection time alignment with the first master data by checking whether its time parameter is less than a first time threshold and whether the first-order data is within the range of plus or minus half a cycle of the first master data. If not, the first-order data is removed from the first elastic queue. This ensures efficient and accurate collection time alignment of operating condition parameters with the same collection frequency but different communication addresses.

[0041] In the above technical solution, the step of determining the second wide table object according to the plurality of first wide table objects specifically includes: determining a second elastic queue according to the plurality of first wide table objects; and determining the second wide table object according to the second elastic queue.

[0042] In this technical solution, the second flexible queue is a collection of operating parameters with different acquisition frequencies. It is understandable that, because the second flexible queue is determined based on multiple first wide table objects, the operating parameters of the same sampling frequency category in the second flexible queue have already been aligned in acquisition time.

[0043] Specifically, the process of determining the second wide table object based on multiple first wide table objects is as follows: the data processing device determines a second elastic queue based on the multiple first wide table objects, and then performs acquisition time alignment processing on the operating condition parameters of different acquisition frequencies in the second elastic queue to determine the second wide table object.

[0044] It should be noted that converting multiple first wide table objects into the second elastic queue is beneficial to time alignment processing.

[0045] In this technical solution, in the process of determining the second wide table object based on multiple first wide table objects, the data processing device improves the rate and accuracy of aligning the acquisition time of operating condition parameters with different acquisition frequencies by converting multiple first wide table objects into a second elastic queue.

[0046] In the above technical solution, determining the second wide table object based on the second elastic queue specifically includes: determining the operating condition parameter of the category with the highest collection frequency in the second elastic queue as the main data; when the second elastic queue contains two main data, confirming whether the time parameter of the first-order data in the second elastic queue is less than the first time threshold; when the time parameter is not less than the first time threshold and the first-order data is between the positive and negative 1 / 2 cycle time of the first main data, updating the second wide table object according to the first digital data, deleting the first-order data from the second elastic queue, and using the next-order data as the first-order data until the time parameter of the first-order data in the second elastic queue is greater than the first time threshold; wherein the time parameter is used to indicate the difference between the collection time of the first-order data and the collection time of the first main data, and the first time threshold is used to indicate the positive or negative value of the time of 1 / 2 cycle of the main data.

[0047] In this technical solution, the time parameter is used to indicate the difference between the acquisition time of the first-order data and the time of acquiring the first main data, and the first time threshold is used to indicate the positive or negative value of the time of acquiring 1 / 2 cycle of the main data.

[0048] Specifically, the process of determining the second wide table object according to the second elastic queue is as follows: the data processing device determines the operating condition parameter of the category with the highest collection frequency in the second elastic queue as the main data.

[0049] Furthermore, when the second elastic queue contains two primary data items, the data processing device determines whether a time parameter of the first-order data item in the second elastic queue is less than a first time threshold. Specifically, if the second elastic queue contains two primary data items, this indicates that the first-order data item is suitable for collection time alignment with the first primary data item. In this case, the magnitude of the time parameter of the first-order data item is determined.

[0050] Furthermore, when the data processing device determines that the time parameter is not less than the first time threshold and the first-order data is between the positive and negative 1 / 2 cycle time of the first main data, the data processing device updates the second wide table object according to the first-order data, deletes the first-order data from the second elastic queue, and uses the next-order data as the first-order data until the time parameter of the first-order data in the second elastic queue is greater than the first time threshold.

[0051] Specifically, if the time parameter is not less than the first time threshold and the first-order data falls within the range of plus or minus half a cycle of the first master data, the first-order data in the second elastic queue has not yet been time-aligned, and the collection time of the first data is close to that of the first master data. In this case, the first-order data is analyzed to align its collection time with that of the first master data. Simultaneously, the first-order data is deleted from the first elastic queue, and the next-order data is used as the new first-order data. The above steps are repeated until the time parameter of the first-order data in the second elastic queue exceeds the first time threshold, that is, until all operating condition parameters in the second elastic queue are updated to the second wide table object.

[0052] It can be understood that since the CAN data format operating condition parameters have been parsed in the process of determining the first wide table object based on the first elastic queue, the data in the second elastic queue is the parsed data. Therefore, in the process of determining the second wide table object based on the second elastic queue, there is no need to parse the operating condition parameters.

[0053] In this technical solution, the data processing device determines the operating condition parameter of the most frequently collected category in the second elastic queue as the primary data. Upon confirming that the second elastic queue contains two primary data, that the time parameter of the first-order data in the second elastic queue is not less than the first time threshold, and that the first-order data is between the plus or minus 1 / 2 cycle time of the first primary data, the data processing device aligns the collection time of the first-order data with that of the first primary data, updates the second wide table object, and simultaneously deletes the first-order data from the second elastic queue and replaces the next-order data with the first-order data, until all operating condition parameters in the second elastic queue are updated to the second wide table object. In this way, the data processing device can align the collection times of operating condition parameters with different collection frequencies, allowing users to conduct multi-dimensional analysis of the operating conditions of engineering equipment in real time based on the second wide table object, thereby gaining a more comprehensive understanding of the working conditions of the engineering equipment.

[0054] According to a second aspect of the present invention, a data processing device is proposed, which includes: an acquisition unit for acquiring an operating condition parameter set of engineering equipment; a processing unit for preprocessing the operating condition parameter set to determine a plurality of first elastic queues with different acquisition frequencies; the processing unit is also used to determine a first wide table object based on the first elastic queue to determine a plurality of first wide table objects with different acquisition frequencies; the processing unit is also used to determine a second wide table object based on the plurality of first wide table objects; wherein the first wide table object is used to indicate a parameter set of operating condition parameters collected at different communication addresses with the same acquisition frequency that have aligned data acquisition time, and the second wide table object is used to indicate a parameter set of operating condition parameters with different acquisition frequencies that have aligned data acquisition time.

[0055] In this technical solution, the above-mentioned first elastic queue is used to indicate a set of operating condition parameters of the same acquisition frequency category; the above-mentioned first wide table object is used to indicate a parameter set of operating condition parameters of different communication addresses with the same acquisition frequency that have aligned data acquisition time; the second wide table object is used to indicate a parameter set of operating condition parameters of different acquisition frequencies that have aligned data acquisition time.

[0056] Specifically, the acquisition unit first acquires the operating parameters of the engineering equipment. Specifically, the engineering equipment is equipped with multiple sensors, which send the collected operating parameters to the Kafka message queue through the data acquisition device according to their corresponding acquisition frequencies. The acquisition unit can obtain the above-mentioned operating parameter set by consuming the Kafka message queue. The above-mentioned Kafka message queue is a message queue for storing operating parameters in a data processing system developed through a program. Generally speaking, the operating parameters of the Kafka message queue are in the CAN data format.

[0057] Furthermore, the processing unit preprocesses the operating condition parameter set to determine multiple first elastic queues with different collection frequencies. Specifically, the processing unit sorts the operating condition parameters based on their collection frequency and the communication address from which they were collected, removing any abnormal data. After preprocessing, the processing unit uses the operating condition parameters of the same collection frequency category as a first elastic queue and determines multiple first elastic queues with different collection frequencies.

[0058] Furthermore, the processing unit determines a first wide table object based on the first flexible queue to identify multiple first wide table objects with different acquisition frequencies. Specifically, the processing unit time-aligns the operating condition parameters in the first flexible queue according to a pre-programmed process to determine the first wide table object. Specifically, the first wide table object is configured with attributes such as the acquisition time, data acquisition device number, pilot pressure, operating pressure, current, engine speed, torque, and temperature of the operating condition parameters. The processing unit determines the first wide table object by filling the time-aligned operating condition parameters into the wide table object.

[0059] It can be understood that a first wide table object can be determined according to the first elastic queue of each sampling frequency, so the processing unit can determine multiple first wide table objects of different sampling frequencies.

[0060] Furthermore, the processing unit determines a second wide table object based on the plurality of first wide table objects. Specifically, the processing unit performs time alignment processing on the operating condition parameters in the plurality of first wide table objects according to a pre-programmed process to determine a second wide table object having operating condition parameters of different acquisition frequencies aligned with the data acquisition time.

[0061] In this technical solution, the operating condition parameter set of the engineering equipment is first obtained by the acquisition unit, and the operating condition parameter set is pre-processed by the processing unit to determine multiple first elastic queues with different acquisition frequencies, and the operating condition parameters in the first elastic queue are time-aligned to determine multiple first wide table objects with different acquisition frequencies. After determining multiple first wide table objects, the processing unit performs time alignment on the operating condition parameters in the first wide table object to determine a second wide table object with operating condition parameters of different acquisition frequencies that have aligned data acquisition times. In this way, the user can perform multi-dimensional analysis of the operating conditions of the engineering equipment in real time based on the second wide table object to fully understand the operating status of the engineering equipment. At the same time, in the technical solution of the present invention, the acquisition time of the operating condition parameters of the same sending frequency is first aligned, and then the acquisition time of all the operating condition parameters of different sending frequencies is aligned. This is conducive to improving the efficiency and accuracy of data acquisition time alignment.

[0062] According to a third aspect of the present invention, a readable storage medium is provided, on which a program or instructions are stored. When executed by a processor, the program or instructions implement the data processing method according to the first aspect of the present invention. Therefore, the readable storage medium has all the beneficial effects of the data processing method according to the first aspect of the present invention, and will not be further described here.

[0063] According to the fourth aspect of the present invention, an electronic device is proposed, comprising: a data processing device as proposed in the second aspect of the present invention, and / or a readable storage medium as proposed in the third aspect of the present invention. Therefore, the electronic device has all the beneficial effects of the data processing device proposed in the second aspect of the present invention and / or the readable storage medium proposed in the third aspect of the present invention, which will not be repeated here.

[0064] Additional aspects and advantages of the invention will become apparent from the description which follows, or may be learned by practice of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments with reference to the following drawings, in which:

[0066] Figure 1 FIG1 shows a flow chart of a data processing method according to an embodiment of the present invention;

[0067] Figure 2 FIG2 shows a second flow chart of a data processing method according to an embodiment of the present invention;

[0068] Figure 3 FIG3 shows a flow chart of a data processing method according to an embodiment of the present invention;

[0069] Figure 4FIG4 shows a fourth flow chart of a data processing method according to an embodiment of the present invention;

[0070] Figure 5 FIG5 shows a fifth flow chart of a data processing method according to an embodiment of the present invention;

[0071] Figure 6 FIG6 shows a sixth flow chart of a data processing method according to an embodiment of the present invention;

[0072] Figure 7 FIG7 shows a flow chart of a data processing method according to an embodiment of the present invention;

[0073] Figure 8 A schematic block diagram of a data processing device according to an embodiment of the present invention is shown;

[0074] Figure 9 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown;

[0075] Figure 10 A schematic diagram of an elastic queue according to an embodiment of the present invention is shown;

[0076] Figure 11 A diagram showing the idea of a data processing method according to an embodiment of the present invention;

[0077] Figure 12 The figure shows an overall flow chart of the data processing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0078] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present invention and the features therein can be combined with each other without conflict.

[0079] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0080] The following combination Figures 1 to 12 , the data processing method, data processing device, readable storage medium and electronic device proposed in the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0081] Example 1:

[0082] Figure 1 A flow chart of a data processing method according to an embodiment of the present invention is shown, wherein the data processing method includes:

[0083] Step S102, obtaining an operating parameter set of engineering equipment;

[0084] Step S104: pre-processing the operating condition parameter set to determine a plurality of first elastic queues with different acquisition frequencies;

[0085] Step S106: determining a first wide table object according to the first elastic queue to determine a plurality of first wide table objects with different collection frequencies;

[0086] Step S108: determining a second wide table object according to the plurality of first wide table objects.

[0087] Among them, the first wide table object is used to indicate the parameter set of operating parameters collected at the same collection frequency but different communication addresses with aligned data collection time, and the second wide table object is used to indicate the parameter set of operating parameters at different collection frequencies with aligned data collection time.

[0088] It should be noted that the execution subject of the data processing method proposed in the present invention can be a data processing device. In order to more clearly illustrate the data processing method proposed in the present invention, the following embodiments are exemplified by taking the execution subject of the data processing method as a data processing device.

[0089] In this embodiment, the first elastic queue is used to indicate a set of operating condition parameters of the same acquisition frequency category; the first wide table object is used to indicate a parameter set of parsed operating condition parameters of different communication addresses with the same acquisition frequency that have aligned data acquisition time; and the second wide table object is used to indicate a parameter set of operating condition parameters of different acquisition frequencies that have aligned data acquisition time.

[0090] Specifically, the data processing device first obtains the operating parameters of the engineering equipment. Specifically, the engineering equipment is equipped with multiple sensors, which transmit the collected operating parameters to a Kafka message queue via a data acquisition device at their corresponding acquisition frequencies. The data processing device consumes the Kafka message queue to obtain the aforementioned operating parameter set. The Kafka message queue is a message queue used to store operating parameters in a data processing system developed through a program. Generally, the operating parameters in the Kafka message queue are in the CAN data format.

[0091] Furthermore, the data processing device preprocesses the operating condition parameter set to determine multiple first elastic queues with different collection frequencies. Specifically, the data processing device sorts the operating condition parameters based on their collection frequency and the communication address from which they were collected, removing any abnormal data. After preprocessing, the data processing device uses the operating condition parameters of the same collection frequency category as a first elastic queue and determines multiple first elastic queues with different collection frequencies.

[0092] Furthermore, the data processing device determines a first wide table object based on the first flexible queue to identify multiple first wide table objects with different acquisition frequencies. Specifically, the data processing device time-aligns the operating condition parameters in the first flexible queue according to a pre-programmed process to determine the first wide table object. Specifically, the first wide table object is configured with attributes such as the acquisition time, data acquisition device number, pilot pressure, operating pressure, current, engine speed, torque, and temperature of the operating condition parameters. The data processing device determines the first wide table object by populating the time-aligned operating condition parameters into the wide table object.

[0093] It can be understood that a first wide table object can be determined according to the first elastic queue of each sampling frequency, so the data processing device can determine multiple first wide table objects of different sampling frequencies.

[0094] Furthermore, the data processing device determines a second wide table object based on the plurality of first wide table objects. Specifically, the data processing device performs time alignment processing on the operating condition parameters in the plurality of first wide table objects according to a pre-programmed process to determine a second wide table object having operating condition parameters of different acquisition frequencies aligned with the data acquisition time.

[0095] Specifically, in this embodiment, since the data obtained by the data processing device through the data acquisition equipment is the operating parameters in the CAN data format, and the CAN data is like a stream, continuously generated and consumed, the data processing method proposed in the above embodiment is a streaming solution, and the smaller the time delay, the better, generally within 3 seconds.

[0096] In this embodiment, the data processing device first obtains the operating condition parameter set of the engineering equipment and preprocesses the operating condition parameter set to determine multiple first elastic queues with different acquisition frequencies. Then, the operating condition parameters in the first elastic queue are time-aligned to determine multiple first wide table objects with different acquisition frequencies. After determining the multiple first wide table objects, the data processing device time-aligns the operating condition parameters in the first wide table object to determine a second wide table object with operating condition parameters of different acquisition frequencies whose data acquisition times are aligned. In this way, users can perform multi-dimensional analysis of the operating conditions of the engineering equipment in real time based on the second wide table object to fully understand the operating status of the engineering equipment. At the same time, in this embodiment, the acquisition time of the operating condition parameters of the same transmission frequency is first aligned, and then the acquisition time of all operating condition parameters of different transmission frequencies is aligned. This is conducive to improving the efficiency and accuracy of data acquisition time alignment.

[0097] Figure 2 A flow chart of a data processing method according to an embodiment of the present invention is shown, wherein the data processing method includes:

[0098] Step S202, obtaining an operating parameter set of engineering equipment;

[0099] Step S204, filtering out target parameters in the operating condition parameter set according to the regular expression;

[0100] Step S206, dividing the target parameters according to the acquisition frequency;

[0101] Step S208 , using the target parameter of the same acquisition frequency as a first elastic queue, determining a plurality of first elastic queues with different acquisition frequencies;

[0102] Step S210: determining a first wide table object according to the first elastic queue to determine a plurality of first wide table objects with different collection frequencies;

[0103] Step S212: Determine a second wide table object according to the plurality of first wide table objects.

[0104] In this embodiment, the regular expression is used to indicate a pre-set filtering rule. For example, it can be used to filter the operating parameters of the first engineering equipment or to remove dirty data from the operating parameter set. Dirty data refers to missing values or data not in CAN format, such as system log data. Specifically, the process of pre-processing the operating parameter set to determine multiple first elastic queues with different collection frequencies is as follows: the data processing device filters the target parameters from the operating parameter set according to the regular expression.

[0105] Furthermore, the data processing device diverts the target parameters according to the acquisition frequency. Specifically, the sampling frequencies of multiple sensors on the engineering equipment may be different, so the data processing device needs to first divert the screened target parameters according to the acquisition frequency, so as to align the acquisition time of the operating parameters of the same acquisition frequency category.

[0106] Furthermore, target parameters of the same acquisition frequency are used as a first elastic queue, and a plurality of first elastic queues with different acquisition frequencies are determined.

[0107] In this embodiment, before determining the first elastic queue, the data processing device filters the target parameters from the operating condition parameter set based on a regular expression. The target parameters are then sorted according to their acquisition frequency. Target parameters within the same acquisition frequency category serve as a first elastic queue, and multiple first elastic queues with different acquisition frequencies are determined. This facilitates time alignment of operating condition parameters within the same acquisition frequency category, ensuring the accuracy of data acquisition time alignment.

[0108] Figure 3 A flow chart of a data processing method according to an embodiment of the present invention is shown, wherein the data processing method includes:

[0109] Step S302, obtaining an operating parameter set of engineering equipment;

[0110] Step S304, filtering out target parameters in the operating condition parameter set according to the regular expression;

[0111] Step S306, dividing the target parameters according to the acquisition frequency;

[0112] Step S308, dividing the target parameters of the same acquisition frequency according to the communication address to determine multiple target parameter subsets;

[0113] Step S310: Processing abnormal parameters of multiple target parameter subsets, determining a first elastic queue based on the processed multiple target parameter subsets, and determining multiple first elastic queues with different collection frequencies;

[0114] Step S312: determining a first wide table object according to the first elastic queue to determine a plurality of first wide table objects with different collection frequencies;

[0115] Step S314: determining a second wide table object according to the plurality of first wide table objects.

[0116] In this embodiment, the process of using target parameters of the same acquisition frequency category as a first elastic queue is as follows: the data processing device divides the target parameters divided by acquisition frequency and then divides them by communication address to determine multiple target parameter subsets. Specifically, the target parameter subsets are used to indicate the set of operating condition parameters with the same acquisition frequency and the same communication address.

[0117] Furthermore, the data processing device processes abnormal parameters in the plurality of target parameter subsets. Specifically, abnormal parameters refer to data where the time interval between two consecutive data is much shorter than the data collection period of this type.

[0118] Furthermore, after determining the abnormal data in the target parameter subset, the data processing device deletes the next piece of data from the target parameter subset, and then determines the first elastic queue according to the processed multiple target parameter subsets.

[0119] It is understandable that dividing the target parameters into target parameter subsets of the same communication address makes it easier to confirm abnormal data, which improves the accuracy of abnormal data processing.

[0120] In this embodiment, when determining the first elastic queue based on target parameters of the same acquisition frequency category, the data processing device also needs to separate the target parameters according to their communication addresses to remove abnormal data from the target parameters. The first elastic queue is then determined based on the processed subsets of target parameters. This ensures that the first elastic queue contains no abnormal data, thereby improving the accuracy of the time alignment of the operating condition parameter acquisition.

[0121] Figure 4 A flow chart of a data processing method according to an embodiment of the present invention is shown, wherein the data processing method includes:

[0122] Step S402, obtaining an operating parameter set of engineering equipment;

[0123] Step S404: pre-processing the operating condition parameter set to determine a plurality of first elastic queues with different acquisition frequencies;

[0124] Step S406, determining the preset type of operating condition parameters in the first elastic queue as primary data;

[0125] Step S408 , when the first elastic queue contains two primary data, confirm whether the time parameter of the first-order data in the first elastic queue is less than a first time threshold;

[0126] Step S410: If the time parameter is not less than the first time threshold and the first-order data is within the range of plus or minus half a period of the first primary data, the first-order data is parsed, the first-order data is deleted from the first elastic queue, and the next-order data is used as the first-order data.

[0127] Step S412: updating the first wide table object according to the parsing result of the first-order data until the time parameter of the first-order data of the first elastic queue is greater than the first time threshold, thereby determining a plurality of first wide table objects with different collection frequencies;

[0128] Step S414: determining a second wide table object according to the plurality of first wide table objects.

[0129] The time parameter is used to indicate the difference between the acquisition time of the first-order data and the time of acquiring the first main data, and the first time threshold is used to indicate the negative value of the time of acquiring 1 / 2 cycle of the main data.

[0130] In this embodiment, the time parameter is used to indicate the difference between the time of collecting the first-order data and the time of collecting the first main data, and the first time threshold is used to indicate the negative value of the time of collecting 1 / 2 cycle of the main data.

[0131] Specifically, the process of determining the first wide table object based on the first elastic queue is as follows: the data processing device determines a preset type of operating condition parameter in the first elastic queue as primary data. Specifically, the preset type of operating condition parameter is used to indicate a highly important operating condition parameter, such as engine speed.

[0132] Furthermore, when the first elastic queue contains two primary data, the data processing device determines whether a time parameter of the first-order data in the first elastic queue is less than a first time threshold. Specifically, if the first elastic queue contains two primary data, it indicates that the first-order data is suitable for collection time alignment with the first primary data. In this case, the magnitude of the time parameter of the first-order data is determined.

[0133] Furthermore, if the data processing device determines that the time parameter is not less than the first time threshold and the first-order data is within a period of plus or minus half of the first master data, the data processing device parses the first-order data, deletes the first-order data from the first elastic queue, and uses the next-order data as the first-order data. Specifically, if the time parameter is not less than the first time threshold and the first-order data is within a period of plus or minus half of the first master data, it indicates that the first-order data in the first elastic queue has not yet been aligned in terms of collection time, and the collection time of the first data is relatively close to that of the first master data. In this case, the first-order data is parsed to align its collection time with that of the first master data. Simultaneously, the first-order data is deleted from the first elastic queue, the next-order data is used as the new first-order data, and the above steps are repeated.

[0134] It is understandable that, since the first-order data is CAN data, it is necessary to parse the first-order data according to the J1939 protocol.

[0135] Furthermore, the data processing device updates the first wide table object based on the above-mentioned analysis results of the first-order data, that is, after aligning the collection time of the first-order data with the first main data, the analysis results are written into the first wide table object according to the attributes until all the operating condition parameters in the first elastic queue are updated to the first wide table object.

[0136] For example, the first elastic queue is as follows: Figure 10As shown in the figure, the gray rectangle represents the master data, T is the master data transmission period, and the other shapes represent other operating parameters in the elastic queue. These parameters will be aligned with the master data, that is, the time of the master data closest to them will be selected as their own time. When two master data appear in the queue, all data within the positive and negative half period of the first master data will use the time of the first master data as their own time. The first wide table object will be updated to achieve the collection time alignment of the operating parameters with the same collection frequency.

[0137] It should be noted that, when determining that the first elastic queue does not contain two master data, the data processing apparatus performs the step of determining the second wide table object according to multiple first elastic queues that do not contain two master data.

[0138] In this embodiment, the data processing device identifies a preset type of operating condition parameter in a first elastic queue as primary data. Upon confirming that the first elastic queue contains two primary data, that the time parameter of the first-order data in the first elastic queue is not less than a first time threshold, and that the first-order data is within the range of plus or minus 1 / 2 cycle time of the first primary data, the data processing device aligns the acquisition time of the first-order data with that of the first primary data, updates the first wide table object, and simultaneously deletes the first-order data from the first elastic queue and replaces the next-order data with the first-order data, until all operating condition parameters in the first elastic queue are updated to the first wide table object. In this way, the data processing device can align the acquisition times of operating condition parameters with the same acquisition frequency but different communication addresses, ensuring the accuracy of the time alignment of all operating condition parameters with different transmission frequencies in subsequent steps.

[0139] Figure 5 A flow chart of a data processing method according to an embodiment of the present invention is shown, wherein the data processing method includes:

[0140] Step S502, obtaining an operating parameter set of engineering equipment;

[0141] Step S504: pre-processing the operating condition parameter set to determine a plurality of first elastic queues with different acquisition frequencies;

[0142] Step S506, determining the preset type of operating condition parameters in the first elastic queue as primary data;

[0143] Step S508 , when the first elastic queue contains two primary data, confirm whether the time parameter of the first-order data in the first elastic queue is less than a first time threshold;

[0144] Step S510: If the time parameter is not less than the first time threshold and the first-order data is not within the time range of plus or minus half a period of the first main data, delete the first-order data from the first elastic queue and use the next-order data as the first-order data.

[0145] Step S512: If the time parameter is not less than the first time threshold and the first-order data is within the time range of plus or minus half a period of the first primary data, the first-order data is parsed, deleted from the first elastic queue, and the next-order data is used as the first-order data.

[0146] Step S514: updating the first wide table object according to the parsing result of the first-order data until the time parameter of the first-order data of the first elastic queue is greater than the first time threshold, thereby determining a plurality of first wide table objects with different collection frequencies;

[0147] Step S516: Determine a second wide table object according to the multiple first wide table objects.

[0148] In this embodiment, after confirming whether the time parameter of the first-priority data in the first elastic queue is less than the first time threshold, if the data processing device determines that the time parameter of the first-priority data is not less than the first time threshold and the first-priority data is not within the positive or negative 1 / 2 cycle time of the first main data, it indicates that the first-priority data cannot be aligned with the first main data in terms of collection time. In this case, the first-priority data is deleted from the first elastic queue, the next-priority data is used as the first-priority data, and the above-mentioned determination process is executed again.

[0149] In this embodiment, the data processing device determines whether the first-order data is suitable for collection time alignment with the first primary data by determining whether its time parameter is less than a first time threshold and whether the first-order data is within a period of plus or minus half a cycle of the first primary data. If not, the first-order data is deleted from the first elastic queue. This ensures the efficiency and accuracy of collection time alignment for operating condition parameters with the same collection frequency but different communication addresses.

[0150] Figure 6 A flow chart of a data processing method according to an embodiment of the present invention is shown, wherein the data processing method includes:

[0151] Step S602, obtaining an operating parameter set of engineering equipment;

[0152] Step S604: pre-processing the operating condition parameter set to determine a plurality of first elastic queues with different acquisition frequencies;

[0153] Step S606: determining a first wide table object according to the first elastic queue to determine a plurality of first wide table objects with different collection frequencies;

[0154] Step S608: determining a second elastic queue according to the plurality of first wide table objects;

[0155] Step S610: Determine a second wide table object according to the second elastic queue.

[0156] In this embodiment, the second flexible queue is a collection of operating condition parameters with different acquisition frequencies. It is understood that because the second flexible queue is determined based on multiple first wide table objects, the operating condition parameters of the same sampling frequency category in the second flexible queue have already been aligned in acquisition time.

[0157] Specifically, the process of determining the second wide table object based on multiple first wide table objects is as follows: the data processing device determines a second elastic queue based on the multiple first wide table objects, and then performs acquisition time alignment processing on the operating condition parameters of different acquisition frequencies in the second elastic queue to determine the second wide table object.

[0158] It should be noted that converting multiple first wide table objects into the second elastic queue is beneficial to time alignment processing.

[0159] In this embodiment, in the process of determining the second wide table object based on multiple first wide table objects, the data processing device improves the rate and accuracy of aligning the acquisition time of operating condition parameters with different acquisition frequencies by converting the multiple first wide table objects into a second elastic queue.

[0160] Figure 7 A flow chart of a data processing method according to an embodiment of the present invention is shown, wherein the data processing method includes:

[0161] Step S702, obtaining an operating parameter set of engineering equipment;

[0162] Step S704: pre-process the operating condition parameter set to determine a plurality of first elastic queues with different acquisition frequencies;

[0163] Step S706: determining a first wide table object according to the first elastic queue to determine a plurality of first wide table objects with different collection frequencies;

[0164] Step S708: determining a second elastic queue according to the plurality of first wide table objects;

[0165] Step S710, determining the operating condition parameter of the category with the highest collection frequency in the second elastic queue as the main data;

[0166] Step S712: When the second elastic queue contains two primary data, confirm whether the time parameter of the first-order data in the second elastic queue is less than the first time threshold;

[0167] In step S714, if the time parameter is not less than the first time threshold and the first-order data is within the range of plus or minus half a period of the first main data, the second wide table object is updated according to the first digital data, the first-order data is deleted from the second elastic queue, and the next-order data is used as the first-order data until the time parameter of the first-order data in the second elastic queue exceeds the first time threshold.

[0168] The time parameter is used to indicate the difference between the acquisition time of the first-order data and the acquisition time of the first main data, and the first time threshold is used to indicate the positive or negative value of the time of 1 / 2 cycle of the main data.

[0169] In this embodiment, the time parameter is used to indicate the difference between the acquisition time of the first-order data and the time of acquiring the first main data, and the first time threshold is used to indicate the positive or negative value of the time of acquiring 1 / 2 cycle of the main data.

[0170] Specifically, the process of determining the second wide table object according to the second elastic queue is as follows: the data processing device determines the operating condition parameter of the category with the highest collection frequency in the second elastic queue as the main data.

[0171] Furthermore, when the second elastic queue contains two primary data items, the data processing device determines whether a time parameter of the first-order data item in the second elastic queue is less than a first time threshold. Specifically, if the second elastic queue contains two primary data items, this indicates that the first-order data item is suitable for collection time alignment with the first primary data item. In this case, the magnitude of the time parameter of the first-order data item is determined.

[0172] Furthermore, when the data processing device determines that the time parameter is not less than the first time threshold and the first-order data is between the positive and negative 1 / 2 cycle time of the first main data, the data processing device updates the second wide table object according to the first-order data, deletes the first-order data from the second elastic queue, and uses the next-order data as the first-order data until the time parameter of the first-order data in the second elastic queue is greater than the first time threshold.

[0173] Specifically, if the time parameter is not less than the first time threshold and the first-order data falls within the range of plus or minus half a cycle of the first master data, the first-order data in the second elastic queue has not yet been time-aligned, and the collection time of the first data is close to that of the first master data. In this case, the first-order data is analyzed to align its collection time with that of the first master data. Simultaneously, the first-order data is deleted from the first elastic queue, and the next-order data is used as the new first-order data. The above steps are repeated until the time parameter of the first-order data in the second elastic queue exceeds the first time threshold, that is, until all operating condition parameters in the second elastic queue are updated to the second wide table object.

[0174] It can be understood that since the CAN data format operating condition parameters have been parsed in the process of determining the first wide table object based on the first elastic queue, the data in the second elastic queue is the parsed data. Therefore, in the process of determining the second wide table object based on the second elastic queue, there is no need to parse the operating condition parameters.

[0175] For example, the second elastic queue is also as follows Figure 10 As shown in the figure, the gray rectangle represents the master data, T is the master data transmission period, and the other shapes represent other operating parameters in the elastic queue. These parameters are aligned with the master data, that is, the time of the master data closest to them is selected as the default time. When two master data appear in the queue, all data within the positive and negative half period of the first master data is set to the time of the first master data as the default time. The first wide table object is updated to achieve time alignment of the operating parameters collected at different frequencies.

[0176] In this embodiment, the data processing device determines the operating condition parameter of the most frequently collected category in the second elastic queue as the primary data. Upon confirming that the second elastic queue contains two primary data, that the time parameter of the first-order data in the second elastic queue is not less than a first time threshold, and that the first-order data is between plus or minus 1 / 2 of the cycle time of the first primary data, the data processing device aligns the collection time of the first-order data with that of the first primary data, updates the second wide table object, and simultaneously deletes the first-order data from the second elastic queue and replaces the next-order data with the first-order data, until all operating condition parameters in the second elastic queue are updated to the second wide table object. In this way, the data processing device can align the collection times of operating condition parameters with different collection frequencies, allowing users to conduct multi-dimensional analysis of the operating conditions of engineering equipment in real time based on the second wide table object, thereby gaining a more comprehensive understanding of the working conditions of the engineering equipment.

[0177] Example 2:

[0178] Figure 8A schematic block diagram of a data processing device according to an embodiment of the present invention is shown, wherein the data processing device 800 includes: an acquisition unit 802 for acquiring an operating condition parameter set of engineering equipment; a processing unit 804 for preprocessing the operating condition parameter set to determine a plurality of first elastic queues with different acquisition frequencies; the processing unit 804 is further configured to determine a first wide table object based on the first elastic queue to determine a plurality of first wide table objects with different acquisition frequencies; the processing unit 804 is further configured to determine a second wide table object based on the plurality of first wide table objects; wherein the first wide table object is configured to indicate a parameter set of operating condition parameters collected at different communication addresses with the same acquisition frequency having aligned data acquisition time, and the second wide table object is configured to indicate a parameter set of operating condition parameters with different acquisition frequencies having aligned data acquisition time.

[0179] In this embodiment, the first elastic queue is used to indicate a set of operating condition parameters of the same acquisition frequency category; the first wide table object is used to indicate a parameter set of parsed operating condition parameters of different communication addresses with the same acquisition frequency that have aligned data acquisition time; and the second wide table object is used to indicate a parameter set of operating condition parameters of different acquisition frequencies that have aligned data acquisition time.

[0180] Specifically, first, the acquisition unit 802 acquires the operating parameters of the engineering equipment. Specifically, the engineering equipment is equipped with multiple sensors, which send the collected operating parameters to the Kafka message queue via the data acquisition device according to their corresponding acquisition frequencies. The acquisition unit 802 can obtain the above-mentioned operating parameter set by consuming the Kafka message queue. The above-mentioned Kafka message queue is a message queue for storing operating parameters in a data processing system developed by a program. Generally speaking, the operating parameters of the Kafka message queue are in the CAN data format.

[0181] Furthermore, processing unit 804 preprocesses the operating condition parameter set to determine multiple first elastic queues with different collection frequencies. Specifically, processing unit 804 sorts the operating condition parameters according to their collection frequency and the communication address from which they were collected, removing any abnormal data. After preprocessing, processing unit 804 uses operating condition parameters of the same collection frequency category as a first elastic queue and determines multiple first elastic queues with different collection frequencies.

[0182] Furthermore, processing unit 804 determines a first wide table object based on the first flexible queue to identify multiple first wide table objects with different acquisition frequencies. Specifically, processing unit 804 time-aligns the operating condition parameters in the first flexible queue according to a pre-programmed process to determine the first wide table object. Specifically, the first wide table object is configured with attributes such as the acquisition time, acquisition device number, pilot pressure, operating pressure, current, engine speed, torque, and temperature of the operating condition parameters. Processing unit 804 determines the first wide table object by filling the time-aligned operating condition parameters into the wide table object according to the corresponding attributes.

[0183] It can be understood that a first wide table object can be determined according to the first elastic queue of each sampling frequency, so the processing unit 804 can determine multiple first wide table objects of different sampling frequencies.

[0184] Furthermore, the processing unit 804 determines a second wide table object based on the plurality of first wide table objects. Specifically, the processing unit 804 performs time alignment processing on the operating condition parameters in the plurality of first wide table objects according to a pre-programmed process to determine a second wide table object having operating condition parameters of different acquisition frequencies aligned with the data acquisition time.

[0185] In this embodiment, the acquisition unit 802 first acquires a set of operating condition parameters for the engineering equipment. The processing unit 804 then preprocesses the set of operating condition parameters to determine multiple first elastic queues with different acquisition frequencies. The operating condition parameters in the first elastic queues are then time-aligned to determine multiple first wide table objects with different acquisition frequencies. After determining the multiple first wide table objects, the processing unit 804 then time-aligns the operating condition parameters in the first wide table objects to determine second wide table objects with operating condition parameters at different acquisition frequencies aligned with data acquisition times. This allows users to conduct multi-dimensional analysis of the operating conditions of the engineering equipment in real time based on the second wide table objects, thereby gaining a comprehensive understanding of the equipment's operational status. Furthermore, in this embodiment, acquisition time alignment is first performed on operating condition parameters with the same transmission frequency, followed by acquisition time alignment on all operating condition parameters with different transmission frequencies. This improves the efficiency and accuracy of data acquisition time alignment.

[0186] Furthermore, in this embodiment, in the step of preprocessing the operating condition parameter set and determining multiple first elastic queues with different acquisition frequencies, the processing unit 804 is specifically used to filter out target parameters in the operating condition parameter set according to a regular expression; divert the target parameters according to the acquisition frequency; and use the target parameters of the same acquisition frequency as a first elastic queue to determine multiple first elastic queues with different acquisition frequencies.

[0187] Furthermore, in this embodiment, in the step of using target parameters of the same acquisition frequency as a first elastic queue, the processing unit 804 is specifically used to divert the target parameters of the same acquisition frequency according to the communication address to determine multiple target parameter subsets; process abnormal parameters of the multiple target parameter subsets, and determine the first elastic queue based on the processed multiple target parameter subsets.

[0188] Furthermore, in this embodiment, in the step of determining the first wide table object based on the first elastic queue, the processing unit 804 is specifically configured to determine a preset type of operating condition parameter in the first elastic queue as primary data; when the first elastic queue contains two primary data, confirm whether a time parameter of the first-order data in the first elastic queue is less than a first time threshold; when the time parameter is not less than the first time threshold and the first-order data is between positive and negative 1 / 2 cycle time of the first primary data, parse the first-order data, delete the first-order data from the first elastic queue, and use the next-order data as the first-order data; and update the first wide table object based on the parsing result of the first digital data until the time parameter of the first-order data in the first elastic queue is greater than the first time threshold; wherein the time parameter is used to indicate the difference between the acquisition time of the first-order data and the acquisition time of the first primary data, and the first time threshold is used to indicate the positive or negative value of the acquisition time of 1 / 2 cycle time of the first primary data.

[0189] Furthermore, in this embodiment, after confirming whether the time parameter of the first-order data in the first elastic queue is less than the first time threshold, the processing unit 804 is also used to delete the first-order data from the first elastic queue and use the next-order data as the first-order data if the time parameter is not less than the first time threshold and the first-order data is not between the positive and negative 1 / 2 cycle time of the first main data.

[0190] Furthermore, in this embodiment, in the step of determining the second wide table object according to the plurality of first wide table objects, the processing unit 804 is specifically configured to determine the second elastic queue according to the plurality of first wide table objects; and determine the second wide table object according to the second elastic queue.

[0191] Furthermore, in this embodiment, in the step of determining the second wide table object based on the second elastic queue, the processing unit 804 is specifically configured to determine the operating condition parameter of the category with the highest collection frequency in the second elastic queue as the primary data; if the second elastic queue contains two primary data, determine whether the time parameter of the first-order data in the second elastic queue is less than a first time threshold; if the time parameter is not less than the first time threshold and the first-order data is between positive and negative 1 / 2 cycle time of the first primary data, update the second wide table object based on parsing the first-order data, delete the first-order data from the second elastic queue, and use the next-order data as the first-order data until the time parameter of the first-order data in the second elastic queue is greater than the first time threshold; wherein the time parameter is used to indicate the difference between the collection time of the first-order data and the collection time of the first primary data, and the first time threshold is used to indicate a positive or negative value of 1 / 2 cycle time of the primary data.

[0192] Example 3:

[0193] According to a third embodiment of the present invention, a readable storage medium is provided, on which a program or instruction is stored. When executed by a processor, the program or instruction implements the data processing method proposed in the above embodiment. Therefore, the readable storage medium has all the beneficial effects of the data processing method proposed in the above embodiment, and no further details are given here.

[0194] Example 4:

[0195] Figure 9 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown, wherein the electronic device 900 includes: a data processing device 800 as proposed in the above embodiment of the present invention, and / or a readable storage medium 902 as proposed in the above embodiment of the present invention. Therefore, the electronic device 900 has all the beneficial effects of the data processing device 800 according to the above embodiment of the present invention and / or the readable storage medium 902 proposed in the above embodiment of the present invention, which will not be repeated here.

[0196] Embodiment 5:

[0197] This embodiment combines Figure 11 and Figure 12 The data processing method proposed in the present invention is exemplified.

[0198] The design idea of the data processing method proposed in this embodiment is as follows: Figure 11 As shown, specifically, the data acquisition end in the figure is used to indicate multiple sensors on engineering equipment, and the platform end corresponds to the above-mentioned data processing device.

[0199] Specifically, this embodiment collects CAN data (operating parameters) on the data acquisition side and processes the data on the platform side. The data acquisition side immediately forwards the collected CAN data. Upon receiving the data, the platform side performs persistent storage, retaining the original CAN data for later data refreshes. Furthermore, the platform performs real-time alignment, parsing, and complement operations on the CAN data to determine a wide table object of CAN data with different acquisition frequencies aligned with the data acquisition time. This allows users to conduct multi-dimensional analysis of the operating conditions of engineering equipment based on the wide table object, providing a comprehensive understanding of the working status of the engineering equipment.

[0200] Specifically, the execution flow of the data processing method proposed in this embodiment is as follows: Figure 12 As shown, the acquisition frequencies include 50 Hz and 5 Hz for exemplary description, wherein the data processing method includes:

[0201] Step S1202, obtaining CAN data of different frequencies;

[0202] Step S1204, the regular expression matches valid data;

[0203] Step S1206, splitting according to frequency type;

[0204] Step S1208, splitting according to the communication address;

[0205] Step S1210, cleaning abnormal data;

[0206] Step S1212, cleaning abnormal data;

[0207] Step S1214, generating an elastic queue (50HZ);

[0208] Step S1216, determine whether there are two master data; if yes, go to step S1218, otherwise go to step S1248;

[0209] Step S1218, creating / updating a first wide table object;

[0210] Step S1220, determine whether the queue head is smaller than the first main data -T 50HZ / 2; if yes, go to step S1222; if no, go to step S1224;

[0211] Step S1222, throw the head of the team;

[0212] Step S1224, determine whether the team head is in the first main data ±T 50HZ / 2; if yes, go to step S1226; if no, go to step S1248;

[0213] Step S1226, throw the queue head object and parse the CAN data;

[0214] Step S1228, splitting according to the communication address;

[0215] Step S1230, cleaning abnormal data;

[0216] Step S1232, cleaning abnormal data;

[0217] Step S1234, generating an elastic queue (5HZ);

[0218] Step S1236, determine whether there are two master data; if yes, go to step S1238, otherwise go to step S1248;

[0219] Step S1238, creating / updating a first wide table object;

[0220] Step S1240, determine whether the queue head is smaller than the first main data - T 5HZ / 2; if yes, go to step S1242; if no, go to step S1244;

[0221] Step S1242, throw the head of the queue;

[0222] Step S1244, determine whether the team head is within the first main data ±T5 HZ / 2; if yes, go to step S1246; if no, go to step S1248;

[0223] Step S1246, throw the queue head object and parse the CAN data;

[0224] Step S1248, generating a global elastic queue (50HZ);

[0225] Step S1250, determine whether there are two master data; if yes, execute step S1252, otherwise execute step S1202;

[0226] Step S1252, creating / updating a global wide table object;

[0227] Step S1254, determine whether the queue head is smaller than the first main data - T 50HZ / 2; if yes, go to step S1256; if no, go to step S1258;

[0228] Step S1256, throw the head of the queue;

[0229] Step S1258, determine whether the team head is in the first master data ±T 50HZ / 2; if yes, go to step S1260; if no, go to step S1262;

[0230] Step S1260, throwing the head of the queue object;

[0231] Step S1262: Determine that the data alignment at this moment is completed.

[0232] In this embodiment, the elastic queue 50HZ and the elastic queue 5HZ correspond to the first elastic queue, the global elastic queue 50HZ corresponds to the second elastic queue, the global wide table object corresponds to the second wide table object, and the head of the queue corresponds to the first-order data.

[0233] In this embodiment, the data processing device obtains CAN data of different frequencies by consuming the Kafka message queue, and filters out data that meets the conditions through regular expressions, such as data that meets the specifications of a certain device.

[0234] Furthermore, the data processing device diverts the CAN data according to the set CAN data sending frequency, and further diverts the CAN data with the same sending frequency according to different communication addresses, and cleans the data with the same communication address, such as removing the last piece of data when the current and last two pieces of data are far less than the sending cycle.

[0235] Furthermore, the data processing device adds the cleaned CAN data with the same sending frequency but different communication addresses into the elastic queue.

[0236] Furthermore, when the elastic queue contains two main data, the data processing device creates a new temporary wide table object (elastic queue 50HZ, elastic queue 5HZ), and then determines whether the time of the head data (first-order data) is less than half the negative cycle time of the first main data (main data: CAN data of a more important type in the elastic queue). If so, the head data is thrown out and discarded, and then the new head data is judged. If not, it is determined whether the data is between the positive and negative half cycle time of the first main data. If so, the head data is thrown out, the CAN data is parsed, the first wide table object is updated, and the next head data is judged until all the data in the elastic queue is updated to the first wide table object.

[0237] Furthermore, the data processing device determines the global elastic queue 50HZ based on multiple first wide table objects, creates a global wide table object, and then determines whether the time of the head data is less than half the negative cycle time of the first main data (main data: CAN data of the highest collection frequency category). If so, the head data is thrown out and discarded, and then the new head data is judged. If not, it is determined whether the data is between the positive and negative half cycle time of the first main data. If so, the head data is thrown out, the global wide table object is updated, and the next head data is judged until all the data in the global elastic queue is updated to the global wide table object.

[0238] In this specification, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance, unless otherwise expressly specified or limited. Terms such as "connect," "install," and "fix" should be interpreted broadly. For example, "connect" can refer to a fixed connection, a detachable connection, or an integral connection; and can be directly connected or indirectly connected through an intermediary. Those skilled in the art will understand the specific meanings of these terms in the present invention based on specific circumstances.

[0239] Throughout this specification, terms such as "one embodiment," "some embodiments," and "specific embodiments" mean that the specific features, structures, materials, or characteristics described in conjunction with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0240] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in this field can implement it. When the combination of technical solutions is mutually contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0241] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A data processing method, characterized in that: The data processing method includes: Obtain the operating parameter set of engineering equipment; Preprocessing the operating condition parameter set to determine a plurality of first elastic queues with different acquisition frequencies; determining a first wide table object according to the first elastic queue to determine a plurality of first wide table objects with different acquisition frequencies; determining a second wide table object according to a plurality of said first wide table objects; The first wide table object is used to indicate a parameter set of operating parameters collected at the same collection frequency but different communication addresses with aligned data collection time, and the second wide table object is used to indicate a parameter set of operating parameters collected at different collection frequencies with aligned data collection time; Preprocessing the operating condition parameter set to determine a plurality of first elastic queues with different acquisition frequencies specifically includes: Filtering out target parameters in the operating condition parameter set according to a regular expression; Dividing the target parameters according to the acquisition frequency; Taking the target parameter of the same acquisition frequency as a first elastic queue, determining a plurality of first elastic queues of different acquisition frequencies; The regular expression is used to indicate a preset screening rule, and the first elastic queue is used to indicate a set of operating condition parameters of the same acquisition frequency category.

2. The data processing method according to claim 1, wherein: The target parameters with the same collection frequency are used as a first elastic queue, specifically including: Divide the target parameters of the same acquisition frequency according to the communication address to determine multiple target parameter subsets; Abnormal parameters of the plurality of target parameter subsets are processed, and the first elastic queue is determined according to the processed plurality of target parameter subsets.

3. The data processing method according to claim 1, wherein: The determining of the first wide table object according to the first elastic queue specifically includes: Determining the preset type of operating condition parameters in the first elastic queue as primary data; In a case where the first elastic queue contains two primary data, determining whether a time parameter of the first-order data in the first elastic queue is less than a first time threshold; If the time parameter is not less than the first time threshold and the first-order data is within the range of plus or minus half a period of the first primary data, the first-order data is parsed, the first-order data is deleted from the first elastic queue, and the next-order data is used as the first-order data; the first time threshold is used to indicate a negative value of half a period of time for collecting primary data; The first wide table object is updated according to the parsing result of the first-order data until a time parameter of the first-order data of the first elastic queue is greater than a second time threshold; the second time threshold is a positive value indicating a time for collecting 1 / 2 cycle of the primary data; The time parameter is used to indicate the difference between the time when the first-order data is collected and the time when the first main data is collected.

4. The data processing method according to claim 3, wherein: After confirming whether the time parameter of the first-order data in the first elastic queue is less than the first time threshold, the data processing method further includes: When the time parameter is not less than the first time threshold and the first-priority data is not between the plus or minus 1 / 2 cycle time of the first main data, the first-priority data is deleted from the first elastic queue and the next-priority data is used as the first-priority data.

5. The data processing method according to claim 1, wherein: The determining of the second wide table object according to the plurality of first wide table objects specifically includes: determining a second elastic queue according to the plurality of first wide table objects; The second wide table object is determined according to the second elastic queue.

6. The data processing method according to claim 5, characterized in that: The determining the second wide table object according to the second elastic queue specifically includes: Determine the operating condition parameters of the category with the highest collection frequency in the second elastic queue as primary data; In a case where the second elastic queue contains two primary data, determining whether a time parameter of the first-order data in the second elastic queue is less than a first time threshold; If the time parameter is not less than the first time threshold and the first-order data is within the time range of plus or minus half a cycle of the first primary data, the second wide table object is updated according to the first-order data, the first-order data is deleted from the second elastic queue, and the next-order data is used as the first-order data until the time parameter of the first-order data in the second elastic queue is greater than the second time threshold. The first time threshold is used to indicate a negative value of the time for collecting half a cycle of the primary data, and the second time threshold is used to indicate a positive value of the time for collecting half a cycle of the primary data. The time parameter is used to indicate the difference between the time when the first-order data is collected and the time when the first main data is collected.

7. A data processing device, characterized in that: include: An acquisition unit, used to acquire a set of operating condition parameters of engineering equipment; a processing unit, configured to pre-process the operating condition parameter set and determine a plurality of first elastic queues with different acquisition frequencies; The processing unit is further configured to determine a first wide table object according to the first elastic queue, so as to determine a plurality of first wide table objects with different acquisition frequencies; The processing unit is further configured to determine a second wide table object based on a plurality of the first wide table objects; The first wide table object is used to indicate a parameter set of operating parameters collected at the same collection frequency but different communication addresses with aligned data collection time, and the second wide table object is used to indicate a parameter set of operating parameters collected at different collection frequencies with aligned data collection time; The processing unit is further configured to filter out target parameters from the operating condition parameter set according to a regular expression; divide the target parameters according to a collection frequency; and use the target parameters of the same collection frequency as a first elastic queue to determine a plurality of first elastic queues of different collection frequencies; The regular expression is used to indicate a preset screening rule, and the first elastic queue is used to indicate a set of operating condition parameters of the same acquisition frequency category.

8. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the data processing method according to any one of claims 1 to 6 are implemented.

9. An electronic device, characterized in that: include: The data processing device according to claim 7; and / or The readable storage medium according to claim 8.

Images