Excavator working condition recognition method and system based on whole machine information space-time fusion

Abstract

The application belongs to the technical field of excavator working condition recognition, and discloses an excavator working condition recognition method and system based on whole machine information space-time fusion, which collects multi-dimensional running information of the excavator, takes data standardization and segmentation as pretreatment, inputs the obtained time sequence matrix into a constructed time attention module and a spatial attention module in parallel, respectively inputs the time sequence matrix into a time feature extraction network and a spatial feature extraction network, splices along a variable axis to obtain a space-time fusion feature space matrix, inputs the obtained space-time fusion feature space matrix into a designed working condition information learning module for processing to obtain a final working condition recognition result, and improves working condition classification precision; the application effectively combines space-time correlation of excavator sensing data, pays attention to intra-class and inter-class distance of classification features, enhances model recognition ability for continuous working conditions of the excavator, and improves recognition precision of action modes of the excavator.

Description

Technical Field

[0001] This invention belongs to the field of excavator operating condition identification technology, and particularly relates to an excavator operating condition identification method and system based on spatiotemporal fusion of whole machine information. Background Technology

[0002] Excavators, as multi-functional construction equipment, are widely used in mining, road construction, and other construction projects due to their strong adaptability, durability, and low cost. However, due to their current discrete power control method, the engine's operating point often falls outside the high-efficiency range during operation, which is detrimental to energy conservation and environmental protection requirements. Therefore, accurate identification of excavator operating conditions is of great significance for designing green, energy-saving, and efficient control strategies.

[0003] The rise of machine learning and deep learning technologies has opened up new avenues for tasks such as excavator condition identification and energy consumption prediction. However, current research on excavator condition identification mainly relies on single signal and video frame image classification, and there is still considerable room for development in the area of ​​overall excavator condition information.

[0004] Therefore, combining deep learning methods to effectively learn features from the excavator's full operating conditions, utilizing the temporal correlation of the excavator's continuous operation and the spatial correlation of its component movements, and combining the complementary characteristics between multi-sensor signal variables, to extract operating condition information with spatiotemporal fusion characteristics, and ultimately achieve high-precision identification of excavator movements and energy-saving and green control of the excavator, has important engineering significance.

[0005] Based on the above analysis, the problems and shortcomings of the existing technology are as follows:

[0006] Current discrete power control methods often cause the engine's operating point to be outside the high-efficiency zone during operation, which is not conducive to energy conservation and environmental protection requirements. Summary of the Invention

[0007] To address the problems existing in the prior art, this invention provides a method for identifying the working condition of excavators based on spatiotemporal fusion of whole machine information.

[0008] This invention is implemented as follows: A method for identifying the working condition of an excavator based on spatiotemporal fusion of whole-machine information includes:

[0009] Step 1: Use a signal acquisition system to acquire multidimensional operating information of the excavator, and perform data processing and labeling to obtain a multivariate time series with standardized characteristics; compile the above standardized multivariate time series data into a dataset to obtain a time series input matrix for working condition classification;

[0010] Step 2: Input the obtained time series matrix in parallel into the constructed time attention module and spatial attention module, the time feature extraction network composed of M long short-term memory networks, and the spatial feature extraction network composed of N residual convolutional networks to obtain time features and spatial features with multivariate working condition information, respectively; concatenate the obtained time features along the variable axis to obtain the spatiotemporal fusion feature space matrix;

[0011] Step 3: Input the spatiotemporal feature space matrix obtained in Step 2 into the constructed working condition information learning module for processing to learn advanced working condition features with identification information; input the above advanced working condition features into the Softmax classifier for classification to generate the final working condition identification result.

[0012] Furthermore, step 1 includes the following steps:

[0013] Step 11: Standardize the obtained raw multidimensional operational information using the maximum-minimum normalization method;

[0014] The calculation method is as follows:

[0015]

[0016] Where χ represents the value of the input sample point, χ o χ represents the numerical value of the original sample points. min χ represents the minimum value in the original data. max Represents the maximum value in the original data;

[0017] Step 12: Use the sliding window segmentation method to segment the standardized multidimensional operational information obtained in Step 1. The sliding window size is L and the step size is S, to obtain several multivariate time series input matrices, which constitute a dataset for operational condition identification.

[0018] Furthermore, step 2 includes the following steps:

[0019] Step 21: Input the dataset obtained in Step 1 into the constructed temporal attention module and spatial attention module in parallel, and extract the temporal attention features and spatial attention features respectively;

[0020] Step 22: Set the structures of the temporal feature extraction module and the spatial feature extraction module respectively. The temporal feature extraction module consists of M cascaded long short-term memory networks. The spatial feature extraction module consists of N residual convolutional networks. Each residual convolutional network consists of two convolutional layers, two batch normalization layers, and one ReLU activation layer, and is connected by branches from input to output to form a complete residual convolutional network. Connect the N residual convolutional networks together to form the spatial feature extraction module.

[0021] Step 23: Input the temporal attention features and spatial attention features extracted in Step 21 into the temporal feature extraction module and the spatial feature extraction module respectively, and extract the temporal features and spatial features with multivariate working condition information respectively.

[0022] Step 24: The temporal and spatial features with multivariable operating condition information obtained in Step 23 above are concatenated according to the variable axis to obtain a spatiotemporal fusion feature space matrix with multivariable operating condition information, which is used to extract advanced operating condition features with identification information in the subsequent process.

[0023] Furthermore, step 3 includes the following steps:

[0024] Step 31: Input the spatiotemporal fusion feature space matrix obtained in Step 24 into the designed working condition feature learning module, define the working condition feature learning loss function as the algorithm for learning advanced working condition information features, and the specific algorithm is as follows:

[0025]

[0026] Where F is the feature space contained in the spatiotemporal fusion feature space matrix, and c i and c j Δc represents the center of the i-th and j-th samples respectively. i and Δc j k represents the distances from the farthest sample center in categories i and j, respectively. i and k j Representing the sample center c respectively i and c j Distance to the category boundary.

[0027] Step 32: Define the excavator working condition classification task as a 5-class classification problem;

[0028] Step 33: Input the advanced working condition information features obtained in Step 31 into a Softmax classifier with a cross-entropy loss function to classify the working condition modes of the excavator. The calculation formula for the cross-entropy loss is as follows:

[0029]

[0030] Among them, y i f(x) represents the sample label value. i () represents the model's predicted value;

[0031] Step 34: The total loss function of the model is expressed as follows:

[0032] Loss = Loss cross-entropy +λLoss CMA

[0033] Where λ is the weight coefficient, used to control the feature learning effect; the obtained result is used as the network output to obtain the final working condition recognition result.

[0034] Another objective of this invention is to provide an excavator operating condition identification system based on spatiotemporal fusion of whole-machine information, comprising:

[0035] The data processing module is used to acquire multidimensional operating information of the excavator using the signal acquisition system, and to perform data processing and labeling to obtain a multivariate time series with standardized characteristics; the standardized multivariate time series data is then used to create a dataset to obtain a time series input matrix for working condition classification.

[0036] The feature extraction module is used to input the obtained time series matrix in parallel into the constructed time attention module and spatial attention module, which are composed of a time feature extraction network consisting of M long short-term memory networks and a spatial feature extraction network consisting of N residual convolutional networks, to obtain time features and spatial features with multivariate working condition information, respectively; the acquired time features are concatenated along the variable axis to obtain the spatiotemporal fusion feature space matrix;

[0037] The classification module is used to input the obtained spatiotemporal feature space matrix into the constructed working condition information learning module for processing, and learn advanced working condition features with identification information; the advanced working condition features are then input into the Softmax classifier for classification to generate the final working condition identification result.

[0038] Another object of the present invention is to provide a computer device, the computer device including a memory and a processor, the memory storing a computer program, the computer program being executed by the processor causing the processor to perform the steps of the excavator working condition identification method based on spatiotemporal fusion of whole machine information.

[0039] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the excavator condition identification method based on spatiotemporal fusion of whole machine information.

[0040] Another objective of this invention is to provide an information data processing terminal, which is used to implement the excavator condition identification system based on spatiotemporal fusion of whole machine information.

[0041] Based on the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by this invention are as follows:

[0042] First, this invention collects multi-dimensional operational information of excavators, standardizes and segments the data as preprocessing, and then inputs the resulting time series matrix into the constructed time attention module and spatial attention module in parallel. These are then input into the time feature extraction network and spatial feature extraction network, respectively, and concatenated along the variable axis to obtain a spatiotemporal fusion feature space matrix. The obtained spatiotemporal fusion feature space matrix is ​​then input into the designed working condition information learning module for processing to obtain the final working condition identification result, thus improving the accuracy of working condition classification. This invention effectively combines the spatiotemporal correlation of excavator sensor data and focuses on the intra-class and inter-class distances of classification features, enhancing the model's ability to identify continuous working conditions of excavators and improving the accuracy of excavator action pattern recognition.

[0043] Second, the technical solution of the present invention solves the following main problems existing in the current excavator working condition identification technology:

[0044] 1. Insufficient data utilization: Existing technologies often only focus on the single dimension or a few dimensions of the excavator's operation information, ignoring the inherent connections and spatiotemporal characteristics between the multidimensional operation information of the whole machine, which limits the accuracy and comprehensiveness of working condition identification.

[0045] 2. Insufficient feature extraction: In terms of feature extraction, existing technologies may lack effective means to simultaneously capture complex working condition features in both the time and spatial domains, which affects the precision and robustness of working condition identification.

[0046] 3. Weak model generalization ability: Due to the aforementioned issues with data utilization and feature extraction, existing models often struggle to adapt to different excavator operating conditions, resulting in weak generalization ability.

[0047] The present invention solves the above problems through the following significant technical advancements:

[0048] 1. Spatiotemporal Fusion of Whole Machine Information: This invention innovatively proposes a working condition identification method based on spatiotemporal fusion of whole machine information, which makes full use of the multidimensional operating information of the excavator and considers the spatiotemporal characteristics of the information, significantly improving the comprehensiveness and accuracy of working condition identification.

[0049] 2. Deep Feature Extraction: By constructing a feature extraction network consisting of a temporal attention module, a spatial attention module, a long short-term memory network, and a residual convolutional network, this invention can deeply extract temporal and spatial features with multivariate working condition information, thereby enhancing the precision and robustness of working condition identification.

[0050] 3. Strong generalization ability model: Thanks to the full utilization of machine information and deep feature extraction, the working condition identification model constructed in this invention has a stronger generalization ability, which can better adapt to the excavator's operating status under different working conditions and improve its practical application value. Attached Figure Description

[0051] Figure 1 This is a flowchart of the excavator working condition identification method based on spatiotemporal fusion of whole machine information provided in the embodiments of the present invention.

[0052] Figure 2 This is a structural block diagram of an excavator working condition identification system based on spatiotemporal fusion of whole machine information provided in an embodiment of the present invention.

[0053] Figure 3 This is a flowchart of signal acquisition provided in an embodiment of the present invention.

[0054] Figure 4 This is a flowchart of multimodal signal feature fusion provided in an embodiment of the present invention.

[0055] Figure 5 This is a diagram showing the effect of comparing the method of the present invention with other existing method examples under test conditions of different working cycles of a real device, as provided in the embodiments of the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0057] like Figure 1 As shown in the figure, the excavator working condition identification method based on spatiotemporal fusion of whole machine information provided by the embodiment of the present invention includes the following steps:

[0058] S101: Use a signal acquisition system to acquire multidimensional operating information of the excavator, and perform data processing and labeling to obtain a multivariate time series with standardized characteristics; compile the above standardized multivariate time series data into a dataset to obtain a time series input matrix for working condition classification;

[0059] S102: The obtained time series matrix is ​​input in parallel into the constructed time attention module and spatial attention module, the time feature extraction network composed of M long short-term memory networks, and the spatial feature extraction network composed of N residual convolutional networks to obtain time features and spatial features with multivariate working condition information, respectively; the obtained time features are concatenated along the variable axis to obtain the spatiotemporal fusion feature space matrix;

[0060] S103: Input the spatiotemporal feature space matrix obtained in S102 into the constructed working condition information learning module for processing, and learn advanced working condition features with identification information; input the above advanced working condition features into the Softmax classifier for classification, and generate the final working condition identification result.

[0061] S101 provided in this embodiment of the invention includes the following steps:

[0062] Step 11: Standardize the obtained raw multidimensional operational information using the maximum-minimum normalization method;

[0063] The calculation method is as follows:

[0064]

[0065] Where χ represents the value of the original sample point, χ min χ represents the minimum value in the original data. max Represents the maximum value in the original data;

[0066] Step 12: Use the sliding window segmentation method to segment the standardized multidimensional operational information obtained in Step 1. The sliding window size is L and the step size is S, to obtain several multivariate time series input matrices, which constitute a dataset for operational condition identification.

[0067] S102 provided in this embodiment of the invention includes the following steps:

[0068] Step 21: Input the dataset obtained in Step 1 into the constructed temporal attention module and spatial attention module in parallel, and extract the temporal attention features and spatial attention features respectively;

[0069] Step 22: Set the structures of the temporal feature extraction module and the spatial feature extraction module respectively. The temporal feature extraction module consists of M cascaded long short-term memory networks. The spatial feature extraction module consists of N residual convolutional networks. Each residual convolutional network consists of two convolutional layers, two batch normalization layers, and one ReLU activation layer, and is connected by branches from input to output to form a complete residual convolutional network. Connect the N residual convolutional networks together to form the spatial feature extraction module.

[0070] Step 23: Input the temporal attention features and spatial attention features extracted in Step 21 into the temporal feature extraction module and the spatial feature extraction module respectively, and extract the temporal features and spatial features with multivariate working condition information respectively.

[0071] Step 24: The temporal and spatial features with multivariable operating condition information obtained in Step 23 above are concatenated according to the variable axis to obtain a spatiotemporal fusion feature space matrix with multivariable operating condition information, which is used to extract advanced operating condition features with identification information in the subsequent process.

[0072] S103 provided in this embodiment of the invention includes the following steps:

[0073] Step 31: Input the spatiotemporal fusion feature space matrix obtained in Step 24 into the designed working condition feature learning module, define the working condition feature learning loss function as the algorithm for learning advanced working condition information features, and the specific algorithm is as follows:

[0074]

[0075] Where F is the feature space contained in the spatiotemporal fusion feature space matrix, and c i and c j Δc represents the center of the i-th and j-th samples respectively. i and Δc j k represents the distances from the farthest sample center in categories i and j, respectively. i and k j Representing the sample center c respectively i and c j Distance to the category boundary.

[0076] Step 32: Define the excavator working condition classification task as a 5-class classification problem;

[0077] Step 33: Input the advanced working condition information features obtained in Step 31 into a Softmax classifier with a cross-entropy loss function to classify the working condition modes of the excavator. The calculation formula for the cross-entropy loss is as follows:

[0078]

[0079] Among them, y i f(x) represents the sample label value. i () represents the model's predicted value;

[0080] Step 34: The total loss function of the model is expressed as follows:

[0081] Loss = Loss cross-entropy +λLoss CMA

[0082] Where λ is the weight coefficient, used to control the feature learning effect; the obtained result is used as the network output to obtain the final working condition recognition result.

[0083] like Figure 2 As shown, an excavator condition identification system based on spatiotemporal fusion of whole-machine information provided by an embodiment of the present invention includes:

[0084] The data processing module is used to acquire multidimensional operating information of the excavator using the signal acquisition system, and to perform data processing and labeling to obtain a multivariate time series with standardized characteristics; the standardized multivariate time series data is then used to create a dataset to obtain a time series input matrix for working condition classification.

[0085] The feature extraction module is used to input the obtained time series matrix in parallel into the constructed time attention module and spatial attention module, which are composed of a time feature extraction network consisting of M long short-term memory networks and a spatial feature extraction network consisting of N residual convolutional networks, to obtain time features and spatial features with multivariate working condition information, respectively; the acquired time features are concatenated along the variable axis to obtain the spatiotemporal fusion feature space matrix;

[0086] The classification module is used to input the obtained spatiotemporal feature space matrix into the constructed working condition information learning module for processing, and learn advanced working condition features with identification information; the advanced working condition features are then input into the Softmax classifier for classification to generate the final working condition identification result.

[0087] Another object of the present invention is to provide a computer device, the computer device including a memory and a processor, the memory storing a computer program, the computer program being executed by the processor causing the processor to perform the steps of the excavator working condition identification method based on spatiotemporal fusion of whole machine information.

[0088] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the excavator condition identification method based on spatiotemporal fusion of whole machine information.

[0089] Another objective of this invention is to provide an information data processing terminal, which is used to implement the excavator condition identification system based on spatiotemporal fusion of whole machine information.

[0090] Specific implementation of the present invention:

[0091] like Figure 1 As shown, a method for excavator condition identification based on spatiotemporal fusion of whole-machine information is provided, including the following steps:

[0092] Step 1: As Figure 3 As shown in the signal acquisition system, the system acquires multidimensional operational information of the excavator, processes and labels the data to obtain a multivariate time series with standardized characteristics. This standardized multivariate time series data is then used to create a dataset, resulting in a time series input matrix for operational condition classification. The specific steps are as follows:

[0093] Step 11: Standardize the obtained raw multidimensional operational information using the maximum-minimum normalization method;

[0094] The calculation method is as follows:

[0095]

[0096] Where χ represents the value of the original sample point, χ min χ represents the minimum value in the original data. max Represents the maximum value in the original data;

[0097] Step 12: Use the sliding window segmentation method to segment the standardized multidimensional operational information obtained in Step 1. The sliding window size is L and the step size is S, to obtain several multivariate time series input matrices, which constitute a dataset for operational condition identification.

[0098] Step 2: As Figure 4 As shown in the flowchart of feature extraction and fusion based on whole-machine information, the obtained time series matrix is ​​input in parallel into the constructed time attention module and spatial attention module, the time feature extraction network composed of M long short-term memory networks, and the spatial feature extraction network composed of N residual convolutional networks, to obtain time features and spatial features with multivariate operating condition information, respectively. The acquired time features are concatenated along the variable axis to obtain the spatiotemporal fusion feature space matrix. The specific operation steps are as follows:

[0099] Step 21: Input the dataset obtained in Step 1 into the constructed temporal attention module and spatial attention module in parallel, and extract the temporal attention features and spatial attention features respectively;

[0100] Step 22: Set the structures of the temporal feature extraction module and the spatial feature extraction module respectively. The temporal feature extraction module consists of M cascaded long short-term memory networks. The spatial feature extraction module consists of N residual convolutional networks. Each residual convolutional network consists of two convolutional layers, two batch normalization layers, and one ReLU activation layer, and is connected by branches from input to output to form a complete residual convolutional network. Connect the N residual convolutional networks together to form the spatial feature extraction module.

[0101] Step 23: Input the temporal attention features and spatial attention features extracted in Step 21 into the temporal feature extraction module and the spatial feature extraction module respectively, and extract the temporal features and spatial features with multivariate working condition information respectively.

[0102] Step 24: The temporal and spatial features with multivariable operating condition information obtained in Step 23 above are concatenated according to the variable axis to obtain a spatiotemporal fusion feature space matrix with multivariable operating condition information, which is used to extract advanced operating condition features with identification information in the subsequent process.

[0103] Step 3: As Figure 4As shown in the flowchart of feature extraction and fusion based on whole machine information, the spatiotemporal feature space matrix obtained in step 2 is input into the constructed working condition information learning module for processing to learn high-level working condition features with identification information; the above high-level working condition features are input into the Softmax classifier for classification to generate the final working condition identification result. The specific operation steps are as follows:

[0104] Step 31: Input the spatiotemporal fusion feature space matrix obtained in Step 24 into the designed working condition feature learning module, define the working condition feature learning loss function as the algorithm for learning advanced working condition information features, and the specific algorithm is as follows:

[0105]

[0106] Where F is the feature space contained in the spatiotemporal fusion feature space matrix, and c i and c j Δc represents the center of the i-th and j-th samples respectively. i and Δc j k represents the distances from the farthest sample center in categories i and j, respectively. i and k j Representing the sample center c respectively i and c j Distance to the category boundary.

[0107] Step 32: Define the excavator working condition classification task as a 5-class classification problem;

[0108] Step 33: Input the advanced working condition information features obtained in Step 31 into a Softmax classifier with a cross-entropy loss function to classify the working condition modes of the excavator. The calculation formula for the cross-entropy loss is as follows:

[0109]

[0110] Among them, y i f(x) represents the sample label value. i () represents the model's predicted value;

[0111] Step 34: The total loss function of the model is expressed as follows:

[0112] Loss = Loss cross-entropy +λLoss CMA

[0113] Where λ is the weight coefficient, used to control the feature learning effect; the obtained result is used as the network output to obtain the final working condition recognition result.

[0114] like Figure 5As shown, under test conditions of different working cycles of a real device, the method of this invention is compared with other existing methods. This invention achieves the highest recognition accuracy, while maintaining stable high accuracy across different cycle counts. In contrast, the accuracy of other methods gradually decreases with increasing cycle counts, demonstrating the effectiveness and robustness of this invention and verifying its validity.

[0115] This invention also provides an excavator operating condition identification system based on spatiotemporal fusion of whole-machine information, comprising:

[0116] The signal acquisition and processing module is used to acquire multi-dimensional operating information of the excavator, perform data processing and data labeling, obtain multivariate time series with standardized characteristics, and then create a time series input matrix for working condition classification.

[0117] The spatiotemporal feature extraction module is used to input the time series matrix into the temporal attention module and the spatial attention module in parallel. Through the temporal feature extraction network composed of a long short-term memory network and the spatial feature extraction network composed of a residual convolutional network, the temporal features and spatial features with multivariate working condition information are obtained respectively, and these features are concatenated into a spatiotemporal fusion feature space matrix.

[0118] The working condition identification module is used to input the spatiotemporal feature space matrix into the working condition information learning module for processing, learn high-level working condition features with identification information, and input these features into the Softmax classifier for classification to produce the final working condition identification result.

[0119] The signal acquisition and processing module further includes:

[0120] The standardization processing unit is used to standardize the original multidimensional operational information using the maximum-minimum value normalization method;

[0121] The data segmentation unit is used to segment the standardized multidimensional operational information using the sliding window segmentation method to obtain several multivariate time series input matrices, which constitute a dataset for operational condition identification.

[0122] The spatiotemporal feature extraction module further includes:

[0123] The attention feature extraction unit is used to input the dataset in parallel into the temporal attention module and the spatial attention module, and extract temporal attention features and spatial attention features respectively;

[0124] The feature extraction network unit is used to set the structure of the temporal feature extraction module and the spatial feature extraction module, respectively extracting temporal features and spatial features with multivariate working condition information, and concatenating these features into a spatiotemporal fusion feature space matrix.

[0125] The operating condition identification module further includes:

[0126] The feature learning unit is used to input the spatiotemporal fusion feature space matrix into the working condition feature learning module, and defines the working condition feature learning loss function as an algorithm for learning advanced working condition information features.

[0127] The classification and loss calculation unit is used to define the excavator working condition classification task. It inputs the learned high-level working condition information features into a Softmax classifier with a cross-entropy loss function for classification, and calculates the total loss function of the model to obtain the final working condition identification result.

[0128] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0129] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for identifying the working condition of an excavator based on spatiotemporal fusion of whole-machine information, characterized in that, Includes the following steps: Step 1: Use a signal acquisition system to acquire multidimensional operating information of the excavator, and perform data processing and labeling to obtain a multivariate time series with standardized characteristics; compile the above standardized multivariate time series data into a dataset to obtain a time series input matrix for working condition classification; Step 2: Input the obtained time series matrix in parallel into the constructed time attention module and spatial attention module, the time feature extraction network composed of M long short-term memory networks, and the spatial feature extraction network composed of N residual convolutional networks, to obtain time features and spatial features with multivariate working condition information, respectively. The acquired temporal features are concatenated along the variable axis to obtain the spatiotemporal fusion feature space matrix; Step 3: Input the spatiotemporal feature space matrix obtained in Step 2 into the constructed working condition feature learning module for processing, and define the working condition feature learning loss function as the algorithm for learning advanced working condition information features. The specific algorithm is as follows: ； in, The feature space contained in the spatiotemporal fusion feature space matrix. and Representing the first , The center of the class sample, and Representing the first , The distance of the sample furthest from the center of the sample in the category. and Representing the sample centers respectively and The distance to the category boundary is calculated; advanced operating condition features with discriminative information are learned; these advanced operating condition features are input into a Softmax classifier for classification to produce the final operating condition identification result.

2. The excavator working condition identification method based on spatiotemporal fusion of whole machine information as described in claim 1, characterized in that, Step 1 includes the following steps: Step 11: Standardize the obtained raw multidimensional operational information using the maximum-minimum normalization method; The calculation method is as follows: ； in, The values ​​representing the original sample points. Represents the minimum value in the original data. Represents the maximum value in the original data; Step 12: Use the sliding window segmentation method to segment the standardized multidimensional operational information obtained in Step 1. The sliding window size is L and the step size is S, to obtain several multivariate time series input matrices, which constitute a dataset for operational condition identification.

3. The excavator operating condition identification method based on spatiotemporal fusion of whole machine information as described in claim 1, characterized in that, Step 2 includes the following steps: Step 21: Input the dataset obtained in Step 1 into the constructed temporal attention module and spatial attention module in parallel, and extract the temporal attention features and spatial attention features respectively; Step 22: Set the structures of the temporal feature extraction module and the spatial feature extraction module respectively. The temporal feature extraction module consists of M cascaded long short-term memory networks. The spatial feature extraction module consists of N residual convolutional networks. Each residual convolutional network consists of two convolutional layers, two batch normalization layers, and one ReLU activation layer. They are connected by branches from input to output to form a complete residual convolutional network. Connect the N residual convolutional networks together to form the spatial feature extraction module. Step 23: Input the temporal attention features and spatial attention features extracted in Step 21 into the temporal feature extraction module and the spatial feature extraction module respectively, and extract the temporal features and spatial features with multivariate working condition information respectively; Step 24: The temporal and spatial features with multivariable operating condition information obtained in Step 23 above are concatenated according to the variable axis to obtain a spatiotemporal fusion feature space matrix with multivariable operating condition information, which is used to extract advanced operating condition features with identification information in the subsequent process.

4. The excavator operating condition identification method based on spatiotemporal fusion of whole machine information as described in claim 1, characterized in that, Step 3 includes the following steps: Step 31: Input the spatiotemporal fusion feature space matrix obtained in step 24 above into the designed working condition feature learning module; Step 32: Define the excavator working condition classification task as a 5-class classification problem; Step 33: Input the advanced working condition information features obtained in Step 31 into a Softmax classifier with a cross-entropy loss function to classify the working condition modes of the excavator. The calculation formula for the cross-entropy loss is as follows: ； in, For sample label values, These are the model's predicted values; Step 34: The total loss function of the model is expressed as follows: ； in, The weight coefficients are used to control the feature learning effect; the results are used as the network output to obtain the final working condition recognition result.

5. A system for identifying excavator operating conditions based on spatiotemporal fusion of whole-machine information, implementing the excavator operating condition identification method based on spatiotemporal fusion of whole-machine information as described in any one of claims 1-4, characterized in that, The excavator operating condition identification system based on spatiotemporal fusion of whole machine information includes: The signal acquisition and processing module is used to acquire multi-dimensional operating information of the excavator, perform data processing and data labeling, obtain multivariate time series with standardized characteristics, and then create a time series input matrix for working condition classification. The spatiotemporal feature extraction module is used to input the time series matrix into the temporal attention module and the spatial attention module in parallel. Through the temporal feature extraction network composed of a long short-term memory network and the spatial feature extraction network composed of a residual convolutional network, the temporal features and spatial features with multivariate working condition information are obtained respectively, and these features are concatenated into a spatiotemporal fusion feature space matrix. The working condition identification module is used to input the spatiotemporal feature space matrix into the working condition information learning module for processing, learn high-level working condition features with identification information, and input these features into the Softmax classifier for classification to produce the final working condition identification result.

6. The excavator working condition identification system based on spatiotemporal fusion of whole machine information as described in claim 5, characterized in that, The signal acquisition and processing module further includes: The standardization processing unit is used to standardize the original multidimensional operational information using the maximum-minimum value normalization method; The data segmentation unit is used to segment the standardized multidimensional operational information using the sliding window segmentation method to obtain several multivariate time series input matrices, which constitute a dataset for operational condition identification.

7. The excavator working condition identification system based on spatiotemporal fusion of whole machine information as described in claim 5, characterized in that, The spatiotemporal feature extraction module further includes: The attention feature extraction unit is used to input the dataset in parallel into the temporal attention module and the spatial attention module, and extract temporal attention features and spatial attention features respectively; The feature extraction network unit is used to set the structure of the temporal feature extraction module and the spatial feature extraction module, respectively extracting temporal features and spatial features with multivariate working condition information, and concatenating these features into a spatiotemporal fusion feature space matrix.

8. The excavator working condition identification system based on spatiotemporal fusion of whole machine information as described in claim 5, characterized in that, The operating condition identification module further includes: The feature learning unit is used to input the spatiotemporal fusion feature space matrix into the working condition feature learning module, and defines the working condition feature learning loss function as an algorithm for learning advanced working condition information features. The classification and loss calculation unit is used to define the excavator working condition classification task. It inputs the learned high-level working condition information features into a Softmax classifier with a cross-entropy loss function for classification, and calculates the total loss function of the model to obtain the final working condition identification result.

9. A computer device, characterized in that, The computer device includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor performs the steps of the excavator working condition identification method based on spatiotemporal fusion of whole machine information as described in any one of claims 1-4.

10. A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the excavator condition identification method based on spatiotemporal fusion of whole-machine information as described in any one of claims 1-4.

Images