Excavator working mode identification method and device, electronic equipment and storage medium

Abstract

The application discloses a working mode recognition method of an excavator, and relates to the technical field of working mode recognition of equipment. The method comprises the following steps: acquiring joint state time series data of each joint of the excavator in a target time period, wherein the joint state time series data is used for indicating the motion state of each joint of the excavator in the target time period; acquiring a corresponding data change period according to the joint state time series data, and dividing the joint state time series data according to the data change period to obtain at least one joint state time series feature; inputting the joint state time series feature into a working mode recognition model, and determining the target working mode of the excavator through the working mode recognition model; acquiring working standard parameters corresponding to the target working mode, and adjusting the current working parameters of the excavator according to the working standard parameters. According to the method, various working modes can be accurately recognized according to the joint state time series data, the limitation of working mode recognition is reduced, and the applicability is improved.

Description

Technical Field

[0001] This application relates to the field of equipment working mode recognition technology, specifically to a method, device, electronic device and storage medium for recognizing the working mode of an excavator. Background Technology

[0002] In unmanned automated excavation technology, the excavator is first remotely controlled by a human, and then automatically excavates according to the manually guided working mode, thus freeing people from repetitive manual labor. Excavators have many working modes, such as digging holes, loading trucks, dumping excavated material, leveling ground, and leveling slopes. Different throttle gears and speeds for different joints are required for different working modes to achieve the desired work objectives. Therefore, accurately identifying the working mode is a crucial foundation for realizing this technology.

[0003] In existing technologies, the working mode recognition of excavators can be based on visual images, such as capturing images of the excavator in operation to identify the working mode. Alternatively, the working mode can be identified based on internal operating parameters of the excavator, such as current, voltage, and hydraulic pressure values.

[0004] Among the existing technologies mentioned above, visual image-based recognition methods rely heavily on cameras, requiring high precision in image acquisition, and are unsuitable for excavation scenarios with poor lighting and complex terrain. Methods that identify working modes based on internal operating parameters of the excavator, such as current, voltage, and hydraulic pressure, can only identify working modes that significantly affect these parameters, such as crushing and excavation, and cannot accurately identify other working modes. Therefore, the existing technologies for identifying working modes have significant limitations and a narrow scope of application, failing to meet the requirements of autonomous driving technology. Thus, improving the applicability of excavator working mode recognition technology has become crucial. Summary of the Invention

[0005] This application provides a method, device, electronic device, and storage medium for identifying the working mode of an excavator. By collecting the status time-series data of each joint of the excavator during remote control, and based on the periodic change pattern of the status time-series data of each joint, the working mode is accurately identified, thereby reducing the limitations of working mode identification technology and meeting the needs of technologies such as automatic excavation, parameter optimization, and service quality monitoring.

[0006] The first aspect of this application provides a method for recognizing the working mode of an excavator, the method comprising:

[0007] Acquire the joint status time-series data of each joint of the excavator within a target time period. The joint status time-series data is used to indicate the motion status of each joint of the excavator within the target time period.

[0008] The corresponding data change period is obtained from the joint state time series data, and the joint state time series data is divided according to the data change period to obtain at least one joint state time series feature.

[0009] The joint state temporal features are input into the working mode recognition model, and the target working mode corresponding to the excavator is determined by the working mode recognition model.

[0010] Obtain the standard working parameters corresponding to the target working mode, and adjust the current working parameters of the excavator according to the standard working parameters.

[0011] A second aspect of this application provides a working mode recognition device, the device comprising:

[0012] The acquisition unit is used to acquire the joint status timing data of each joint of the excavator within a target time period. The joint status timing data is used to indicate the motion status of each joint of the excavator within the target time period.

[0013] The segmentation unit is used to obtain the corresponding data change period based on the joint state time series data, and to segment the joint state time series data according to the data change period to obtain at least one joint state time series feature.

[0014] The determination unit is used to input the temporal features of the joint state into the working mode recognition model, and to determine the target working mode corresponding to the excavator through the working mode recognition model.

[0015] The adjustment unit is used to obtain the working standard parameters corresponding to the target working mode and adjust the current working parameters of the excavator according to the working standard parameters.

[0016] A third aspect of this application also provides an electronic device, including: a memory and a processor, coupled together. The memory stores one or more computer instructions. The processor executes one or more computer instructions to implement the method described above.

[0017] Furthermore, this application also provides a computer-readable storage medium storing one or more computer instructions that are executed by a processor to implement the above-described method.

[0018] The method provided in the above-described embodiments of this application first acquires the state time-series data of each joint of the excavator in working condition, and then divides the state time-series data periodically according to the corresponding data change cycle, thereby extracting the corresponding statistical features indicating the motion state of each joint. These statistical features are then input into a trained working mode recognition model to determine the target working mode. After determining the target working mode, the current working parameters are adjusted according to the corresponding standard working parameters, so that the excavator works according to the standard parameters of the target working mode, thereby achieving the purpose of automatic excavation or parameter optimization.

[0019] Based on the differences in the periodic changes of each joint under different working modes, this method only needs to collect joint state time series data, extract features based on the period of the joint state time series data itself, and perform working mode recognition through a working mode recognition model. It does not rely on cameras, can recognize a large number of working modes, and is suitable for harsh working environments, thereby reducing the limitations of working mode recognition and improving applicability. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the excavator working mode recognition method provided in the embodiments of this application;

[0021] Figure 2 This is a flowchart illustrating how the data change period is determined based on the data changes of each joint, as described in an embodiment of this application.

[0022] Figure 3 This is a flowchart illustrating how the data change period is determined based on the overall data changes of each joint, as described in an embodiment of this application.

[0023] Figure 4 This is a training method for the working mode recognition model involved in the embodiments of this application;

[0024] Figure 5 This is a schematic diagram of the working mode recognition device provided in the embodiments of this application;

[0025] Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0026] To enable those skilled in the art to better understand the technical solutions of this application, the application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. However, this application can be implemented in many other ways different from those described above. Therefore, based on the embodiments provided in this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0027] It should be noted that the terms "first," "second," "third," etc., in the claims, specification, and drawings of this application are used to distinguish similar objects and are not used to describe a specific order or sequence. Such data are interchangeable where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that shown or described herein. Furthermore, the terms "comprising," "having," and their variations are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatuses.

[0028] Traditional excavator operation involves manual control to perform excavation tasks in different modes. These modes require repetitive mechanical labor. For example, when digging a pit, the operator controls the bucket's joints to rotate and perform the digging motion, while also controlling the boom, arm, and cab joints to assist bucket movement. Finally, the operator controls the bucket's rotation to dump the soil. This process is cyclical and repetitive, continuously executing the series of actions: digging, moving the bucket, and dumping. Automated excavation, on the other hand, allows for the automatic repetition of these mechanical actions after manual control, until the working mode is changed, thus avoiding repetitive manual labor. Therefore, accurately identifying the working mode after manual control is a crucial foundation for realizing this technology.

[0029] In existing technologies, one approach is excavator working mode recognition based on visual images. Specifically, while the excavator is working, a third-person perspective camera captures images of the work, and then the state of the excavator in the images is identified. However, this method relies excessively on images, requires cameras to be set up at specific locations in the excavation environment, and demands high image accuracy, making it unsuitable for complex environments and low-light excavation scenarios.

[0030] Another method involves identifying the operating mode based on the excavator's internal operating parameters, such as current, voltage, and hydraulic pressure. These internal parameters directly affect the control effect. Specifically, by monitoring the excavator's internal parameters, such as current, voltage, and hydraulic pressure, the method can identify the operating mode based on the significant differences in parameters when the excavator uses different types of working heads, such as using a breaker head for crushing and using a bucket head for digging. However, this method cannot accurately identify the operating mode for different operating modes of the same working head, such as using a bucket head for digging and loading, as the changes in internal parameters such as current, voltage, and hydraulic pressure are not significant.

[0031] Both of the aforementioned existing technologies have significant limitations and poor applicability in harsh working environments, failing to meet the needs of automatic excavation, parameter optimization, or quality monitoring in such conditions. Correspondingly, this application provides a method for identifying the working mode of an excavator. Based on the dynamic laws governing the different periodic changes of each joint in different working modes, a working mode identification model accurately identifies various working modes, thereby automatically adjusting working parameters to meet the needs of automatic excavation, parameter optimization, or quality monitoring in harsh environments.

[0032] The method is as follows: Figure 1 As shown, Figure 1 This is a flowchart illustrating the excavator working mode recognition method provided in an embodiment of this application. It should be noted that the steps shown in this flowchart can be executed in a computer system such as a set of computer-executable instructions, and in some cases, the steps shown may be executed in a different logical order than that shown in the flowchart.

[0033] like Figure 1 As shown, the method includes steps S101-S104, as detailed below:

[0034] S101. Obtain the joint status time sequence data of each joint of the excavator within the target time period.

[0035] First, when the excavator is in manual operation mode, the time-series data of the joint status of each joint of the excavator is acquired within a target time period. This joint status time-series data is used to indicate the movement status of each joint of the excavator within the target time period.

[0036] Specifically, upon receiving operational instructions from the operator for each joint of the excavator, the excavator responds and begins digging. Simultaneously, this step begins, data acquisition. The joint state time-series data is a collection of joint state data from consecutive moments across all joints. This data primarily includes joint parameter data at various moments, specifically various dynamic data such as angles, velocities, and torques, without limitation. In other words, joint state data includes joint parameter data characterizing the motion state acquired at the time of data acquisition. The collection of joint state data from consecutive moments constitutes the joint state time-series data, and the joint parameter data within this data exhibits a contextual relationship across consecutive moments.

[0037] Furthermore, data acquisition can be performed using various sensors, followed by data transmission via wired or wireless communication, without limitation. The acquisition frequency of each sensor can be adjusted according to different needs. For example, high-frequency acquisition at the millisecond level can be used to meet the identification requirements of short-cycle operating modes, while low-frequency acquisition can be used to meet the identification requirements of longer-cycle operating modes. Generally, the acquisition frequency is a preset fixed frequency, but it can also be a floating frequency that automatically changes or corrects, without limitation.

[0038] In addition, it should be understood that the working mode recognition method provided in this application embodiment needs to comprehensively consider the movement of all joints at the same time. When obtaining the joint state time sequence data of each joint, the data acquisition cycle should be kept consistent, that is, the sensor should keep the acquisition frequency consistent, and the joint parameter data of all joints should be acquired at the same data acquisition time.

[0039] Furthermore, when acquiring joint parameter data, multiple types of initial data can be acquired simultaneously. The data types can then be filtered according to requirements to obtain the necessary joint state time-series data. For example, when the sensor collects data, it can acquire three joint parameter data—angle, velocity, and torque—at each acquisition moment. Then, when determining the target state time-series data, angle and velocity joint parameter data are selected as the target state time-series data based on requirements.

[0040] When acquiring joint state time-series data, it's also necessary to determine the corresponding time period, i.e., to identify the target time period, in order to obtain the data that best represents the current working mode. Specifically, during the excavation process, there may be situations where operators temporarily adjust the working mode. For example, an operator might start by digging in manual mode but then switch to loading shortly afterward. Therefore, for working mode recognition, the joint parameter data collected during the digging time period becomes erroneous data that affects the recognition results and needs to be removed. In this case, it's only necessary to set the target time period as the loading process time period, and then determine the joint state time-series data based on the target time period.

[0041] In addition, to ensure that the joint state timing data is sufficient to determine the working mode, acquisition conditions for the joint state timing data can be set. These conditions can be the minimum value of the change cycle corresponding to the joint state timing data. For example, let S represent the number of cycles; the cycle condition can be S≥2, meaning the working mode corresponding to the joint state timing data must have executed at least two complete rounds. For instance, if the joint state timing data corresponds to the digging working mode, then the joint state timing data must at least include joint state data for at least two complete rounds of digging action.

[0042] Alternatively, the acquisition condition can be a quantity, meaning the joint status time-series data must include at least a preset number of joint status data points. For example, if the quantity threshold is set to 100 sets of data, then the joint status time-series data must include joint status data corresponding to at least 100 consecutive acquisition times. Alternatively, the acquisition condition can also be a time period set according to the acquisition frequency. For example, if the time period is set to 1 minute, then when the excavator is working, joint status data spanning a 1-minute period will be collected.

[0043] In addition, the collected data may have defects, such as insufficient data collection at certain times or obvious errors in some data. In such cases, data preprocessing can be performed to ensure that erroneous data does not affect the judgment of working modes. For example, noise filtering, missing value handling, and normalization can be used to obtain more accurate joint state time series data.

[0044] Once the joint state timing data is acquired, it can be periodically divided, as shown in the following step.

[0045] S102. Obtain the corresponding data change period based on the joint state time series data, and divide the joint state time series data according to the data change period to obtain at least one joint state time series feature.

[0046] After acquiring joint state time-series data, the data change period can be obtained by analyzing the data changes at each moment in the joint state time-series data. Then, the joint state time-series data is divided according to the data change period, and the corresponding statistical characteristics, i.e., joint state time-series features, are extracted from the results of the periodic division. The data change period, obtained from the joint state time-series data, reflects the changing pattern of the joint state time-series data. The joint state time-series features are the feature data obtained after further processing the periodic division of the joint state time-series data; they represent the statistical characteristics of the changes in each joint.

[0047] Specifically, joint state time-series data is a dataset of joint state data including at least one data type. Joint state time-series features are statistical feature data segments with context extracted after periodically dividing this dataset. The context of the joint state time-series features consists of segmented data features from consecutive periods.

[0048] Specifically, the process of obtaining the temporal features of joint states can be as follows: The temporal data of joint states is divided according to the data change period corresponding to the joint state temporal data to obtain N consecutive state data segments. The N segment data features corresponding to the N consecutive state data segments are concatenated to obtain the joint state data, where N is a positive integer greater than 1.

[0049] In other words, based on the data change cycles of each joint reflected in the joint state time-series data, the joint state time-series data is divided into multiple sets of data in continuous time segments. Feature extraction is performed on the data within each data set to obtain statistical features corresponding to each period segment, i.e., segment data features. After concatenating the segment data features, the joint state time-series features corresponding to the joint state time-series data are obtained, which are used to characterize the continuous changes in the motion state of each joint during the target time period.

[0050] It's important to understand that the extracted features are specific data with representative meaning, such as maximum, minimum, mean, and variance. In other words, segment data features are descriptive statements of the feature values ​​within each period segment, corresponding to a single period. Joint state time-series features, on the other hand, are descriptive statements of the concatenated features from consecutive periods, corresponding to the entire process of the joint state time-series data.

[0051] After dividing the joint state time series data, the obtained joint state time series features can be one or multiple. The joint state time series features can be composed of a single type of statistical feature or multiple types of statistical features, without any restriction.

[0052] For example, if the time series data of a certain joint state corresponds to n periods of data change, when determining the time series features of the joint state, the time series data of the joint state is first divided into n state data segments. a types of segment data features are extracted from the data segments of each period. These segment data features are then concatenated according to their categories. The modulus value of each joint state data segment in each period is extracted as the statistical feature of all joints, resulting in a vector of size [5, a × (n+1)] as the time series feature of the joint state. Here, 'a' represents the number of statistical feature categories in each period, (n+1) represents the statistical feature of n periods and one overall feature, and 5 represents the feature with five dimensions: features of four joints and a modulus value used to statistically analyze the data of all joints.

[0053] When multiple joint state time-series features are acquired, the meaning of each feature varies depending on the acquisition method. For example, different joint state time-series features can be obtained based on different types of data. In this case, multiple joint state time-series features still represent the changes in the working mode corresponding to the entire stage of the joint state time-series data. For instance, if two joint state time-series features are obtained—one related to angle and the other to torque—the changes in these two features can be considered simultaneously during working mode recognition. Alternatively, multiple joint state time-series data can be used to reflect the changes in the working mode at different stages. For example, after dividing the period by peak and valley values, the joint state time-series data is divided into six consecutive period segments. After extracting the segment data features corresponding to each of these six periods, the segment data features corresponding to every two periods are concatenated in front-to-back order, resulting in three joint state time-series data, corresponding to the early, middle, and late stages of the joint state time-series data, respectively.

[0054] After obtaining the joint status data, the joint status data can be input into the working mode recognition model to perform working mode recognition, as shown in the following steps.

[0055] S103. Input the joint state time sequence features into the working mode recognition model, and determine the target working mode corresponding to the excavator through the working mode recognition model.

[0056] After obtaining the temporal features of the joint states, the input sample for the working pattern recognition model is obtained. By inputting this sample into the working pattern recognition model, the corresponding working pattern can be determined based on the model's output.

[0057] The working pattern recognition model can be a convolutional neural network (CNN) model, a long short-term memory neural network (LSTM) model, or other similar models; there are no specific limitations. After inputting joint state data into the working pattern recognition model, the model will match the working patterns based on the input samples, thereby outputting a result indicating the target working pattern.

[0058] The output of the working mode recognition model can be of various types, as long as it can determine the target working mode. The specific form of the output is not limited here. Specifically, the output can be a determination of the working mode, such as: the target working mode is digging a pit. Alternatively, it can output the matching degree between joint state data and the working mode, thereby further determining the target working mode. For example, if the output shows: the matching degree between the working mode and the digging working mode is 80%, and the matching degree with the loading working mode is 20%, then the target working mode can be further determined to be digging a pit.

[0059] Once the target working mode is determined, the excavator can enter the automatic digging mode, as shown in the following steps.

[0060] S104. Obtain the working standard parameters corresponding to the target working mode, and adjust the current working parameters of the excavator according to the working standard parameters.

[0061] Once the target working mode is determined, automatic excavation or optimization of excavator working parameters can be performed based on this mode. This is achieved by adjusting the current working parameters according to the standard working parameters of the target working mode. These working parameters control the excavator's operation and include all control-related parameters such as the excavator's internal parameters and joint parameters. The standard working parameters are the optimal working parameters set for the corresponding working mode.

[0062] For example, the target working mode is determined to be the flat ground mode. The working standard parameters corresponding to the flat ground mode are obtained, including the optimal voltage, optimal hydraulic value, and optimal joint parameters of each joint. The current parameters are then adjusted to achieve the optimal working mode and optimize the excavator's working parameters.

[0063] In addition, automatic excavation is achieved by automatically adjusting the joint parameters of each joint of the excavator. Specifically, when entering automatic excavation mode, the standard joint parameters corresponding to the target working mode are acquired, and then the parameters of each joint are adjusted according to the parameter change process of the standard joint parameters to achieve automatic excavation according to the target working mode.

[0064] The standard working parameters are pre-set parameters corresponding to each working mode, enabling automatic data mining. These standard working parameters can be changed through manual input, automatic collection, or other methods, without restriction. Furthermore, these standard working parameters can be fixed standard parameters, or functional relationships or mapping relationships between multiple parameters, without restriction.

[0065] When using this standard working parameter, you can directly adjust the current working parameter to be consistent with the fixed standard parameter of the standard working parameter, or you can continuously adjust the current working parameter according to the functional relationship of the standard working parameter. There are no specific restrictions on this here.

[0066] Alternatively, after determining the working mode, the working parameters can be adjusted directly, meaning that automatic digging or parameter optimization can be performed without responding to manual operation commands, until the working mode is manually turned off or switched. Another option is to provide feedback to the operator after determining the working mode, and then adjust the working parameters after receiving a command from the user to enter automatic digging mode. This method specifically involves: providing feedback on the identification result for the target working mode, responding to user operations based on the identification result, and adjusting the excavator's current working parameters according to the standard working parameters.

[0067] The above describes the working mode recognition method involved in this application. This method extracts the joint state temporal features that can characterize the periodic changes of the joints by analyzing the periodic change patterns of the motion parameters of each joint of the excavator under various working modes. Combined with the powerful computing capabilities of the working mode recognition model, it accurately identifies the working mode of the excavator under manual control mode. Then, based on the determined target working mode, it adjusts the parameters according to the working standard parameters to achieve the purpose of automatic excavation.

[0068] Alternatively, after identifying the target operating mode, parameters such as throttle, gear, and voltage can be adjusted accordingly to achieve parameter optimization. Or, the target operating mode can be determined for service quality monitoring. Specific application scenarios and methods can be changed according to needs and are not limited here.

[0069] In the above method, the process of dividing the joint state time-series data according to the data change period corresponding to each joint is particularly important, as it affects whether the acquired joint state data can accurately reflect the changes of each joint within the target time period. There are many methods for dividing the data change period, and a reasonable period division method will directly affect the working pattern recognition results.

[0070] Since the joint state time series data is a data set of joint parameters at continuous moments of all joints, the joint state time series data includes multiple sets of joint state time series sub-data corresponding to multiple joints. The joint state time series sub-data is the data set of joint state data at continuous moments of each joint.

[0071] Correspondingly, this embodiment provides two methods for determining the data change period based on the timing sub-data of the joint state corresponding to each joint.

[0072] Method 1: Determine the data change cycle based on the data changes of each joint.

[0073] like Figure 2 As shown, Figure 2 The flowchart illustrates the process of determining the data change cycle based on the data changes of each joint. The method includes steps S201-S203.

[0074] S201. Based on the time sequence data of each joint state, determine the angle position change curve corresponding to each joint.

[0075] First, based on the state time sequence data of each joint, the data representing the angle position at each moment within the target time period can be obtained. Thus, the state time sequence data can be transformed into an angle position change curve with continuous moments as the horizontal axis and angle position data as the vertical axis.

[0076] S202. Determine the curve change period based on the angle position change curve corresponding to each joint.

[0077] Because the working mode of an excavator is a periodic repetitive action, the angle position change curve will also show a periodic change pattern. Furthermore, due to the reciprocating motion of each joint, the angle position change curve corresponding to each joint will have its own peak and trough values. Based on its peak and trough values, the corresponding curve change cycle can be determined.

[0078] S203. Determine the data change period corresponding to the joint state timing data based on the curve change period.

[0079] The corresponding data change period is determined by analyzing the curve's cycle. There are many methods for this; for example, the curve's cycle can be directly set as the data change period, or a multiple of the curve's cycle can be used—there are no restrictions here. For instance, if the curve's cycle is determined based on its peaks and troughs, and the time to complete one cycle is T, then the data change period can be T. Alternatively, the data change period can be twice the curve's cycle, meaning the time to complete one cycle is 2T. It's important to understand that when dividing the cycle based on the curve's peaks and troughs, or rather, based on the data's changing characteristics, different rates of curve change do not affect the determination of the cycle.

[0080] Furthermore, when determining the curve change period based on the angle position change curve corresponding to each joint, there are cases where the data for some joints remains stable. In these cases, the stable data for these joints has little impact on the identification of the working mode. Therefore, it is only necessary to identify the joints that exhibit changes. Additionally, the period can also be determined for individual joints with clearly defined characteristics.

[0081] This method for determining the curve change period specifically involves: determining the target angle change curve based on the angle position change curve corresponding to each joint, where the target angle change curve is a periodic change curve; determining the target curve change period corresponding to the target angle change curve; and determining the data change period corresponding to the joint state timing data based on the target curve change period.

[0082] For example, when an excavator is in a flat-ground working mode, its cab angle position remains basically unchanged, but other joints undergo periodic changes. In the angle position change curves corresponding to each joint, the cab joint angle change curve is a straight line that changes with a straight line, while the angle change curves corresponding to the boom, arm, and bucket joints are periodic curves. In this case, the target angle change curve can be determined as one or more of the angle change curves corresponding to the boom, arm, and bucket joints.

[0083] Method 2: Determine the data change cycle based on the overall data changes of each joint.

[0084] like Figure 3 As shown, Figure 3 The flowchart illustrates the process of determining the data change cycle based on the overall data changes of each joint. The method includes steps S301-S303.

[0085] S301. Based on the joint state time sequence sub-data corresponding to each joint, determine the angle position of each joint at each target time within the target time period.

[0086] Based on the joint state time sequence sub-data corresponding to each joint, the data representing the angular position at each moment within the target time period is obtained, that is, the angular position of each joint at each target moment within the target time period is determined. The determined angular position is represented by the corresponding angle value and used as reference data for subsequent calculations. Furthermore, each joint can use the same or different methods to determine the angle value. For example, the angle value can be obtained through angular velocity and time, or the angle data can be directly used as the angle value; there is no restriction here.

[0087] S302. Determine the multidimensional vector corresponding to each target time based on the angular position of each joint at each target time.

[0088] Once the angular position of each joint at each target time point within the target time period is determined, a multi-dimensional vector is generated for each target time point based on the angular position of each joint at each target time point. In other words, a multi-dimensional vector is used to represent the comprehensive information of the angular positions of all joints at each time point.

[0089] In this step, the temporal data of the joint states corresponding to each joint were aggregated, which means that the temporal data of each joint state were processed into a data set that can represent the information of all joints. This allows for a comprehensive consideration of all joints when determining the data change period.

[0090] S303. Determine the data change period corresponding to the joint state time sequence data based on the multi-dimensional vector corresponding to each target time.

[0091] Based on the multidimensional vectors determined in the previous step, a set of multidimensional vectors for consecutive moments within the target time period is obtained. Since each joint of the excavator undergoes consistent repetitive motion when in working mode, the periodic variation pattern of each joint is consistent, meaning all joints have the same variation period. Correspondingly, the variation pattern of the multidimensional vectors corresponding to each target moment within the target time period also exhibits a periodic variation pattern. By corresponding to this pattern, the data variation period can be determined based on the multidimensional vectors corresponding to each target moment.

[0092] When determining the data change period based on the multidimensional vector corresponding to each target time, the change curve method can also be used to directly determine the data change period. Specifically: calculate the vector magnitude of the multidimensional vector corresponding to each target time, determine the magnitude change curve based on the vector magnitude of the multidimensional vector corresponding to each target time, and determine the data change period corresponding to the joint state time series data based on the magnitude change period corresponding to the magnitude change curve.

[0093] The module length variation curve is plotted with the target time on the x-axis and the calculated module length related to all joint parameters on the y-axis. When determining the data variation period corresponding to the joint state timing data based on the module length variation period corresponding to the module length variation curve, the module length variation period corresponding to the module length variation curve can be directly set as the data variation period, or a multiple of the module length variation period corresponding to the module length variation curve can be used as the data variation period. This will not be elaborated further here.

[0094] It is important to understand that the data change cycle is determined based on the state time series data. In addition to the two methods mentioned above, other different methods for determining the data change cycle can be set according to changes in requirements, and no restrictions are imposed here.

[0095] Furthermore, the target working pattern is determined using a working pattern recognition model. This model is pre-trained and used during the working pattern recognition process. The training process of this working pattern recognition model is briefly described below.

[0096] like Figure 4 As shown, Figure 4The training method for the working mode recognition model involved in the embodiments of this application includes steps S401-S403.

[0097] S401. Obtain training samples. The training samples are the temporal feature samples of the joint states of each joint of the excavator collected in the standard working mode.

[0098] First, training samples are obtained. Training samples can be obtained by collecting joint parameters of each joint under ideal conditions in the standard working mode, obtaining joint state time series data in the standard working mode, i.e. joint state time series acquisition data, and then extracting the corresponding joint state time series features as training samples.

[0099] For example, for multiple working modes such as "digging holes", "loading trucks", "shoveling", "leveling ground", and "leveling slopes", operators are instructed to manually operate the excavator according to the operating methods of each working mode, so that each working mode is executed for 100 cycles, and the sensors of each joint collect data at a sampling frequency of 100 Hz, collecting the joint state time sequence data in each working mode, and then extracting the corresponding joint state time sequence features. Each joint state time sequence feature is a joint state time sequence feature sample, and each joint state time sequence feature sample is assigned a corresponding working mode label.

[0100] In addition, data acquisition and sample extraction can be completed with different orientations, different joint movement amplitudes, and different joint movement rates for different working modes, so as to make the training samples more diverse, without any restrictions.

[0101] S402. Input the training samples into the working mode recognition model, and obtain the output working mode corresponding to the training samples through the working mode recognition model.

[0102] After the training samples are input into the working pattern recognition model, the model will output a result indicating the target working pattern based on the training samples, and determine the output working pattern corresponding to the training samples based on this result.

[0103] S403. Determine the result loss between the standard working mode and the output working mode.

[0104] Once the output working pattern is obtained, the difference between the standard working pattern and the output working pattern is calculated, i.e., the result loss between the standard working pattern and the output working pattern is determined. Specifically, the standard working pattern can be determined by the working pattern labels carried in the training samples. The result loss can be calculated using a loss function, and the loss function can be the cross-entropy loss function commonly used in classification problems, or other applicable loss functions; no specific restrictions are placed here.

[0105] S404. Adjust the model parameters of the working mode recognition model based on the result loss.

[0106] After obtaining the result loss, the model parameters in the working pattern recognition model are adjusted according to the result loss to continuously reduce the result loss until the correct working pattern can be determined.

[0107] In addition, the training samples should be consistent with the joint state data type input to the work mode recognition model when recognizing the work mode. For example, if the joint state temporal feature type of the training samples is mean, the joint state temporal feature type used for work mode recognition should also be mean, so as to ensure that the work mode recognition model can accurately output the recognition results.

[0108] Corresponding to the method for obtaining joint state temporal features, the method for obtaining joint state temporal feature samples is as follows: Collect joint state temporal acquisition data for each joint of the excavator within a preset acquisition time under standard working mode. Divide the joint state temporal acquisition data into m consecutive joint state temporal acquisition sub-data. Concatenate the segment acquisition features corresponding to the m consecutive joint state temporal acquisition sub-data to obtain the joint state temporal feature samples, where m is a positive integer greater than 1.

[0109] Of course, the training samples can also be a set of collected joint parameters, that is, a set of time-series data of joint states of each joint within a preset collection time under standard working mode. These data are then processed into joint state time-series feature samples by the working mode recognition model. No specific restrictions are imposed here.

[0110] In addition, to ensure a sufficiently rich sample of joint state temporal features, data augmentation can be performed. Specifically, since the joint state temporal acquisition data corresponding to the training samples also exhibits a periodic variation pattern, the joint state temporal acquisition data can be augmented according to the periodic variation pattern of the two-dimensional curve, and then features are extracted to obtain augmented joint state temporal feature samples.

[0111] During implementation, the decisive role of the cockpit in the working position can be taken into account. The joint state time series data can be augmented with respect to the cockpit data by three methods: rotation, reversal, and amplification. Then, the corresponding joint state time series feature samples can be extracted.

[0112] Specifically, the augmentation method of rotation mainly takes into account the range of changes in the cockpit angle position. Therefore, the joint state time sequence data can be translated as a whole curve, indicating that the cockpit performs work according to the data in the data sample after rotating a certain angle. For example, it can be translated by 2π / 10. The corresponding extracted joint state time sequence feature sample indicates the data change when the cockpit rotates by 2π / 10 to perform work.

[0113] The reverse augmentation method is mainly based on the consideration of the two working modes of "loading" and "scraping". In these two working modes, the cockpit will rotate periodically. For example, the joint state timing data collection means that after digging in front, the truck is loaded on the right side. If we want to represent that after digging in front, the truck is loaded on the left side, we can reverse the cockpit data of this process. This is equivalent to adding joint state timing data collection in the symmetrical direction of rotation. The corresponding extracted joint state timing feature samples represent the changes in the work after rotation.

[0114] The amplification method mainly involves amplifying the data on the cockpit's rotation amplitude. For example, in the "loading" and "shoveling" working modes, the sample data represents digging soil directly in front and dumping soil at a 90° position on the right. To expand the data, multiplying the overall amplitude of the cockpit data by 2 can represent digging soil directly in front and dumping soil at a position directly behind, etc. Similarly, the amplitude can be reduced to obtain sample data representing dumping soil at different angles, which is not limited here.

[0115] Alternatively, other data augmentation methods can be used to obtain more training samples. The idea is the same as the three augmentation methods mentioned above, and they will not be listed here.

[0116] Once the model has been trained sufficiently to correctly identify the working patterns, the training process can be terminated. Specifically, training of the working pattern recognition model ends when the training conditions are met. The training conditions are reaching a preset number of training iterations or the result loss between the standard working pattern and the output working pattern being less than a preset loss threshold.

[0117] Furthermore, specific test samples can be set up to test the working pattern recognition model. The process of obtaining test samples is similar to that of obtaining training samples, and the testing method can also be compared with the training method, so it will not be elaborated on here.

[0118] In addition, when using the work pattern recognition model to identify work patterns, the successfully identified joint state data can be used as new training samples to continuously update the training sample library and conduct iterative training, so that the work pattern recognition model can more accurately identify work patterns for various work scenarios.

[0119] The above provides a detailed description of the excavator working mode recognition method provided in the embodiments of this application. The following, in conjunction with... Figure 5The working mode recognition device is introduced, among which, Figure 5 A schematic diagram of the working mode recognition device provided by the present invention is shown below. Figure 5 As shown, this device corresponds to any feasible working mode recognition method in the above embodiments, specifically including:

[0120] Acquisition unit 501 is used to acquire the joint status timing data of each joint of the excavator within a target time period. The joint status timing data is used to indicate the motion status of each joint of the excavator within the target time period.

[0121] The segmentation unit 502 is used to obtain the corresponding data change period based on the joint state time series data, and to segment the joint state time series data according to the data change period to obtain at least one joint state time series feature.

[0122] The determining unit 503 is used to input the joint state temporal features into the working mode recognition model, and determine the target working mode corresponding to the excavator through the working mode recognition model.

[0123] The adjustment unit 504 is used to obtain the working standard parameters corresponding to the target working mode and adjust the current working parameters of the excavator according to the working standard parameters.

[0124] In one feasible implementation, the partitioning unit 502 is further configured to partition the joint state time series data according to the data change period corresponding to the joint state time series data to obtain N consecutive state data segments.

[0125] The joint state temporal features are obtained by concatenating the N segment data features corresponding to N consecutive state data segments. N is a positive integer greater than 1.

[0126] The determining unit 503 is also used to determine the angle position change curve corresponding to each joint based on the timing sub-data of each group of joint states.

[0127] The curve change period is determined based on the angle position change curve corresponding to each joint.

[0128] The data change period corresponding to the joint state time sequence data is determined based on the curve change period.

[0129] Specifically, it is used to determine the target angle change curve based on the angle position change curve corresponding to each joint. The target angle change curve is a periodic change curve.

[0130] Determine the target curve change period corresponding to the target angle change curve.

[0131] The data change period corresponding to the joint state time series data is determined based on the curve change period, including:

[0132] The data change period corresponding to the joint state time sequence data is determined based on the change period of the target curve.

[0133] In one feasible implementation, the determining unit 503 is further configured to determine the angular position of each joint at each target time within the target time period based on the joint state timing sub-data corresponding to each joint.

[0134] Based on the angular position of each joint at each target time, determine the multidimensional vector corresponding to each target time.

[0135] The data change period corresponding to the joint state time series data is determined based on the multi-dimensional vector corresponding to each target time.

[0136] Specifically, it is used to calculate the vector magnitude of the multidimensional vector corresponding to each target time.

[0137] The magnitude change curve is determined based on the vector magnitude of the multidimensional vector corresponding to each target time.

[0138] The data change period corresponding to the joint state timing data is determined based on the modulus change period corresponding to the modulus change curve.

[0139] In one possible implementation, the determining unit 503 is also used to provide feedback on the identification results for the target operating mode.

[0140] The adjustment unit 504 is used to adjust the current working parameters of the excavator according to the working standard parameters in response to user operations on the recognition results.

[0141] In one feasible implementation, a training unit 505 is also included, which is used to acquire training samples, which are time-series feature samples of joint states collected by each joint of the excavator in the standard working mode.

[0142] The training samples are input into the working mode recognition model, and the output working mode corresponding to the training samples is obtained through the working mode recognition model.

[0143] Determine the result loss between the standard working mode and the output working mode.

[0144] Adjust the model parameters of the working pattern recognition model based on the result loss.

[0145] Training unit 505 is also used for,

[0146] Collect time-series data of the joint status of each joint of the excavator within a preset collection time under standard working mode.

[0147] The joint state time-series acquisition data is divided into m consecutive joint state time-series acquisition sub-data.

[0148] The segment features corresponding to m consecutive temporal acquisition sub-data of joint states are concatenated to obtain the temporal feature sample of joint states. m is a positive integer greater than 1.

[0149] Training unit 505 is also used for,

[0150] Training of the work pattern recognition model ends when the training conditions are met.

[0151] The training conditions are to reach a preset number of training iterations or for the result loss between the standard working mode and the output working mode to be less than a preset loss threshold.

[0152] This application also provides an electronic device, such as... Figure 6 The diagram shows the structure of the electronic device, which includes a processor 601 and a memory 600. The memory 600 stores computer-executable instructions that can be executed by the processor 601. The processor 601 executes the computer-executable instructions to implement the above-described method.

[0153] exist Figure 6 In the illustrated embodiment, the electronic device further includes a bus 602 and a communication interface 603, wherein the processor 601, the communication interface 603, and the memory 600 are connected via the bus 602.

[0154] The memory 600 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 603 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network. The bus 602 may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. The bus 102 can be divided into an address bus, a data bus, and a control bus. For ease of representation, Figure 6 The symbol is represented by a single rectangular bar, but this does not mean that there is only one bus or one type of bus.

[0155] Processor 601 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 601 or by software instructions. Processor 601 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc. It can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in the memory, and the processor 601 reads the information in the memory and, in conjunction with its hardware, completes the steps of the method in the aforementioned embodiment.

[0156] This application also provides a computer-readable storage medium, which includes computer instructions. When executed by a processor, the computer instructions are used to implement any feasible technical solution in this application.

[0157] It should be noted that the information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0158] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope defined in the claims of the present invention.

Claims

1. A method for recognizing the working mode of an excavator, characterized in that, The identification method includes: The excavator acquires joint state time-series data of each joint within a target time period using sensors; the joint state time-series data is used to indicate the motion state of each joint within the target time period; the joint state time-series data includes a set of joint parameter data characterizing the motion state at consecutive time points; the joint parameter data has a contextual relationship at consecutive time points. The corresponding data change period is obtained based on the joint state time series data, and the joint state time series data is divided according to the data change period to obtain at least one joint state time series feature. The joint state temporal features are input into the working mode recognition model, and the target working mode corresponding to the excavator is determined by the working mode recognition model. Obtain the standard working parameters corresponding to the target working mode, and adjust the current working parameters of the excavator according to the standard working parameters.

2. The identification method according to claim 1, characterized in that, The step of dividing the joint state time-series data according to the data change period to obtain at least one joint state time-series feature includes: The joint state time series data is divided according to the data change period corresponding to the joint state time series data to obtain N continuous state data segments; The joint state temporal features are obtained by concatenating the N segment data features corresponding to the N consecutive state data segments; where N is a positive integer greater than 1.

3. The identification method according to claim 1, characterized in that, The joint state timing data includes multiple sets of joint state timing sub-data corresponding to multiple joints; The step of obtaining the corresponding data change period based on the joint state time series data includes: Based on the time sequence sub-data of each joint state, determine the angle position change curve corresponding to each joint; The curve change period is determined based on the angle position change curve corresponding to each joint. The data change period corresponding to the joint state timing data is determined based on the curve change period.

4. The identification method according to claim 3, characterized in that, The step of determining the curve change period based on the angle position change curve corresponding to each joint includes: The target angle change curve is determined based on the angle position change curve corresponding to each joint; the target angle change curve is a periodic change curve. Determine the target curve change period corresponding to the target angle change curve; The step of determining the data change period corresponding to the joint state timing data based on the curve change period includes: The data change period corresponding to the joint state timing data is determined based on the change period of the target curve.

5. The identification method according to claim 1, characterized in that, The joint state timing data includes multiple sets of joint state timing sub-data corresponding to multiple joints; The step of obtaining the corresponding data change period based on the joint state time series data includes: Based on the joint state time sequence sub-data corresponding to each joint, determine the angular position of each joint at each target time within the target time period; Based on the angular position of each joint at each target time, determine the multidimensional vector corresponding to each target time. The data change period corresponding to the joint state time sequence data is determined based on the multidimensional vector corresponding to each target time.

6. The identification method according to claim 5, characterized in that, The step of determining the data change period corresponding to the joint state time series data based on the multidimensional vector corresponding to each target time includes: Calculate the vector magnitude of the multidimensional vector corresponding to each target time. The magnitude change curve is determined based on the vector magnitude of the multidimensional vector corresponding to each target time. The data change period corresponding to the joint state timing data is determined based on the modulus change period corresponding to the modulus change curve.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: Feedback on the identification results for the target working mode; The adjustment of the excavator's current operating parameters according to the operating standard parameters includes: In response to user actions regarding the identification results, the current operating parameters of the excavator are adjusted according to the operating standard parameters.

8. The identification method according to claim 7, characterized in that, The method further includes: Acquire training samples, which are time-series feature samples of joint states collected by each joint of the excavator in standard working mode; The training samples are input into the working mode recognition model, and the output working mode corresponding to the training samples is obtained through the working mode recognition model. Determine the result loss between the standard operating mode and the output operating mode; Adjust the model parameters of the working mode recognition model based on the resulting loss.

9. The identification method according to claim 8, characterized in that, The method further includes: Collect time-series data of the joint status of each joint of the excavator within a preset collection time under the standard working mode; The joint state time-series acquisition data is divided into m consecutive joint state time-series acquisition sub-data. The segment acquisition features corresponding to m consecutive joint state temporal acquisition sub-data are concatenated to obtain the joint state temporal feature sample; where m is a positive integer greater than 1.

10. The identification method according to any one of claims 8 to 9, characterized in that, The method further includes: When the training conditions are met, the training of the working pattern recognition model is terminated. The training conditions are that a preset number of training sessions are reached or the result loss between the standard working mode and the output working mode is less than a preset loss threshold.

11. A working mode recognition device, characterized in that, The identification device includes: The acquisition unit is used to acquire joint state time-series data of each joint of the excavator within a target time period through sensors; the joint state time-series data is used to indicate the motion state of each joint of the excavator within the target time period; the joint state time-series data includes a set of joint parameter data characterizing the motion state at consecutive time points; the joint parameter data has a contextual relationship at consecutive time points; The segmentation unit is used to obtain the corresponding data change period based on the joint state time series data, and to segment the joint state time series data according to the data change period to obtain at least one joint state time series feature. The determining unit is used to input the joint state temporal features into the working mode recognition model, and determine the target working mode corresponding to the excavator through the working mode recognition model; The adjustment unit is used to obtain the working standard parameters corresponding to the target working mode, and adjust the current working parameters of the excavator according to the working standard parameters.

12. An electronic device, characterized in that, include: The memory and the processor are coupled; The memory is used to store one or more computer instructions; The processor is configured to execute one or more computer instructions to implement the method as described in any one of claims 1-10.

13. A computer-readable storage medium storing one or more computer instructions thereon, characterized in that, The instruction is executed by the processor to implement the method as described in any one of claims 1-10.

Images