Posture recognition method and device, model training method and device, operation machine and medium

Abstract

The embodiment of the invention discloses a posture recognition method, a model training method and device, an operation machine and a medium, and belongs to the technical field of equipment recognition. The posture recognition method comprises the following steps: acquiring first working condition data and second working condition data of an operation machine; inputting the first working condition data into the trained mining cycle identification model, and determining a mining cycle stage of the operation machine; according to the excavation cycle stage and the second working condition data, determining the body posture of the operation machine through an inclination angle calculation model; and determining the working posture of the working machine based on the excavation cycle stage and the body posture of the working machine. Compared with only determination of the excavation circulation stage, the method has the advantages that the more accurate working posture of the working machine is determined on the basis of the excavation circulation stage and the machine body posture of the working machine, and then more accurate information support is provided for operation and control of the working machine.

Description

Technical Field

[0001] The present application relates to the technical field of equipment identification, and specifically relates to a posture recognition method, a model training method, a device, a construction machine, and a medium. Background Art

[0002] With the rapid development of the manufacturing technology of construction machinery, construction machines are widely used in various operation scenarios such as construction and mining. Usually, construction machines need to perform various operation contents such as slope brushing operation, crushing operation, and excavation operation to meet the requirements of actual operation scenarios. By identifying the real-time operation content of construction machines, the operation process of construction machines can be optimized, and thus the operation efficiency of construction machines can be improved.

[0003] When a large number of sensors are set on construction machines for work content recognition, the time cost of monitoring data such as the flow rate of construction machines is too high, and a large amount of data needs to be processed, resulting in poor real-time performance of recognition results. When the number of detection devices such as sensors on construction machines is reduced, incomplete information easily leads to insufficient accuracy and versatility of recognition. It can only identify the general work content of construction machines, resulting in the inability to accurately identify the posture of construction machines during work. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to provide a posture recognition method, a model training method, a device, a construction machine, and a medium, so as to solve the problem that the working posture of the recognized construction machine in the prior art is inaccurate.

[0005] To achieve the above purpose, the first aspect of the present application provides a posture recognition method for a construction machine, and the posture recognition method for a construction machine includes:

[0006] Obtain first working condition data and second working condition data of the construction machine, where the first working condition data includes the pilot pressure of each working device, and the second working condition data includes the included angle corresponding to each working device;

[0007] Input the first working condition data into the trained excavation cycle recognition model to determine the excavation cycle stage of the construction machine;

[0008] According to the excavation cycle stage and the second working condition data, determine the body posture of the construction machine through the inclination angle calculation model;

[0009] Based on the excavation cycle stage and the body posture of the construction machine, determine the working posture of the construction machine.

[0010] In the embodiments of the present application, the excavation cycle stage includes at least two action stages that sequentially change cyclically based on time. Inputting the first working condition data into the excavation cycle recognition model to determine the excavation cycle stage of the construction machine includes:

[0011] Input the first working condition data into the excavation cycle recognition model, and obtain the first excavation cycle recognition result according to the output results of the excavation cycle recognition model for a continuous preset number of times;

[0012] Obtain the second excavation cycle recognition result according to the pilot pressure of each working device;

[0013] In the case where the first excavation cycle recognition result and / or the second excavation cycle recognition result indicates that the excavation cycle stage has switched, determine the switched action stage as the excavation cycle stage of the working machine.

[0014] In the embodiments of the present application, the pilot pressure of the working device includes a plurality of pilot pressure values based on a preset time interval;

[0015] Obtaining the second excavation cycle recognition result according to the pilot pressure of each working device includes:

[0016] Determine the target working device corresponding to the next excavation stage;

[0017] In the case where the target pilot pressure value of the target working device reaches the preset pressure peak value, determine that the second excavation cycle recognition result is that the excavation cycle stage has switched; or

[0018] In the case where the preset number of target pilot pressure values do not reach the preset pressure peak value within the preset time period, and the sum of the preset number of target pilot pressure values is greater than the preset pressure threshold value, determine that the second excavation cycle recognition result is that the excavation cycle stage has switched.

[0019] In the embodiments of the present application, determining the working attitude of the working machine based on the excavation cycle stage and the body attitude of the working machine includes:

[0020] Determine the included angle corresponding to each working device according to the excavation cycle stage of the working machine;

[0021] Determine the attitude of each working device based on the included angle corresponding to each working device;

[0022] Determine the working attitude of the working machine according to the excavation cycle stage, the attitude of each working device, and the body attitude of the working machine.

[0023] In the embodiments of the present application, determining the body attitude of the working machine through the inclination calculation model according to the excavation cycle stage and the second working condition data includes:

[0024] In the case where the rotation angle of the working machine is obtained, use the rotation angle as a known quantity, update the equations corresponding to the inclination calculation model to equations based on time series, and obtain the updated inclination calculation model;

[0025] Determine the front - rear inclination angle and left - right inclination angle of the body of the working machine through the updated inclination angle calculation model according to the slewing angle, the included angle corresponding to each working device, and the inclination angle of each working device;

[0026] Determine the body attitude of the working machine according to the front - rear inclination angle and left - right inclination angle of the body.

[0027] The second aspect of the present application provides a training method for a digging cycle recognition model. The training method for the digging cycle recognition model includes:

[0028] Obtain historical working condition data;

[0029] Determine the historical working condition data as sample features, and determine the digging cycle stage corresponding to the historical working condition data as sample labels to construct a training sample set;

[0030] Perform iterative training on a preset initial model based on the training sample set until the preset initial model converges to obtain a digging cycle recognition model.

[0031] The third aspect of the present application provides a body attitude recognition device for a working machine. The body attitude recognition device for the working machine includes:

[0032] A working condition data acquisition module, configured to acquire the first working condition data and the second working condition data of the working machine, where the first working condition data includes the pilot pressure of each working device, and the second working condition data includes the included angle corresponding to each working device;

[0033] A cycle stage determination module, configured to input the first working condition data into the trained digging cycle recognition model to determine the digging cycle stage of the working machine;

[0034] A body attitude determination module, configured to determine the body attitude of the working machine through the inclination angle calculation model according to the digging cycle stage and the second working condition data;

[0035] A working attitude determination module, based on the digging cycle stage and the body attitude of the working machine, determines the working attitude of the working machine.

[0036] The fourth aspect of the present application provides a training device for a digging cycle recognition model. The training device for the digging cycle recognition model includes:

[0037] A historical data acquisition module, configured to acquire historical working condition data;

[0038] A training sample construction module, configured to determine the historical working condition data as sample features, and determine the digging cycle stage corresponding to the historical working condition data as sample labels to construct a training sample set;

[0039] An identification model obtaining module, configured to iteratively train a preset initial model based on a training sample set until the preset initial model converges, so as to obtain a mining cycle identification model.

[0040] The fifth aspect of the present application provides a construction machine, including a memory, a processor, and at least one working device;

[0041] The memory is configured to store instructions;

[0042] The processor is configured to call instructions from the memory and, when executing the instructions, be capable of implementing the posture recognition method of the above-mentioned construction machine and / or the training method of the above-mentioned mining cycle identification model;

[0043] At least one working device is configured to execute a preset action.

[0044] The sixth aspect of the present application provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute the posture recognition method of the above-mentioned construction machine and / or the training method of the above-mentioned mining cycle identification model.

[0045] The present application provides a posture recognition method for a construction machine, including: obtaining first working condition data and second working condition data of the construction machine; inputting the first working condition data into a trained mining cycle identification model to determine the mining cycle stage of the construction machine; according to the mining cycle stage and the second working condition data, determining the fuselage posture of the construction machine through an inclination calculation model; and determining the working posture of the construction machine based on the mining cycle stage and the fuselage posture of the construction machine. Compared with only determining the mining cycle stage, the present application determines a more accurate working posture of the construction machine based on the mining cycle stage and the fuselage posture of the construction machine, thereby providing more accurate information support for the control of the construction machine. In addition, the construction machine does not need to be additionally equipped with measuring devices such as gyroscopes and spirit levels, reducing the time cost of data processing, and thus can efficiently obtain the accurate working posture of the construction machine.

[0046] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific implementation part. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The drawings are used to provide a further understanding of the embodiments of the present application, and constitute a part of the specification. They are used to explain the embodiments of the present application together with the following specific implementation manners, but do not constitute a limitation to the embodiments of the present application. In the drawings:

[0048] Figure 1 Schematically shows a flowchart of a posture recognition method for a construction machine according to an embodiment of the present application;

[0049] Figure 2Schematically shows an example diagram of the inclination angle of a work machine according to an embodiment of the present application;

[0050] Figure 3 Schematically shows an example diagram of a work machine vector according to an embodiment of the present application;

[0051] Figure 4 Schematically shows a flow diagram of a method for training a mining cycle recognition model according to an embodiment of the present application;

[0052] Figure 5 Schematically shows a structural diagram of an attitude recognition device for a work machine according to an embodiment of the present application;

[0053] Figure 6 Schematically shows a structural diagram of a training device for a mining cycle recognition model according to an embodiment of the present application;

[0054] Figure 7 Schematically shows an example diagram of a machine-readable storage medium according to an embodiment of the present application. Detailed implementation manners

[0055] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. It should be understood that the specific implementation manners described herein are only used to illustrate and explain the embodiments of the present application, and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0056] It should be noted that the acquisition, transmission, storage, use, processing, etc. of data in the technical solutions of the present application all comply with the relevant regulations of national laws and regulations. In the embodiments of the present application, some industry-existing solutions such as certain software, components, models, etc. may be mentioned. They should be regarded as exemplary, and their purpose is only to illustrate the feasibility in the implementation of the technical solutions of the present application, but it does not mean that the applicant has already or necessarily used this solution.

[0057] It should be noted that if there are directional indications (such as up, down, left, right, front, back...) involved in the embodiments of the present application, the directional indications are only used to explain the relative positional relationship and movement conditions between components in a specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indications will also change accordingly.

[0058] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0059] Example 1

[0060] Figure 1 The flowchart of a posture recognition method for a working machine according to an embodiment of the present application is schematically shown. Figure 1 As shown, the embodiment of the present application provides a posture recognition method for a working machine, comprising the following steps:

[0061] S110, obtaining first working condition data and second working condition data of the working machine, wherein the first working condition data includes the pilot pressure of each working device, and the second working condition data includes the angle corresponding to each working device.

[0062] The first working condition data and the second working condition data are obtained when the operating machine is running. In this embodiment, the operating machine includes a variety of working devices, two main pumps, an engine, a cylinder and other devices. The type of working device is set according to actual needs and is not limited here. For ease of understanding, the working device in the embodiment of this application includes a boom, a bucket, and a dipper rod.

[0063] The first working condition data includes the pilot pressure of each working device and the first other working condition data. The pilot pressure of the working device includes the dipper stick extension pilot pressure, the dipper stick retraction pilot pressure, the boom lowering pilot pressure, the boom raising pilot pressure, the bucket excavation pilot pressure and the bucket unloading pilot pressure, that is, the pilot pressure of 6 working devices. The first other working condition data includes the pressure of the two main pumps, the engine speed, the swing pilot pressure, the left travel pilot pressure, the right travel pilot pressure and the engine torque.

[0064] The second working condition data includes the angle corresponding to each working device and the first other working condition data. The angles corresponding to the working devices include the boom angle, the bucket angle and the dipper rod angle. The second other working condition data includes the dipper rod cylinder large chamber pressure, the dipper rod cylinder small chamber pressure, the boom cylinder large chamber pressure, the boom cylinder small chamber pressure, the bucket cylinder large chamber pressure and the bucket cylinder small chamber pressure.

[0065] S120, input the first working condition data into the trained excavation cycle recognition model to determine the excavation cycle stage of the construction machinery.

[0066] The excavation cycle stage is set according to actual requirements and will not be limited here. For ease of understanding, in the embodiments of the present application, the excavation cycle stage includes four action stages, namely the material excavation stage, the lifting and slewing stage, the discharging stage, and the empty bucket return stage. When the construction machinery is not in a non-excavation state, the excavation cycle stage changes based on time and sequentially cycles through one of the action stages of the material excavation stage, the lifting and slewing stage, the discharging stage, and the empty bucket return stage. Input the first working condition data into the trained excavation cycle recognition model to determine the excavation cycle stage of the construction machinery.

[0067] When the excavation cycle recognition model outputs that the excavation cycle stage is the material excavation stage, the bucket cylinder extends until it reaches the maximum. It starts from when the cutting teeth of the bucket contact the material, to when the bucket cuts the material, until the material fully fills the bucket.

[0068] When the excavation cycle recognition model outputs that the excavation cycle stage is the lifting and slewing stage, the boom cylinder continuously extends to lift the boom to the first preset height. When the bucket is lifted to the second preset height, the turntable of the construction machinery slews to the preset slewing angle to slew the material in the bucket to the designated discharging position. The values of the first preset height, the second preset height, and the preset slewing angle are all set according to actual requirements and will not be limited here.

[0069] When the excavation cycle recognition model outputs that the excavation cycle stage is the discharging stage, after the bucket reaches the discharging position, the bucket cylinder contracts to open the bucket outward, thereby unloading the material in the bucket to the designated discharging position.

[0070] When the excavation cycle recognition model outputs that the excavation cycle stage is the empty bucket return stage, the turntable of the construction machinery slews in the reverse direction to return the bucket of the construction machinery from the discharging position to the excavation point position for excavating the material. When the bucket slews above the excavation point position, the turntable brakes and the boom descends, and the bucket and the dipper stick are adjusted to move the bucket to the excavation point position to wait to enter the excavation training stage again.

[0071] In the embodiments of the present application, the excavation cycle stage includes at least two action stages that sequentially cycle based on time. Inputting the first working condition data into the excavation cycle recognition model to determine the excavation cycle stage of the construction machinery includes:

[0072] Input the first working condition data into the excavation cycle recognition model, and obtain the first excavation cycle recognition result according to the output results of the excavation cycle recognition model for a continuous preset number of times;

[0073] Obtain the second excavation cycle recognition result according to the pilot pressure of each working device;

[0074] In the case where the first excavation cycle recognition result and / or the second excavation cycle recognition result indicates a switch in the excavation cycle stage, determine the switched action stage as the excavation cycle stage of the construction machine.

[0075] Input the first working condition data into the excavation cycle recognition model. In this embodiment, all the first working condition data is input into the excavation cycle recognition model as a one-dimensional float (floating) array. The data sequence is the retraction pilot pressure of the arm, the extension pilot pressure of the arm, the lowering pilot pressure of the boom, the raising pilot pressure of the boom, the excavation pilot pressure of the bucket, the unloading pilot pressure of the bucket, the pressure of the first main pump, the pressure of the second main pump, the slewing pilot pressure, the engine speed, and the engine torque. In the embodiment of the present application, the excavation cycle stage includes four action stages, that is, the excavation cycle stage sequentially cycles through one of the material excavation stage, the lifting and slewing stage, the unloading stage, and the empty bucket return stage based on time.

[0076] The actual first working condition data is continuous data. Continuously input the first working condition data into the excavation cycle recognition model, so that the excavation cycle recognition model continuously outputs multiple results. Obtain the first excavation cycle recognition result according to the continuously preset number of output results of the excavation cycle recognition model. The preset number is set according to actual requirements and is not limited here. For ease of understanding, the preset number is 8 in the embodiment of the present application. The excavation cycle stage of the construction machine can only be sequentially switched to the material excavation stage, the lifting and slewing stage, the unloading stage, and the empty bucket return stage. In the case where the continuously 8 output results of the excavation cycle recognition model indicate a switch in the excavation cycle stage, the first excavation cycle recognition result is that the excavation cycle stage has switched. Specifically, in the case where the excavation cycle stage is previously determined to be the material excavation stage, and the continuously 8 output results of the excavation cycle recognition model indicate a switch to the lifting and slewing stage, the first excavation cycle recognition result is that the excavation cycle stage has switched to the lifting and slewing stage.

[0077] According to the pilot pressure of each working device, directly determine whether the pilot pressure of the working device reaches the peak value, and then obtain the second excavation cycle recognition result. In fact, there is a situation where the first excavation cycle recognition result is inconsistent with the second excavation cycle recognition result. In this embodiment, in the case where the first excavation cycle recognition result and / or the second excavation cycle recognition result indicates a switch in the excavation cycle stage, determine the switched action stage as the excavation cycle stage of the construction machine.

[0078] It should be understood that the first working condition data can also be normalized to avoid the influence of outliers and extreme values. For different pilot pressures, the normalization quantiles are set according to actual requirements and are not limited here. Taking the quartile normalization as an example, the normalized pilot pressure = (current pilot pressure value - lower quartile) / (upper quartile - lower quartile). When calculating extreme values, the largest and smallest quarters are removed.

[0079] The first working condition data is normalized to obtain the normalized first working condition data. The normalized first working condition data is input into the excavation cycle recognition model, and according to the output results of the excavation cycle recognition model for a continuous preset number of times, the first excavation cycle recognition result is obtained. According to the normalized pilot pressure of each working device, the second excavation cycle recognition result is obtained, and then according to the first excavation cycle recognition result and / or the second excavation cycle recognition result, the excavation cycle stage of the construction machine is determined.

[0080] In the embodiments of the present application, the pilot pressure of the working device includes a plurality of pilot pressure values based on a preset time interval;

[0081] Obtaining the second excavation cycle recognition result according to the pilot pressure of each working device includes:

[0082] Determine the target working device corresponding to the next excavation stage;

[0083] When the target pilot pressure value of the target working device reaches the preset pressure peak, it is determined that the second excavation cycle recognition result is that the excavation cycle stage has changed; or

[0084] When the preset number of target pilot pressure values do not reach the preset pressure peak within the preset time period and the sum of the preset number of target pilot pressure values is greater than the preset pressure threshold, it is determined that the second excavation cycle recognition result is that the excavation cycle stage has changed.

[0085] The pilot pressure of the working device is used as the input, and the input time interval is 10 Hz. When the excavation cycle stage of the construction machine is known, the change order of the excavation cycle stage can be directly determined. For example, if the excavation cycle stage is the material excavation stage, then according to the change order of the excavation cycle stage, it is determined whether the excavation cycle stage has switched to the lifting and slewing stage, and the construction machine only needs to determine whether the switch occurs based on the change order.

[0086] The target working device is the working device for which the excavation cycle stage is to be determined, which will not be elaborated here. The preset pressure peak is set according to actual requirements and is not limited here. For ease of understanding, the preset pressure peak in the embodiments of the present application is 0.5. When the target pilot pressure value of the target working device reaches the preset pressure peak, that is, when the pilot pressure of the target working device reaches 0.5, it is determined that the second excavation cycle recognition result is that the excavation cycle stage has switched.

[0087] When the boom retraction pilot pressure and the bucket excavation pilot pressure reach the preset pressure peak, it is determined that the machine has switched to the material excavation stage. When the boom raising pilot pressure reaches the preset pressure peak and the bucket unloading pilot pressure does not reach the preset pressure peak, it is determined that the machine has switched to the lifting and slewing stage. When the bucket unloading pilot pressure reaches the preset pressure peak, it is determined that the machine has switched to the discharging stage. When the boom lowering pilot pressure reaches the preset pressure peak, it is determined that the machine has switched to the empty bucket return stage. In addition, if the duration for which the working machine remains in the same state exceeds the preset duration, it is determined that the working machine is in a non-excavation state.

[0088] The preset quantity, preset duration period, and preset pressure threshold are all set according to actual requirements and are not limited here. For ease of understanding, in the embodiments of the present application, the preset quantity is 10, the preset duration period is 4 seconds, and the preset pressure threshold is 9. The user does not push the pilot handle to the end, but operates the working device with small amplitude and high frequency to complete relatively delicate operations. When the preset number of pilot pressure values do not reach the preset pressure peak within the preset duration period and the sum of the preset number of pilot pressure values is greater than the preset pressure threshold, it is determined that the second excavation cycle recognition result is the next excavation stage. In this embodiment, if the sum of the data of 40 pilot pressures within 4 seconds is greater than 9, it is determined that the second excavation cycle recognition result is the next excavation stage.

[0089] S130. According to the excavation cycle stage and the second working condition data, determine the body attitude of the working machine through the inclination angle calculation model;

[0090] According to the excavation cycle stage and the second working condition data, calculate the front-back inclination angle and the left-right inclination angle of the body of the working machine through the inclination angle calculation model. The front-back inclination angle of the body refers to the angle between the chassis and the horizontal plane in the front-back direction of the body of the working machine. The left-right inclination angle of the body refers to the angle between the chassis and the horizontal plane in the left-right direction of the body. According to the front-back inclination angle and the left-right inclination angle of the body, determine the body attitude of the working machine, that is, in this embodiment, the body attitude of the working machine includes the front-back attitude and the left-right attitude of the body of the working machine.

[0091] In the embodiments of the present application, the inclination angle calculation model is obtained through the following steps:

[0092] Determine the included angle and inclination angle corresponding to each working device as known quantities, and determine the longitudinal inclination angle and lateral inclination angle of the body of the working machine as unknown quantities, construct a system of equations, and obtain an inclination angle calculation model.

[0093] Figure 2 Schematically shows an exemplary diagram of the inclination angle of a working machine according to an embodiment of the present application.

[0094] As shown in the figure, the data set in the inclination angle calculation model includes the longitudinal inclination of the body, the lateral inclination angle of the body, the slewing angle of the body, the boom included angle, the arm included angle, the bucket included angle, the boom inclination angle, the arm inclination angle, the bucket inclination angle, the Z-axis unit vector, the X-axis unit vector, the Y-axis unit vector, the body axial unit vector, the body forward unit vector, and the body leftward unit vector.

[0095] Figure 3 Schematically shows an exemplary diagram of a working machine vector according to an embodiment of the present application.

[0096] Since the included angle corresponding to each working device can be determined according to the excavation cycle stage of the working machine, determine the included angle and inclination angle corresponding to each working device as known quantities, and determine the longitudinal inclination angle and lateral inclination angle of the body of the working machine as unknown quantities, construct a system of equations, and solve for the body axial unit vector, the body forward unit vector, and the body leftward unit vector:

[0097] z r = R x R y z Formula (1)

[0098] x r = cos(sw)R x R y x + sin(sw)(z r ×(R x R y x)) Formula (2)

[0099]

[0100] Among them, z r is the body axial unit vector, x r is the body forward unit vector, x is the X-axis unit vector, z is the Z-axis unit vector, sw is the body slewing angle, swx is the lateral inclination angle of the body, swy is the longitudinal inclination angle of the body, R x is the rotation matrix of the X-axis, R y is the rotation matrix of the Y-axis.

[0101] Based on the above formulas, with the longitudinal inclination of the body, the lateral inclination angle of the body, and the body slewing angle as unknowns, a system of equations can be constructed:

[0102]

[0103] where z r is the unit vector of the fuselage axis, z is the unit vector of the Z-axis, x r is the unit vector in the forward direction of the fuselage, boomAngle is the boom angle, armAngle is the arm angle, bucketAngle is the bucket angle, θ1 is the boom inclination angle, θ2 is the arm inclination angle, and θ3 is the bucket inclination angle.

[0104] Regarding the forward and backward inclination angles and left and right inclination angles of the fuselage of the working machine as unknowns, a system of equations is constructed to obtain an inclination angle calculation model. The system of equations has a unique solution within one period of sin . The system of equations can be solved by the Gauss-Newton method, Newton method, and Levenberg-Marquardt method, which will not be elaborated here. For ease of understanding, a numerical solution method is adopted in the embodiments of the present application. Since the inclination angle of the fuselage is generally not too large, p(sw, swx, swy) = (0, 0, 0) is used as the starting point, where sw is the slewing angle of the fuselage, swx is the left and right inclination angle of the fuselage, and swy is the forward and backward inclination angle of the fuselage. The Newton method is used to solve the local convergent solution within a neighborhood of the starting point of the system of equations. When gradually iterating until △p < 0.01, the approximate solution is used as the output of the model.

[0105] In the embodiments of the present application, according to the excavation cycle stage and the second working condition data, the fuselage attitude of the working machine is determined through the inclination angle calculation model, including:

[0106] When the slewing angle of the working machine is obtained, the slewing angle is regarded as a known quantity, and the system of equations corresponding to the inclination angle calculation model is updated to a system of equations based on time series to obtain an updated inclination angle calculation model;

[0107] According to the slewing angle, the angle corresponding to each working device, and the inclination angle of each working device, the forward and backward inclination angles and left and right inclination angles of the fuselage of the working machine are determined through the updated inclination angle calculation model;

[0108] According to the forward and backward inclination angles and left and right inclination angles of the fuselage, the fuselage attitude of the working machine is determined.

[0109] During the solution process of the system of equations in formula (5), the accuracy of the solution is affected by the accuracy of the angle, and the boom angle, arm angle, and bucket angle are all estimated values obtained according to the excavation cycle stage. When the slewing angle of the working machine is obtained, the system of equations corresponding to the inclination angle calculation model can be updated to a system of equations based on time series to obtain an updated inclination angle calculation model. In the case of including multiple working devices, the calculation can be performed through the angle and inclination angle corresponding to one of the working devices. For ease of understanding, in the embodiments of the present application, a system of equations based on time series is obtained based on the angle and inclination angle of the boom:

[0110]

[0111] Among them, z r is the fuselage axial unit vector, z is the Z-axis unit vector, x r is the forward unit vector of the fuselage, boomAngle is the boom angle, θ 1_tn is the boom inclination angle at the tnth moment.

[0112] The pilot pressure can be used to determine the time period when the boom does not move significantly, and then an equation can be constructed for each moment to obtain a group of equations based on the time series. The steps of the group of equations based on the boom and bucket data are not repeated here. According to the rotation angle, the angle corresponding to each working device, and the inclination angle of each working device, the updated inclination calculation model is used to determine the front and rear inclination angle and the left and right inclination angle of the fuselage of the operating machine. The number of equations in the group of equations is much larger than the number of unknown quantities, and they can be solved separately, verified with each other, or the error can be reduced by the average value. According to the front and rear inclination angle and the left and right inclination angle of the fuselage, the body posture of the operating machine is determined.

[0113] S140, determining the working posture of the working machine based on the excavation cycle stage and the body posture of the working machine.

[0114] Usually, the working posture of the working machine is determined only according to the excavation cycle stage of the working machine. Since only the approximate working content of the working machine is identified, the working posture of the working machine cannot be accurately identified. Compared with only determining the excavation cycle stage, the present application determines a more accurate working posture of the working machine based on the excavation cycle stage and the body posture of the working machine, thereby providing more accurate information support for the control of the working machine. Specifically, in the case where the working posture includes the excavation cycle stage as the material excavation stage, the working posture is determined to be abnormal in combination with the working posture including the body posture, that is, whether the front and rear inclination angle of the body of the working machine and the left and right inclination angle of the body match the range of the body inclination angle corresponding to the material excavation stage. In the case where the front and rear inclination angle of the body of the working machine and the left and right inclination angle of the body do not match the range of the body inclination angle corresponding to the material excavation stage, the working posture of the working machine is determined to be abnormal, thereby realizing the intelligent control of the working machine such as bucket content weighing and anti-rollover detection. In addition, the working machine does not need to be equipped with additional measuring devices such as gyroscopes and levels, which reduces the time cost of data processing, and thus can efficiently obtain accurate working postures of the working machine.

[0115] In an embodiment of the present application, determining the working posture of the working machine based on the excavation cycle stage and the body posture of the working machine includes:

[0116] Determine the angle corresponding to each working device according to the digging cycle stage of the working machine;

[0117] Determine the posture of each working device based on the included angle corresponding to each working device;

[0118] Determine the working posture of the working machine according to the excavation cycle stage, the posture of each working device, and the body posture of the working machine.

[0119] After determining the excavation cycle stage, the included angle corresponding to each working device can be determined according to the excavation cycle stage of the working machine. Determine the posture of each working device based on the included angle corresponding to each working device. Determine the working posture of the working machine according to the excavation cycle stage, the posture of each working device, and the body posture of the working machine. Identify the determined working posture of the working machine, which is used to determine whether there are abnormalities in the posture of the working device and the body posture of the working machine corresponding to the excavation training stage, and can then be used in intelligent control scenarios of the working machine such as excavation times counting, weighing the contents of the bucket, and anti-tipping detection, which will not be elaborated here.

[0120] In the embodiments of the present application, the posture recognition method further includes:

[0121] Send the working posture, the first working condition data, and the second working condition data of the working machine to the cloud device;

[0122] Update the excavation cycle recognition model according to the update information sent by the received cloud device.

[0123] The excavation cycle recognition model is a machine learning model, and the type of the excavation cycle recognition model is set according to actual needs, which can be a random forest model, a hidden Markov model, etc., and will not be limited here. The working machine sends the output working posture, the first working condition data, and the second working condition data to the cloud device through remote communication.

[0124] The cloud device trains and optimizes the excavation cycle recognition model to generate update information of the excavation cycle recognition model. The cloud device sends the update information to the working machine, and the working machine updates the excavation cycle recognition model according to the received update information sent by the cloud device to improve the model performance, so that the excavation cycle recognition model can more accurately identify the excavation cycle stage.

[0125] The present application provides a method for recognizing the attitude of a working machine, including: obtaining first working condition data and second working condition data of the working machine; inputting the first working condition data into a trained excavation cycle recognition model to determine the excavation cycle stage of the working machine; according to the excavation cycle stage and the second working condition data, determining the body attitude of the working machine through an inclination calculation model; and determining the working attitude of the working machine based on the excavation cycle stage and the body attitude of the working machine. Compared with only determining the excavation cycle stage, the present application determines a more accurate working attitude of the working machine based on the excavation cycle stage and the body attitude of the working machine, thereby providing more accurate information support for the control of the working machine. In addition, the working machine does not need to be additionally equipped with measuring devices such as gyroscopes and spirit levels, reducing the time cost of data processing, and thus can efficiently obtain an accurate working attitude of the working machine.

[0126] Embodiment 2

[0127] Figure 4 Schematically shows a flowchart of a training method for an excavation cycle recognition model according to an embodiment of the present application. As Figure 4 shown, the embodiment of the present application provides a training method for an excavation cycle recognition model, including the following steps:

[0128] S210, obtaining historical working condition data.

[0129] It should be understood that when training the excavation cycle recognition model, historical working condition data of different models of working machines are obtained to improve the performance of the machine learning model. Since the sensor positions of different models of working machines are also different, there is a situation of data misalignment. In this embodiment, null value filling and data alignment are pre-performed on the historical working condition data to obtain aligned historical working condition data. Since the acquisition of data signals is affected by vibration and interference, the signals are often mixed with noise. In this embodiment, the aligned historical working condition data is subjected to Kalman filtering to use the measured value at the current moment to correct the estimated value in the prediction stage and obtain the posterior estimated value.

[0130] S220, determining the historical working condition data as sample features and determining the excavation cycle stage corresponding to the historical working condition data as sample labels to construct a training sample set.

[0131] It should be understood that during the actual excavation operation process, the duration of each bucket of material excavated by the working machine varies, resulting in no fixed cycle in the actual excavation cycle stage. However, the excavation cycle stage is sequentially switched to the material excavation stage, the lifting and slewing stage, the unloading stage, and the empty bucket return stage. For ease of understanding, in the embodiments of the present application, the acquired historical working condition data includes historical pilot pressure and historical main pump pressure, and the historical working condition data is data corresponding to the known excavation cycle stage. The historical working condition data is determined as the sample feature, and the excavation cycle stage corresponding to the historical working condition data is determined as the sample label, that is, the historical pilot pressure and historical main pump pressure are used as the main sample features and the excavation cycle stage is used as the label to construct a training sample set including a large number of training samples.

[0132] S230. Iteratively train the preset initial model based on the training sample set until the preset initial model converges to obtain an excavation cycle recognition model.

[0133] Iteratively train the preset initial model based on the training sample set until the preset initial model converges to obtain an excavation cycle recognition model. The model type of the preset initial model is set according to actual needs, which can be a random forest model or a hidden Markov model, and is not limited here. For ease of understanding, in the embodiments of the present application, the preset initial model is a random forest model. The random forest model is an ensemble learning model that makes predictions by constructing and combining multiple decision trees. In this embodiment, the random forest model generates 100 decision trees. After each decision tree independently makes a prediction based on the input training samples, the final prediction result is synthesized by voting, so that the preset initial model converges, and then an excavation cycle recognition model is obtained.

[0134] In addition, during the training process of the preset initial model, the training samples are sampled with replacement, and the sampling amount is equal to the number of samples to construct new training samples. Train a decision tree on each new training sample, select the optimal attribute for division, and each division does not use all features, but randomly selects a part of the features from all features and then selects the optimal feature from them for division.

[0135] Embodiment 3

[0136] Figure 5 Schematically shows a structural schematic diagram of an attitude recognition device of a working machine according to an embodiment of the present application. As Figure 5 shown, the embodiments of the present application provide an attitude recognition device for a working machine, including:

[0137] A working condition data acquisition module 310, configured to acquire first working condition data and second working condition data of the working machine, where the first working condition data includes the pilot pressure of each working device, and the second working condition data includes the included angle corresponding to each working device;

[0138] A cycle stage determination module 320, configured to input first working condition data into a trained excavation cycle recognition model to determine the excavation cycle stage of the construction machinery;

[0139] A fuselage attitude determination module 330, configured to determine the fuselage attitude of the construction machinery through an inclination angle calculation model according to the excavation cycle stage and second working condition data;

[0140] A working attitude determination module 340, configured to determine the working attitude of the construction machinery based on the excavation cycle stage and the fuselage attitude of the construction machinery.

[0141] In an embodiment of the present application, the excavation cycle stage includes at least two action stages that cyclically change in sequence based on time, and the cycle stage determination module 320 includes:

[0142] A first result determination sub-module, configured to input first working condition data into the excavation cycle recognition model, and obtain a first excavation cycle recognition result according to the output results of the excavation cycle recognition model for a continuous preset number of times;

[0143] A second result determination sub-module, configured to obtain a second excavation cycle recognition result according to the pilot pressure of each working device;

[0144] A stage determination sub-module, configured to, when the first excavation cycle recognition result and / or the second excavation cycle recognition result indicates that the excavation cycle stage has switched, determine the switched action stage as the excavation cycle stage of the construction machinery.

[0145] In an embodiment of the present application, the pilot pressure of the working device includes a plurality of pilot pressure values based on a preset time interval;

[0146] The second result determination sub-module is further configured to determine a target working device corresponding to the next excavation stage;

[0147] When the target pilot pressure value of the target working device reaches a preset pressure peak value, it is determined that the second excavation cycle recognition result indicates that the excavation cycle stage has switched; or

[0148] When a preset number of target pilot pressure values within a preset time period do not reach the preset pressure peak value, and the sum of the preset number of target pilot pressure values is greater than a preset pressure threshold value, it is determined that the second excavation cycle recognition result indicates that the excavation cycle stage has switched.

[0149] In an embodiment of the present application, the working attitude determination module 340 includes:

[0150] An included angle determination sub-module, configured to determine an included angle corresponding to each working device according to the excavation cycle stage of the construction machinery;

[0151] The working device attitude determination sub-module is used to determine the attitude of each working device based on the included angle corresponding to each working device;

[0152] The construction machine attitude determination sub-module is used to determine the working attitude of the construction machine according to the excavation cycle stage, the attitude of each working device, and the body attitude of the construction machine.

[0153] In the embodiments of the present application, the body attitude determination module 330 includes:

[0154] The model update sub-module is used to, when the slewing angle of the construction machine is obtained, take the slewing angle as a known quantity, update the equations corresponding to the inclination calculation model to equations based on time series, and obtain an updated inclination calculation model;

[0155] The body inclination determination sub-module is used to determine the front-back inclination and left-right inclination of the body of the construction machine through the updated inclination calculation model according to the slewing angle, the included angle corresponding to each working device, and the inclination of each working device;

[0156] The construction machine body attitude determination sub-module is used to determine the body attitude of the construction machine according to the front-back inclination and left-right inclination of the body.

[0157] Embodiment 4

[0158] Figure 6 Schematically shows a structural diagram of a training device for an excavation cycle recognition model according to an embodiment of the present application. As Figure 6 shown, the embodiments of the present application provide a training device for an excavation cycle recognition model, including:

[0159] The historical data acquisition module 410 is used to acquire historical working condition data;

[0160] The training sample construction module 420 is used to determine the historical working condition data as sample features, and determine the excavation cycle stage corresponding to the historical working condition data as sample labels, and construct a training sample set;

[0161] The recognition model acquisition module 430 is used to perform iterative training on a preset initial model based on the training sample set until the preset initial model converges, and obtain an excavation cycle recognition model.

[0162] The embodiments of the present application further provide a construction machine, including a memory, a processor, and at least one working device;

[0163] The memory is configured to store instructions;

[0164] A processor, configured to call instructions from a memory and capable of implementing the above-mentioned posture recognition method of the working machine and / or the above-mentioned training method of the excavation cycle recognition model when executing the instructions;

[0165] At least one working device, configured to perform a preset action.

[0166] The processor includes a kernel, and the kernel retrieves corresponding program units from the memory. One or more kernels can be set, and the above-mentioned posture recognition method of the working machine and / or the above-mentioned training method of the excavation cycle recognition model are implemented by adjusting the kernel parameters.

[0167] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of, for example, read-only memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.

[0168] During the process of the working device performing the preset action, the control device acquires first working condition data and second working condition data, determines the excavation cycle stage of the working machine through the excavation cycle recognition model, and determines the body posture of the working machine through the inclination calculation model. Based on the excavation cycle stage and the body posture of the working machine, a more accurate working posture of the working machine is determined, thereby providing more accurate information support for the control of the working machine.

[0169] The embodiment of the present application also provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to cause a machine to execute the above-mentioned posture recognition method of the working machine and / or the above-mentioned training method of the excavation cycle recognition model.

[0170] Please refer to Figure 7 , Figure 7 which schematically shows an example diagram of a machine-readable storage medium according to an embodiment of the present application.

[0171] The image acquisition device includes a memory 510, a processor 520, and at least one working device 530. The working device 530 can be a boom, a bucket, an arm, etc., which is not limited herein. For ease of understanding, only one working device 530 is shown in the figure. The memory 510, the processor 520, and the working device 530 are connected through a system bus. The processor 520 is used to provide computing and control capabilities, and the memory 510 includes a machine-readable storage medium 511. The memory 510 provides an environment for the operation of the machine-readable storage medium 511. Instructions are stored on the machine-readable storage medium 511, and the instructions are used to cause a machine to execute the above-mentioned posture recognition method of the working machine and / or the above-mentioned training method of the excavation cycle recognition model.

[0172] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0173] The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0174] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that implements the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0175] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0176] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and a memory.

[0177] The memory may include non-permanent memory in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.

[0178] A computer-readable medium includes both permanent and non-permanent, removable and non-removable media and can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, disk storage or other magnetic storage devices, or any other non-transitory medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer-readable medium does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0179] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or apparatus comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or apparatus. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising the element.

[0180] The above are only embodiments of the present application and are not used to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A posture recognition method for a work machine, characterized in that, The attitude recognition method of the construction machinery includes: Obtain the first working condition data and the second working condition data of the construction machinery, where the first working condition data includes the pilot pressure of each working device, and the second working condition data includes the included angle corresponding to each working device; Input the first working condition data into the trained excavation cycle recognition model to determine the excavation cycle stage of the construction machinery; According to the excavation cycle stage and the second working condition data, determine the body attitude of the construction machinery through the inclination angle calculation model; Based on the excavation cycle stage and the body attitude of the construction machinery, determine the working attitude of the construction machinery.

2. The attitude recognition method of the working machine according to claim 1, characterized in that The excavation cycle stage includes at least two action stages that change cyclically in sequence based on time. The step of inputting the first working condition data into the excavation cycle recognition model to determine the excavation cycle stage of the construction machinery includes: Input the first working condition data into the excavation cycle recognition model, and obtain the first excavation cycle recognition result according to the output results of the excavation cycle recognition model for a continuous preset number of times; Obtain the second excavation cycle recognition result according to the pilot pressure of each working device; In the case where the first excavation cycle recognition result and / or the second excavation cycle recognition result indicates that the excavation cycle stage has switched, determine the switched action stage as the excavation cycle stage of the construction machinery.

3. The attitude recognition method of the working machine according to claim 2, characterized in that, The pilot pressure of the working device includes a plurality of pilot pressure values based on a preset time interval; The step of obtaining the second excavation cycle recognition result according to the pilot pressure of each working device includes: Determine the target working device corresponding to the next excavation stage; In the case where the target pilot pressure value of the target working device reaches the preset pressure peak value, determine that the second excavation cycle recognition result is that the excavation cycle stage has switched; or In the case where a preset number of the target pilot pressure values do not reach the preset pressure peak value within a preset time period and the sum of the preset number of the target pilot pressure values is greater than the preset pressure threshold value, determine that the second excavation cycle recognition result is that the excavation cycle stage has switched.

4. The attitude recognition method of the working machine according to claim 1, characterized in that The step of determining the working attitude of the construction machinery based on the excavation cycle stage and the body attitude of the construction machinery includes: Determine the included angle corresponding to each working device according to the excavation cycle stage of the construction machinery; Based on the included angle corresponding to each working device, determine the attitude of each working device; According to the excavation cycle stage, the attitude of each working device, and the body attitude of the construction machinery, determine the working attitude of the construction machinery.

5. The attitude recognition method of the construction machine according to claim 1, characterized in that, According to the excavation cycle stage and the second working condition data, determining the body attitude of the construction machinery through the inclination angle calculation model includes: In the case where the rotation angle of the construction machinery is obtained, use the rotation angle as a known quantity, update the equations corresponding to the inclination angle calculation model to equations based on time series, and obtain the updated inclination angle calculation model; According to the rotation angle, the included angle corresponding to each working device, and the inclination angle of each working device, determine the front-back inclination angle and the left-right inclination angle of the body of the construction machinery through the updated inclination angle calculation model; Determine the body attitude of the working machine according to the front-back inclination angle and the left-right inclination angle of the body.

6. A training method for a mining loop recognition model, characterized in that, The training method of the excavation cycle recognition model includes: Obtain historical working condition data; Determine the historical working condition data as sample features, and determine the excavation cycle stage corresponding to the historical working condition data as sample labels to construct a training sample set; Perform iterative training on a preset initial model based on the training sample set until the preset initial model converges to obtain an excavation cycle recognition model.

7. An attitude recognition device for a working machine, characterized in that, The attitude recognition device of the working machine includes: A working condition data acquisition module for acquiring first working condition data and second working condition data of the working machine, where the first working condition data includes the pilot pressure of each working device, and the second working condition data includes the included angle corresponding to each working device; A cycle stage determination module for inputting the first working condition data into the trained excavation cycle recognition model to determine the excavation cycle stage of the working machine; A body attitude determination module for determining the body attitude of the working machine through an inclination angle calculation model according to the excavation cycle stage and the second working condition data; A working attitude determination module for determining the working attitude of the working machine based on the excavation cycle stage and the body attitude of the working machine.

8. A training device for a mining cycle recognition model, characterized in that The training device of the excavation cycle recognition model includes: A historical data acquisition module for acquiring historical working condition data; A training sample construction module for determining the historical working condition data as sample features, and determining the excavation cycle stage corresponding to the historical working condition data as sample labels to construct a training sample set; An identification model obtaining module for performing iterative training on a preset initial model based on the training sample set until the preset initial model converges to obtain an excavation cycle recognition model.

9. An operating machine, characterized in that, Includes a memory, a processor, and at least one working device; The memory is configured to store instructions; The processor is configured to call the instructions from the memory and be able to implement the attitude recognition method of the working machine according to any one of claims 1 to 5, and / or the training method of the excavation cycle recognition model according to claim 6 when executing the instructions; The at least one working device is configured to perform preset actions.

10. A machine-readable storage medium, characterized in that, Instructions are stored on this machine-readable storage medium, and these instructions are used to cause the machine to execute the attitude recognition method of the working machine according to any one of claims 1 to 5, and / or the training method of the excavation cycle recognition model according to claim 6.

Images