Analysis device, analysis method, program, learned machine learning model, and machine learning method

Abstract

This analysis device comprises: an acquisition unit that acquires time-series data representing at least the operation state of a first movable part and the operation state of a second movable part of a work machine having the first movable part and the second movable part; a first operation state classification unit that classifies the operation state of the first movable part on the basis of the time-series data; a division unit that divides the time axis of the time-series data into a plurality of sections on the basis of the classification result of the operation state of the first movable part; a second operation state classification unit that classifies the operation state of the second movable part for each of the sections on the basis of the time-series data; and a work classification unit that classifies work by the work machine on the basis of the classification result of the operation state of the first movable part and the classification result of the operation state of the second movable part.

Description

Analysis device, analysis method, program, trained machine learning model, and machine learning method

[0001] This application claims priority to Japanese Patent Application No. 2023-196595, filed on November 20, 2023, the contents of which are incorporated herein by reference.

[0002] Patent Document 1 discloses a technique for determining each work mode included in a cycle of work based on a numerical value that indicates the state of the wheel loader when the wheel loader is operated by an operator to perform cyclic work (loading work in this example) that repeats each of the work modes of moving forward empty, excavating, moving backward loaded, moving forward loaded, loading, and moving backward empty.

[0003] International Publication No. 2022 / 244632

[0004] Incidentally, wheel loaders perform multiple types of work, including loading, sorting of excavated material, shoveling, etc. Furthermore, with work machines such as wheel loaders, there is a need to efficiently analyze what type of work the work machine is performing based on time-series data that indicates the operating status of the work machine.

[0005] The present disclosure has been made in consideration of the above circumstances, and aims to provide an analysis device, an analysis method, a program, a trained machine learning model, and a machine learning method that can efficiently classify work based on time-series data representing the operating status of a work machine.

[0006] The analysis device disclosed herein includes an acquisition unit that acquires time series data representing at least the operating state of a first movable part and the operating state of a second movable part of a work machine having the first movable part; a first operating state classification unit that classifies the operating state of the first movable part based on the time series data; a division unit that divides a time axis of the time series data into a plurality of sections based on the classification results of the operating state of the first movable part; a second operating state classification unit that classifies the operating state of the second movable part for each of the sections based on the time series data; and a work classification unit that classifies work performed by the work machine based on the classification results of the operating state of the first movable part and the classification results of the operating state of the second movable part.

[0007] The analysis method disclosed herein also includes the steps of acquiring time series data representing at least the operating state of a first movable part and the operating state of a second movable part of a work machine having the first movable part and the second movable part, classifying the operating state of the first movable part based on the time series data, dividing the time axis of the time series data into a plurality of sections based on the classification results of the operating state of the first movable part, classifying the operating state of the second movable part for each section based on the time series data, and classifying work performed by the work machine based on the classification results of the operating state of the first movable part and the classification results of the operating state of the second movable part.

[0008] The program disclosed herein also causes a computer to execute the following steps: acquiring time series data representing at least the operating state of a first movable part and the operating state of a second movable part of a work machine having the first movable part; classifying the operating state of the first movable part based on the time series data; dividing the time axis of the time series data into a plurality of sections based on the classification results of the operating state of the first movable part; classifying the operating state of the second movable part for each section based on the time series data; and classifying work performed by the work machine based on the classification results of the operating state of the first movable part and the classification results of the operating state of the second movable part.

[0009] In addition, the trained machine learning model of the present disclosure is a machine learning model that inputs predetermined input information based on time series data representing at least the operating state of a first movable part and the operating state of a second movable part of a work machine having the first movable part and the second movable part, and outputs output information representing the results of classifying the operating state of the second movable part, wherein the input information includes a plurality of predetermined feature quantities related to the operating state of the second movable part for each of a plurality of sections obtained by dividing the time axis of the time series data into a plurality of sections based on the classification results of the operating state of the first movable part, and is machine-trained using the input information labeled with the type of operating state of the second movable part.

[0010] The machine learning method of the present disclosure is also a machine learning method for a machine learning model that inputs predetermined input information based on time series data representing at least the operating state of a first movable part and the operating state of a second movable part of a work machine having the first movable part and the second movable part, and outputs output information representing the results of classifying the operating state of the second movable part, wherein the input information includes a plurality of predetermined feature quantities related to the operating state of the second movable part for each of a plurality of sections obtained by dividing a time axis of the time series data into a plurality of sections based on the classification results of the operating state of the first movable part, and the machine learning model is trained using the input information labeled with the type of operating state of the second movable part.

[0011] The analysis device, analysis method, program, trained machine learning model, and machine learning method disclosed herein classify the operating status of the second movable part into multiple sections obtained by dividing time-series data into sections based on the classification results of the first movable part, thereby enabling efficient classification of the operating status of the second movable part. Accordingly, it is possible to efficiently classify the work of a work machine having a first movable part and a second movable part based on the time-series data.

[0012] FIG. 1 is a schematic diagram showing an example configuration of an analysis system according to an embodiment of the present disclosure. FIG. 2 is a side view showing an example configuration of a work machine according to an embodiment of the present disclosure. FIG. 3 is a schematic diagram showing an example configuration of a time-series data file according to an embodiment of the present disclosure. FIG. 4 is a schematic diagram showing an example behavior determination flag of a traveling device (undercarriage of a work machine) according to an embodiment of the present disclosure. FIG. 5 is a diagram showing an example of time-series data related to behavior determination of a traveling device according to an embodiment of the present disclosure. FIG. 6 is a diagram showing an example of time-series data related to behavior determination of a traveling device according to an embodiment of the present disclosure. FIG. 7 is a diagram showing an example of time-series data related to behavior determination of a work machine according to an embodiment of the present disclosure. FIG. 8 is a diagram showing an example of time-series data divided into a plurality of sections according to an embodiment of the present disclosure. FIG. 9 is a diagram showing an example of time-series data related to behavior determination of a work machine according to an embodiment of the present disclosure. FIG. 10 is a diagram showing an example of the operating state of a work machine in section (k) shown in FIGS. 7 and 8. FIG. 11 is a diagram showing an example of the operating state of a work machine in section (k+1) shown in FIGS. 7 and 8. FIG. 12 is a diagram showing an example of the operating state of a work machine in section (k+2) shown in FIGS. 7 and 8. FIG. 13 is a diagram showing an example of the operating state of a work machine in section (k+3) shown in FIGS. 7 and 8. FIG. 14 is a diagram showing an example of the operating state of a work machine in section (k+4) shown in FIGS. 7 and 8. FIG. 15 is a diagram showing an example of feature amounts according to an embodiment of the present disclosure. 7 and 8 . A diagram showing an example of a feature amount for a section (k) according to an embodiment of the present disclosure. A diagram showing an example of a feature amount for a section (k) according to an embodiment of the present disclosure. A diagram showing an example of a behavior determination flag for a work machine according to an embodiment of the present disclosure. A diagram showing an example of a feature amount for a section (k+1) according to an embodiment of the present disclosure. A diagram showing an example of a feature amount for a section (k+2) according to an embodiment of the present disclosure. A diagram showing an example of a feature amount for a section (k+3) according to an embodiment of the present disclosure. A diagram showing an example of a feature amount for a section (k+4) according to an embodiment of the present disclosure. A diagram showing an example of teacher data according to an embodiment of the present disclosure. A schematic diagram showing an example of a task classification according to an embodiment of the present disclosure. A schematic diagram for explaining an example of a task classification according to an embodiment of the present disclosure. A schematic diagram for explaining an example of a task classification for sections (k) to (k+4) shown in FIGS. 7 and 8 . A diagram showing an example of a classification result according to an embodiment of the present disclosure. A diagram showing an example of a result of a detailed analysis according to an embodiment of the present disclosure. A diagram showing an example of a result of a detailed analysis according to an embodiment of the present disclosure.10 is a diagram illustrating an example of a result of a detailed analysis according to an embodiment of the present disclosure;

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In each drawing, the same or corresponding components are designated by the same reference numerals, and the description thereof will be omitted as appropriate.

[0014] (Analysis System) FIG. 1 is a schematic diagram showing an example configuration of an analysis system according to an embodiment of the present disclosure. The analysis system SY1 shown in FIG. 1 includes a wheel loader 1 as an example of a work machine to be analyzed, a data management server 20 communicatively connected to the wheel loader 1 via a communication network NW, and an analysis device 30. The wheel loader 1 includes a vehicle body control device 100 and an operation information recording device 101. The vehicle body control device 100 inputs sensor information measured by multiple sensors provided in the wheel loader 1 and controls each part of the wheel loader 1 in accordance with the operation of an operator. The operation information recording device 101 records time series data representing the operating state of the wheel loader 1, such as predetermined control signals and sensor information generated by the vehicle body control device 100, for a certain period of time (or a certain amount of data), and transmits the data to the data management server 20 via the communication network NW. The data management server 20 accumulates the time series data received from the operation information recording device 101 as a time series data file 21. In this embodiment, the analysis device 30 is provided separately from the data management server 20, but this is not limiting. The analysis device 30 may be provided within the data management server 20.

[0015] (Working Machine) Here, the wheel loader 1 will be described with reference to Fig. 2. Fig. 2 is a side view showing an example configuration of a working machine (wheel loader 1) according to an embodiment of the present disclosure. As shown in Fig. 2, the wheel loader 1 has a vehicle body 2, a cab 3, a traveling device 4, and a working implement 10. The wheel loader 1 travels to a work site using the traveling device 4. The wheel loader 1 performs work at the work site using the working implement 10. The wheel loader 1 can perform work such as excavation work, loading work, transporting work, and snow removal work using the working implement 10, for example.

[0016] The cab 3 is supported by the vehicle body 2. Inside the cab 3, a driver's seat where an operator sits, operation devices, a display input unit, etc. are arranged.

[0017] The traveling device 4 has a power transmission device, rotatable wheels 5, etc., which are not shown. The wheels 5 support the vehicle body 2. The wheel loader 1 is able to travel on a road surface (or ground surface) RS by means of the traveling device 4. Note that only the left front wheel 5F and rear wheel 5R are shown in FIG. 2. The power transmission device transmits driving force from a power source, which will be described later, to the wheels 5. In this embodiment, the power transmission device has a transmission with four forward speeds, a neutral position, and four reverse speeds (speed stages). Note that the traveling device 4 is one example configuration of the "traveling section" of this disclosure.

[0018] The work implement 10 is supported by the vehicle body 2. The work implement 10 is made up of a bucket 12, which is an example of a work tool, and a movable support unit 17 that changes the position and attitude of the bucket 12. In the example shown in Figure 2, the movable support unit 17 includes a boom 11, a boom cylinder 13, a bucket cylinder 14, a bell crank 15, and a link 16.

[0019] The boom 11 is rotatably supported on the vehicle body 2 and moves up and down in response to the extension and contraction of the boom cylinder 13. The boom cylinder 13 is an actuator that generates power for moving the boom 11, with one end connected to the vehicle body 2 and the other end connected to the boom 11. When an operator operates a boom operation device (not shown), the boom cylinder 13 extends and contracts. This causes the boom 11 to move up and down. The boom cylinder 13 is, for example, a hydraulic cylinder. In this embodiment, the angle of the boom 11 is expressed as a value, as indicated by arrow A1, with the horizontal direction being 0 [deg], where positive values ​​indicate an upward angle and negative values ​​indicate a downward angle.

[0020] The bucket 12 has a cutting edge 12T and is a work tool used to excavate and load soil and other excavation targets. The bucket 12 is rotatably connected to the boom 11 and one end of a link 16. The other end of the link 16 is rotatably connected to one end of a bell crank 15. The bell crank 15 has a central portion rotatably connected to the boom 11 and the other end rotatably connected to one end of a bucket cylinder 14. The other end of the bucket cylinder 14 is rotatably connected to the vehicle body 2. The bucket 12 is actuated by power generated by the bucket cylinder 14. The bucket cylinder 14 is an actuator that generates power to move the bucket 12. When an operator operates a bucket operating device (not shown), the bucket cylinder 14 extends and retracts, causing the bucket 12 to swing. The bucket cylinder 14 is, for example, a hydraulic cylinder. The cutting edge 12T has a shape such as a crested blade or a flat blade, and is replaceably attached to the end of the bucket 12. In this embodiment, the angle of the bucket 12 (the angle in the direction in which the cutting edge 12T faces (toward the cutting edge)) is expressed as a value, as shown by arrow A2, with the horizontal direction being 0 degrees, and being positive when facing upward and negative when facing downward.

[0021] In this embodiment, the position of the bucket 12c in which the cutting edge 12T faces downward is referred to as the dump position. The dump position is, for example, a position (earth-discharging position) in which the excavated material in the bucket 12 can be loaded onto a transport vehicle, hopper, or the like. The position of the bucket 12a in which the cutting edge 12T faces upward is referred to as the tilt position (holding position). The tilt position is, for example, a position (transport position) in which the excavated material can be held in the bucket 12. The position of the bucket 12b in which the cutting edge 12T faces horizontally (including substantially horizontally) with the road surface RS is referred to as the digging position (or traveling position during excavation). The digging position is, for example, a position when starting to excavate soil or other excavation targets or traveling toward the excavation target (or a position suitable for starting to excavate or traveling). The position of the bucket 12 in which the cutting edge 12T is in contact with the road surface RS when the boom 11 is lowered is referred to as the ground contact position. The wheel loader 1 starts excavating an object to be excavated that is located ahead by, for example, placing the bucket 12 in an excavation posture (or a posture in which the cutting edge 12T is lower than the road surface RS from the excavation posture) and traveling forward. Note that with the wheel loader 1, the excavation posture can also be called the horizontal posture because the cutting edge direction is substantially horizontal to the road surface RS.

[0022] The wheel loader 1 includes a power source, a power take-off (PTO), a hydraulic pump, a control valve, an operating device, a display input unit, and other components (not shown). The power source generates driving force for operating the wheel loader 1. Examples of the power source include an internal combustion engine and an electric motor. The PTO transmits at least a portion of the driving force of the power source to the hydraulic pump. The PTO distributes the driving force of the power source to the traveling device 4 and the hydraulic pump. The hydraulic pump is driven by the power source and discharges hydraulic oil. At least a portion of the hydraulic oil discharged from the hydraulic pump is supplied to each of the boom cylinder 13 and the bucket cylinder 14 via a control valve. The control valve controls the flow rate and direction of the hydraulic oil supplied from the hydraulic pump to each of the boom cylinder 13 and the bucket cylinder 14. The work implement 10 is operated by the hydraulic oil from the hydraulic pump.

[0023] The operating device is disposed inside the cab 3. The operating device is operated by an operator. The operator operates the operating device to adjust the traveling direction and traveling speed of the wheel loader 1, switch between forward and reverse travel, and operate the work implement 10. The operating device includes, for example, a steering wheel, a shift lever, an accelerator pedal, a brake pedal, and operating devices for operating the boom 11 and bucket 12 of the work implement 10.

[0024] The wheel loader 1 is also equipped with sensors such as a GNSS (Global Navigation Satellite System) receiver, an engine rotation speed sensor, a vehicle speed sensor, a fuel consumption sensor, a work equipment load sensor, a boom angle sensor, and a bucket angle sensor. The GNSS receiver has two receivers (antennas), each of which acquires position information (latitude, longitude, and altitude). The orientation of the wheel loader 1 can be calculated based on the two pieces of position information. The engine rotation speed sensor detects the rotation speed of the engine (power source). The vehicle speed sensor detects the traveling speed of the wheel loader 1. The fuel consumption sensor detects the fuel consumption of the wheel loader 1. The work equipment load sensor detects, for example, the weight (load) of the excavated material held in the bucket 12. The boom angle sensor detects the angle of the boom 11. The bucket angle sensor detects the angle of the bucket 12.

[0025] (Relationship Between First Movable Part and Second Movable Part in the Present Disclosure) As described in the section on Means for Solving the Problems, the work machine to be analyzed by the analysis device in the present disclosure has a first movable part and a second movable part. The first movable part is, for example, a traveling part (traveling body), and in this embodiment corresponds to the traveling device 4. The second movable part is, for example, a work implement, and in this embodiment corresponds to the work implement 10, boom 11, and bucket 12. If the work machine to be analyzed is, for example, a hydraulic excavator, the first movable part corresponds to the rotating part (swivel), and the second movable part corresponds to the work implement (bucket, arm, and boom). Alternatively, the first movable part corresponds to the rotating part and traveling part, and the second movable part corresponds to the work implement. Alternatively, the first movable part corresponds to the traveling part, and the second movable part corresponds to the rotating part and work implement. Other examples of work machines in the present disclosure include dump trucks and bulldozers.

[0026] (Time Series Data) FIG. 3 is a schematic diagram showing an example configuration of a time series data file according to an embodiment of the present disclosure. The time series data file 21 shown in FIG. 3 includes work machine information 22, operator information 23, and time series data 24. The work machine information 22 includes, for example, information such as the individual identification information, model, and manufacturing date of the wheel loader 1. The operator information 23 includes, for example, identification information of the operator of the wheel loader 1. The time series data 24 is time series data that represents the operating status of each part of the wheel loader 1. In the example shown in FIG. 3 , the time series data 24 includes data that represents the date and time, position information, engine rotation speed, gear position, vehicle speed, fuel consumption, load, boom angle, bucket angle, etc., sampled at a predetermined sampling period (for example, every second). Each time series data file 21 can be created, for example, for each time period from when the engine key is turned on to when it is turned off, or can be created by dividing it into multiple files at predetermined intervals. For example, the gear position and vehicle speed are examples of data representing the operating state of the first movable part, and the boom angle and bucket angle are examples of data representing the operating state of the second movable part. In this case, the time-series data 24 is data representing at least the operating state of the first movable part and the operating state of the second movable part of the work machine (wheel loader 1) having a first movable part (traveling device 4) and a second movable part (work implement 10).

[0027] (Configuration of Analysis Device) The analysis device 30 can be configured using a computer such as a personal computer or a tablet terminal. The analysis device 30 has the following functional components, which are configured by a combination of hardware such as a computer and its peripheral devices, and software executed by the computer. That is, the analysis device 30 has the following functional components: an acquisition unit 31, a first operation status classification unit 32, a division unit 33, a second operation status classification unit 34, a trained machine learning model 35, an operation classification unit 36, a detailed analysis unit 37, a memory unit 38, and an output unit 39.

[0028] The acquisition unit 31 acquires a time-series data file 21 including the time-series data 24 to be analyzed, and stores the time-series data file 21 in the storage unit 38. The time-series data file 21 may be automatically acquired from the data management server 20 and stored in the storage unit 38, or may be manually acquired and stored in the storage unit 38.

[0029] The first operating state classification unit 32 classifies the operating state of the traveling device 4 (first movable part) based on the time-series data 24. FIG. 4 is a schematic diagram showing an example of a behavior discrimination flag for the traveling device 4 (undercarriage of a work machine; in this embodiment, the traveling device 4 is also simply referred to as the traveling device) according to an embodiment of the present disclosure. As shown in FIG. 4 , the first operating state classification unit 32 classifies the behavior of the traveling device into four types (stopped, forward, reverse, or neutral coasting) and sets a traveling device behavior discrimination flag (1, 2, 3, or 4) for each type. Here, neutral coasting is a state in which the gear is in neutral and the vehicle travels while lightly applying the brake, for example, to limit the vehicle speed. In the example shown in FIG. 4 , when the vehicle speed value is "0" [km / h], the operating state of the traveling device 4 is classified as "stopped" regardless of the gear value. Also, if the vehicle speed value is not "0", the gears are "forward" in forward 1st to 4th gears (F1 to F2), "reverse" in reverse 1st to 4th gears (R1 to R2), and "neutral coasting" in neutral (N).

[0030] 5 and 6 are diagrams illustrating an example of time-series data related to behavior discrimination of a traveling device according to an embodiment of the present disclosure. In FIGS. 5 and 6, the horizontal axis represents time, the vertical axis (left side) represents vehicle speed and gear position, and the vertical axis (right side) represents the traveling device's behavior discrimination flag, showing changes in the vehicle speed, gear position, and value of the traveling device's behavior discrimination flag. Note that the vehicle speed is shown as an absolute value regardless of whether the vehicle is traveling forward or backward, and the gear position is represented as +1 to +4 for forward gears 1 to 4, 0 for neutral, and −1 to −4 for reverse gears 1 to 4. In the example shown in FIG. 5, the vehicle speed reaches 0 at 10 seconds and 15 seconds, and the value of the traveling device's behavior discrimination flag is 1 ("stop"). If this continues, the behavior will frequently be determined to be "stop" when switching between forward and reverse. Therefore, in the example shown in Figure 6, a correction process is performed to prevent the vehicle from being stopped if the stops are not continuous.The vehicle speed is 0 at 10 seconds and 15 seconds, but if the time when it is 0 does not continue (for example, if it is 0 for only one sampling), the value of the behavior determination flag for the traveling device is not set to 1 ("stop"), but is changed to the value of the behavior before the vehicle speed became 0.

[0031] The segmentation unit 33 segments the time axis of the time-series data 24 into multiple sections based on the classification result of the operating state of the traveling device 4 (first movable part) by the first operating state classification unit 32. FIG. 7 is a diagram illustrating an example of segmenting time-series data into multiple sections according to an embodiment of the present disclosure. In the example illustrated in FIG. 7 , the segmentation unit 33 segments the time axis of the time-series data so that each period classified by the first operating state classification unit 32 as "forward," "reverse," or "stop" constitutes one section. Note that the period classified as "neutral coasting" is combined with the previous classification period and included in the previous type of period. Based on the classification result of the period (0 to 42 seconds) shown in FIG. 6 , the segmentation unit 33 segments the period (0 to 42 seconds) into five sections, section (k) to section (k+4), as shown in FIG. 7 (k is an arbitrary natural number). Note that in this embodiment, each section is divided into 1-second intervals, and the end time of each section is the start time of the next section. Therefore, for example, in the period from 0 to 43 seconds, the time 43 seconds is classified as the next period (k+5), not shown. As will be described later, the period (0 to 42 seconds) is classified as loading work, and in the example shown in Figure 7, the period from period (k) to period (k+4) is one loading work period, and period (k) corresponds to the forward movement (1), period (k+1) corresponds to reverse movement (1), period (k+2) corresponds to forward movement (2), period (k+3) corresponds to stop, and period (k+4) corresponds to reverse movement (2).

[0032] The second operating state classification unit 34 classifies the operating state of the work machine 10 (second movable part) for each section based on the time-series data 24. The second operating state classification unit 34 is a machine learning model that receives input information including a plurality of predetermined feature amounts related to the operating state of the work machine 10 (second movable part) for each section, and outputs output information representing the results of classifying the operating state of the work machine 10 (second movable part). The second operating state classification unit 34 classifies the operating state of the work machine 10 (second movable part) using a trained machine learning model 35 that has been subjected to supervised machine learning using input information labeled with the type of operating state of the work machine 10 (second movable part) as training data. FIG. 8 is a diagram illustrating an example of time-series data related to behavior determination of the work machine 10 (note that in this embodiment, the work machine 10 is also simply referred to as the work machine) according to an embodiment of the present disclosure.

[0033] The trained machine learning model 35 is configured, for example, by a combination of a program that performs calculations from input to output and weighting coefficients (parameters) used in the calculations. Furthermore, the coefficients used in the calculations are optimized by machine learning so that a desired solution is output for a large amount of input data. In this embodiment, it has been confirmed that good accuracy can be achieved by using the k-nearest neighbor method as the algorithm for the machine learning model. However, there are no limitations specific to this embodiment regarding the algorithm for the machine learning model.

[0034] Figure 8 shows time on the horizontal axis, vehicle speed, boom angle, and bucket angle on the vertical axis (left side), and the traveling gear behavior discrimination flag on the vertical axis (right side), and shows changes in the vehicle speed, boom angle, bucket angle, and traveling gear behavior discrimination flag values. The values ​​on the time axis and the vehicle speed and traveling gear behavior discrimination flag values ​​are the same as those in Figure 7. Figures 9 to 13 are diagrams showing examples of the operating state of the work implement 10 in sections (k) to (k+4) shown in Figures 7 and 8. The horizontal axis is the boom angle, and the vertical axis is the bucket angle, and points based on the boom angle and bucket angle at the same time within the target section are plotted.

[0035] FIG. 14 is a diagram illustrating an example of feature quantities according to an embodiment of the present disclosure. Note that feature quantities are numerical values ​​that quantitatively represent the characteristics of the analysis target. In this embodiment, four types of feature quantities are defined. For example, for each plot for each section shown in FIGS. 9 to 13 , the first is the “area” (FV1), which can be expressed by the maximum and minimum boom angle values ​​and the maximum and minimum bucket angle values ​​for the purpose of grasping the position of the work implement 10. The second is the “entrance / exit coordinates” (FV2), which can be expressed by the initial and final boom angle values ​​for the section and the initial and final bucket angle values ​​for the section for the purpose of grasping movement. The third is the “center of gravity coordinates” (FV3), which can be expressed by the average boom angle value and the average bucket angle value for the purpose of grasping movement within the area. The fourth is the "number of peaks" (FV4), which can be expressed as the number of peak-like changes (such as a local maximum value or a change in the slope of the curve exceeding a predetermined threshold) in the curve connecting the plots of the boom angle and bucket angle over time, with the aim of understanding repetition within the area. FIG. 15 is a diagram illustrating an example of feature quantities for section (k) according to an embodiment of the present disclosure. FIG. 16 is a diagram illustrating an example of feature quantities according to an embodiment of the present disclosure. FIG. 15 is a diagram in which figures illustrating feature quantities FV1 to FV3 are added to FIG. 9. FIG. 16 also illustrates an example including a peak (an example during warm-up operation).

[0036] FIG. 17 is a schematic diagram illustrating an example of a behavior discrimination flag for a work machine according to an embodiment of the present disclosure. The second operating state classification unit 34 classifies the operating state of the work machine 10 (second movable part) and sets the value of the behavior discrimination flag for the work machine shown in FIG. 17 for the corresponding section. Furthermore, in this embodiment, the trained machine learning model 35 inputs the four types of feature values ​​shown in FIG. 14 and outputs the value of the behavior discrimination flag for the work machine shown in FIG. 17. In the example shown in FIG. 17, the behavior of the work machine is categorized into 13 types. The association of the behavior discrimination flag for the work machine with the "section," the "posture" and "movement" of the "bucket," and the "posture" and "movement" of the "boom" shown in FIG. 17 is a rough criterion for labeling input information in supervised learning, for example. Note that the "leveler" indicates a state in which the function for automatically adjusting the level is in use. FIGS. 18 to 21 are diagrams illustrating example features for sections (k+1) to (k+4) according to an embodiment of the present disclosure. 18 to 21 are diagrams in which diagrams illustrating feature quantities FV1 to FV3 are added to FIGS. 10 to 13. When the feature quantities shown in FIG. 15 are input, the trained machine learning model 35 outputs, for example, a work machine behavior discrimination flag of "20." The work machine behavior discrimination flag "20" corresponds to the behavior of the work machine in a "scooping" operation. The "scooping" operation is, for example, an operation in which the boom 11 is lowered while moving forward to scoop the excavated material into the bucket 12, and then the bucket 12 is scooped up. When the feature quantities shown in FIGS. 18, 19, 20, and 21 are input, the trained machine learning model 35 outputs, for example, work machine behavior discrimination flags of "31," "51," "12," and "41."

[0037] 22 is a diagram illustrating an example of training data according to an embodiment of the present disclosure. Training data (data set) 351 illustrated in Fig. 22 includes a plurality of sets of data that associate the initial value, maximum value, minimum value, average value, number of peaks, and final value of the boom angle in a target section with the initial value, maximum value, minimum value, average value, number of peaks, and final value of the bucket angle and the value of the behavior determination flag of the work machine (this is the label value).

[0038] The trained machine learning model 35 of this embodiment can be understood as follows. That is, the trained machine learning model 35 is a machine learning model that receives predetermined input information based on time-series data representing at least the operating states of the first and second movable parts of a work machine having a first and second movable part, and outputs output information representing the results of classifying the operating states of the second movable part. The input information includes a plurality of predetermined feature quantities related to the operating states of the second movable part for each of a plurality of sections obtained by dividing the time axis of the time-series data into a plurality of sections based on the results of classifying the operating states of the first movable part. The trained machine learning model 35 is trained using input information labeled with the type of operating state of the second movable part. The second movable part is, for example, a work machine having a plurality of partial movable parts, and the plurality of feature quantities may include one or more values ​​representing the respective movable amounts of the plurality of partial movable parts corresponding to each other on the time axis as coordinate values ​​on a plurality of different coordinate axes. Here, the plurality of partial movable parts correspond to, for example, the boom 11 and the bucket 12. In this case, if the work machine includes three or more movable parts (e.g., a bucket, an arm, and a boom), the coordinate axes can include one or more values ​​represented by coordinate values ​​on the three or more coordinate axes. The one or more values ​​represented by the coordinate values ​​can include values ​​corresponding to the initial value, maximum value, minimum value, average value, or final value of each coordinate value for each section. The plurality of feature quantities can also include a value representing the number of repeated operations of the second movable part that have occurred for each section.

[0039] The task classification unit 36 ​​classifies tasks performed by the wheel loader 1 based on the classification results of the operating status of the traveling device 4 (first movable unit) and the classification results of the operating status of the work implement 10 (second movable unit). The task classification unit 36 ​​classifies tasks by one or more sections. FIG. 23 is a schematic diagram showing an example of task classification according to an embodiment of the present disclosure. In the example shown in FIG. 23 , tasks are classified into nine types: "loading," "raising," "leveling," "sorting," "moving," "parking," "loading-related work," "sorting-related work," and "other." The task classification flag is information that identifies the type of task. "Loading" refers to, for example, loading excavated materials onto the bed or hopper of a transport vehicle. "raising" refers to piling up excavated materials. "Leveling" refers to leveling the work site. "Sorting" refers to, for example, separating materials to be excavated from materials that will not be excavated. "Moving" refers to traveling a predetermined distance or more. "Stopping" refers to work in which the vehicle stops for a predetermined period of time or more. "Loading-related work" refers to, for example, loading work in which a single scooping operation is usually repeated two or more times. "Sorting-related work" refers to, for example, sorting work in which more sorting operations than usual are performed. "Other" refers to work that does not fall into the above categories.

[0040] For example, if the transition of the combination of the travel device behavior and the work machine behavior in five consecutive sections is as shown in FIG. 24 , the task classification unit 36 ​​classifies the five sections as loading work sections. That is, the task classification unit 36 ​​predetermines the combination of the travel device behavior and the work machine behavior in one or more sections corresponding to each type of work, or the transition of that combination. If the combination of the travel device behavior and the work machine behavior in one or more sections in the time-series data 24, or the transition of that combination, matches, the task classification unit 36 ​​identifies the work in that section. For example, if there is a typical transition of the combination of the travel device behavior and the work machine behavior in each section to be classified, the task classification unit 36 ​​preferentially selects that section to determine the type. Next, for sections to which no work has been classified, the task classification unit 36 ​​searches for a section that matches the next-typical combination of the travel device behavior and the work machine behavior in one or more sections, different from the first classified work, or the transition of that combination, and if there is a match, determines the type. The task classification unit 36 ​​classifies the work in all target sections by, for example, repeating the above process. As described above, by giving priority to identifying work types that are easy to identify, highly accurate identification is possible. Figure 25 is a schematic diagram for explaining an example of the work classification for sections (k) to (k+4) shown in Figures 7 and 8. The work classification unit 36 ​​determines, for example, that the transition of the combination of the traveling device behavior discrimination flag and the work machine behavior discrimination flag for each section divided according to the classification results of the traveling device behavior matches the transition shown in Figure 24, and classifies the work in that section (k) to (k+4) as loading work.

[0041] 26 is a diagram illustrating an example of a classification result according to an embodiment of the present disclosure. For example, as shown in FIG. 26 , the first operating state classification unit 32, the second operating state classification unit 34, and the task classification unit 36 ​​generate time-series data 24R in which the values ​​of the traveling device behavior determination flag, the work machine behavior determination flag, and the task classification flag are set for each piece of time-series data (each sampling), and store the data in the storage unit 38. Note that the classification result is not limited to the example shown in FIG. 26 . For example, the classification result may be represented by recording the values ​​of the traveling device behavior determination flag, the work machine behavior determination flag, and the task classification flag in association with information such as a timestamp of the time-series data 24 that identifies each section.

[0042] The detailed analysis unit 37 statistically analyzes the time-series data 24 for each interval based on, for example, the operator's operations. FIGS. 27 to 31 are diagrams showing examples of the results of detailed analysis by the detailed analysis unit 37. FIG. 27 is a pie chart showing the proportion of work performed for a specific period, a specific work machine, and a specific operator. FIG. 28 is a histogram showing the frequency of boom angles during unloading during loading work for a specific period, a specific work machine, and a specific operator. The detailed analysis unit 37 calculates the frequency of boom angles for each interval where the work classification result is "loading work" and the work machine behavior discrimination flag shown in FIG. 17 is classified as "50," "51," or "52" (unloading). FIG. 29 is a scatter diagram showing the relationship between fuel efficiency [L / h] and work volume [ton / h] during loading work for a specific period, a specific work machine, and a specific operator. The scatter diagram in FIG. 29 includes a regression line. Figure 30 is a histogram showing the frequency of cycle times in loading operations for a specific period, a specific work machine, and a specific operator. Figure 31 is a histogram showing the frequency of cycle times in loading operations for a specific period, a specific work machine, and two different operators, allowing comparison between the operators.

[0043] The output unit 39 outputs the analysis results of the detailed analysis unit 37 to a display device or the like, for example.

[0044] (Example of Operation of Analysis Device) FIG. 32 is a flowchart illustrating an example of operation of the analysis device according to an embodiment of the present disclosure. When an operator specifies an analysis target and instructs the analysis device 30 to execute an analysis, the acquisition unit 31 first acquires a file of the time-series data 24 (step S1). Next, the first operating state classification unit 32 classifies the behavior of the traveling gear for the entire target time of the time-series data 24 based on the vehicle speed and gear position (step S2). Next, the division unit 33 divides the time axis of the time-series data 24 into multiple sections based on the classification results of the traveling gear behavior (step S3). Next, the second operating state classification unit 34 calculates multiple feature amounts for each section based on the boom angle and bucket angle (step S4). Next, the second operating state classification unit 34 classifies the behavior of the work machine for each section using the trained machine learning model 35 (step S5). To train a machine learning model, training data is prepared, and machine learning is performed, for example, in the processing of step S5, to generate a trained machine learning model 35, and then classification processing is performed. Alternatively, the trained machine learning model 35 may be generated in advance, for example, before starting the processing shown in Figure 32.

[0045] Next, the task classification unit 36 ​​classifies the tasks into one or more sections based on the classification results of the traveling device behavior and the work machine behavior (step S6). Next, the detail analysis unit 37 statistically analyzes the time-series data, for example, by section, and the output unit 39 outputs the analysis results (step S7).

[0046] (Actions and Effects) According to the analysis device, analysis method, program, trained machine learning model, and machine learning method disclosed herein, the time-series data 24 is divided into a plurality of sections based on the classification results of the traveling unit 4 (first movable unit), and the operating state of the work unit 10 (second movable unit) is classified for each section, so it is possible to efficiently classify the operating state of the work unit 10 (second movable unit). Therefore, it is possible to efficiently classify the work of the wheel loader 1 (work machine) having the traveling unit 4 (first movable unit) and the work unit 10 (second movable unit) based on the time-series data 24.

[0047] Although the embodiments of the present invention have been described above with reference to the drawings, the specific configuration is not limited to the above-described embodiments, and design modifications and the like are also included within the scope of the gist of the present invention. Furthermore, part or all of the programs executed by the computer in the above-described embodiments can be distributed via computer-readable recording media or communication lines.

[0048] (Additional Note) The analysis device, analysis method, program, trained machine learning model, and machine learning method according to the present disclosure may be understood, for example, as follows.

[0049] (Supplementary Note 1) An analysis device comprising: an acquisition unit that acquires time series data representing at least an operating state of a first movable part and an operating state of a second movable part of a work machine having the first movable part; a first operating state classification unit that classifies the operating state of the first movable part based on the time series data; a division unit that divides a time axis of the time series data into a plurality of sections based on the classification results of the operating state of the first movable part; a second operating state classification unit that classifies the operating state of the second movable part for each section based on the time series data; and a work classification unit that classifies work performed by the work machine based on the classification results of the operating state of the first movable part and the classification results of the operating state of the second movable part.

[0050] (Supplementary Note 2) The analysis device according to Supplementary Note 1, wherein the task classification unit classifies the tasks by using one or more of the sections as units.

[0051] (Supplementary Note 3) The second operating state classification unit is a machine learning model that inputs input information including a plurality of predetermined features related to the operating state of the second moving part for each section and outputs output information representing the result of classifying the operating state of the second moving part, and classifies the operating state of the second moving part using a trained machine learning model that has been trained by machine learning using the input information labeled with the type of operating state of the second moving part. This is the analysis device described in Supplementary Note 1 or Supplementary Note 2.

[0052] (Supplementary Note 4) The analysis device according to any one of Supplementary Notes 1 to 3, wherein the first movable part is a traveling part, and the second movable part is a working machine.

[0053] (Supplementary Note 5) The analysis device according to any one of Supplementary Notes 1 to 4, further comprising a detailed analysis unit that statistically analyzes the time-series data in units of the intervals.

[0054] (Supplementary Note 6) An analysis method including the steps of: acquiring time series data representing at least an operating state of a first movable part and an operating state of a second movable part of a work machine having the first movable part; classifying the operating state of the first movable part based on the time series data; dividing a time axis of the time series data into a plurality of sections based on the classification results of the operating state of the first movable part; classifying the operating state of the second movable part for each section based on the time series data; and classifying work performed by the work machine based on the classification results of the operating state of the first movable part and the classification results of the operating state of the second movable part.

[0055] (Supplementary Note 7) A program causing a computer to execute the following steps: acquiring time series data representing at least the operating state of a first movable part and the operating state of a second movable part of a work machine having the first movable part and the second movable part; classifying the operating state of the first movable part based on the time series data; dividing the time axis of the time series data into a plurality of sections based on the classification results of the operating state of the first movable part; classifying the operating state of the second movable part for each section based on the time series data; and classifying work performed by the work machine based on the classification results of the operating state of the first movable part and the classification results of the operating state of the second movable part.

[0056] (Supplementary Note 8) A trained machine learning model that inputs predetermined input information based on time series data representing at least the operating state of a first movable part and the operating state of a second movable part of a work machine having the first movable part and the second movable part, and outputs output information representing the results of classifying the operating state of the second movable part, wherein the input information includes a plurality of predetermined feature quantities related to the operating state of the second movable part for each of a plurality of sections obtained by dividing a time axis of the time series data into a plurality of sections based on the classification results of the operating state of the first movable part, and the trained machine learning model is performed using the input information labeled with the type of operating state of the second movable part.

[0057] (Supplementary Note 9) The trained machine learning model described in Supplementary Note 8, wherein the second movable part is a work machine having a plurality of partial movable parts, and the plurality of feature quantities include one or more values ​​that represent the movable amounts of the plurality of partial movable parts that correspond to each other on a time axis as coordinate values ​​on a plurality of different coordinate axes.

[0058] (Supplementary Note 10) The trained machine learning model according to Supplementary Note 9, wherein the one or more values ​​represented by each coordinate value include values ​​corresponding to an initial value, a maximum value, a minimum value, an average value, or a final value of each coordinate value for each of the divisions.

[0059] (Supplementary Note 11) The trained machine learning model according to any one of Supplementary Notes 8 to 10, wherein the plurality of feature amounts include a value representing the number of repeated movements of the second movable part that occurred for each of the sections.

[0060] (Supplementary Note 12) A machine learning method for a machine learning model that inputs predetermined input information based on time series data representing at least the operating state of a first movable part and the operating state of a second movable part of a work machine having the first movable part and the second movable part, and outputs output information representing a result of classifying the operating state of the second movable part, wherein the input information includes a plurality of predetermined feature quantities related to the operating state of the second movable part for each of a plurality of sections obtained by dividing a time axis of the time series data into a plurality of sections based on the classification result of the operating state of the first movable part, and the machine learning method machine-learns the machine learning model using the input information labeled with the type of the operating state of the second movable part.

[0061] According to each aspect of the present disclosure, the operating state of the second movable part is classified for each of a plurality of sections obtained by dividing time-series data based on the classification result of the first movable part, so that the operating state of the second movable part can be efficiently classified. Therefore, the work of a work machine having a first movable part and a second movable part can be efficiently classified based on the time-series data.

[0062] 1 Wheel loader, 24 Time series data, 30 Analysis device, 31 Acquisition unit, 32 First operating state classification unit, 33 Segmentation unit, 34 Second operating state classification unit, 35 Trained machine learning model, 36 Work classification unit, 37 Detailed analysis unit

Claims

1. An analysis device comprising: an acquisition unit that acquires time series data representing at least an operating state of a first movable part and an operating state of a second movable part of a work machine having a first movable part and a second movable part; a first operating state classification unit that classifies the operating state of the first movable part based on the time series data; a division unit that divides a time axis of the time series data into a plurality of sections based on the classification results of the operating state of the first movable part; a second operating state classification unit that classifies the operating state of the second movable part for each of the sections based on the time series data; and a work classification unit that classifies work performed by the work machine based on the classification results of the operating state of the first movable part and the classification results of the operating state of the second movable part.

2. The analysis device according to claim 1, wherein the task classification unit classifies the tasks into units of one or more of the sections.

3. The analysis device described in claim 2, wherein the second operating condition classification unit is a machine learning model that inputs input information including a plurality of predetermined features related to the operating condition of the second moving part for each section and outputs output information representing the result of classifying the operating condition of the second moving part, and classifies the operating condition of the second moving part using a trained machine learning model that has been machine-trained using the input information labeled with the type of the operating condition of the second moving part.

4. The analysis device according to claim 3, wherein the first movable part is a traveling part, and the second movable part is a working machine.

5. The analysis device according to claim 4, further comprising a detailed analysis unit that statistically analyzes the time series data by using the interval as a unit.

6. An analysis method comprising the steps of: acquiring time series data representing at least an operating state of a first movable part and an operating state of a second movable part of a work machine having a first movable part and a second movable part; classifying the operating state of the first movable part based on the time series data; dividing a time axis of the time series data into a plurality of sections based on the classification results of the operating state of the first movable part; classifying the operating state of the second movable part for each of the sections based on the time series data; and classifying work performed by the work machine based on the classification results of the operating state of the first movable part and the classification results of the operating state of the second movable part.

7. A program causing a computer to execute the steps of: acquiring time series data representing at least an operating state of a first movable part and an operating state of a second movable part of a work machine having a first movable part and a second movable part; classifying the operating state of the first movable part based on the time series data; dividing the time axis of the time series data into a plurality of sections based on the classification results of the operating state of the first movable part; classifying the operating state of the second movable part for each of the sections based on the time series data; and classifying work performed by the work machine based on the classification results of the operating state of the first movable part and the classification results of the operating state of the second movable part.

8. A trained machine learning model that inputs specified input information based on time series data representing at least an operating state of a first movable part and an operating state of a second movable part of a work machine having the first movable part and a second movable part, and outputs output information representing a result of classifying the operating state of the second movable part, wherein the input information includes a plurality of specified feature amounts related to the operating state of the second movable part for each of a plurality of intervals obtained by dividing a time axis of the time series data into a plurality of intervals based on the classification result of the operating state of the first movable part, and the trained machine learning model is trained by machine learning using the input information labeled with the type of the operating state of the second movable part.

9. The trained machine learning model described in claim 8, wherein the second movable part is a work machine having a plurality of partial movable parts, and the plurality of feature quantities include one or more values ​​that represent the respective movable amounts of the plurality of partial movable parts that correspond to each other on a time axis as respective coordinate values ​​on a plurality of different coordinate axes.

10. The trained machine learning model of claim 9, wherein the one or more values ​​represented by each coordinate value include values ​​corresponding to the initial value, maximum value, minimum value, average value, or final value of each coordinate value for each of the sections.

11. The trained machine learning model described in claim 10, wherein the plurality of features include a value representing the number of repeated movements of the second movable part that occurred for each of the sections.

12. A machine learning method for a machine learning model that inputs specified input information based on time series data representing at least the operating state of a first movable part and the operating state of the second movable part of a work machine having a first movable part and a second movable part, and outputs output information representing a result of classifying the operating state of the second movable part, wherein the input information includes a plurality of specified feature amounts related to the operating state of the second movable part for each of a plurality of intervals obtained by dividing a time axis of the time series data into a plurality of intervals based on the classification result of the operating state of the first movable part, and the machine learning method machine-learns the machine learning model using the input information labeled with the type of the operating state of the second moving part.

Images