Image Evaluation Device

Abstract

To properly and easily evaluate whether or not a captured image obtained by capturing an abrasion part of a work machine is an image properly suited for measurement of an abrasion amount to thereby achieve a simplified and efficient measurement work of the abrasion amount.SOLUTION: An image evaluation apparatus 40 according to the present invention evaluates whether or not a captured image 461 of a specific part of a work machine 6 is suitable for a measurement image for use in measurement of an abrasion amount. A feature point extracting unit 43 has carried out machine learning of a learning image including a contour of the specific part and a plurality of feature points for use in specifying the contour of the specific part included in the learning image as teacher data. A reference value storing unit 45 stores a reference value X used as a reference for evaluation of the captured image 461. An evaluation value calculating unit 47, based on the feature points of the captured image 461 and the reference value X thereof, calculates an evaluation value for use in evaluating whether or not the captured image 461 is suitable for the measurement image. An image determining unit 48 determines, in response to the calculated evaluation value, whether or not the captured image 461 can be used as the measurement image.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an image evaluation device. [Background technology]

[0002] Continuous operation of a work machine may cause wear in the components of the work machine. In order to properly manage the wear state of the parts that wear, the amount of wear of the parts may be measured. One method for measuring the amount of wear is to take an image of the part that wears using an imaging device such as a camera, identify the outline of the part from the image, and compare it with the outline of the part before wear, thereby measuring the amount of wear of the part.

[0003] In the above measurement method, it is important that the captured image of the worn part is an image suitable for measuring the amount of wear. For example, in a crawler-type hydraulic excavator, the sprocket of the undercarriage may wear out. When measuring the amount of wear of a sprocket, it is difficult to ensure the measurement accuracy of the amount of wear unless the image is taken from directly in front of the sprocket by facing the imaging device directly in front of the sprocket.

[0004] As a technique for acquiring an image captured from a specific direction such as a front image, for example, there is a technique disclosed in Patent Document 1. Patent Document 1 discloses an image processing device including an imaging means for capturing an image of a subject plane, a plane measurement means for measuring the orientation of the subject plane with respect to the imaging means, and an image distortion correction means for correcting distortion caused by oblique shooting of the image captured by the imaging means based on the measured orientation of the subject plane. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2000-307947 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, when the technology of Patent Document 1 is used to measure the amount of wear, it is necessary to prepare a special means called a planar measuring means before imaging and to use the planar measuring means to image the part that wears out during imaging. In this case, since the special means called the planar measuring means must be distributed to maintenance personnel and users of the work machine, it is not realistic to use the technology of Patent Document 1 to measure the amount of wear. In particular, even if the technology of Patent Document 1 is used, it is difficult to casually image the part that wears out using a smartphone or the like and measure the amount of wear of the part.

[0007] Furthermore, the technology of Patent Document 1 requires the use of a special means called a planar measurement means when capturing images of the wearing area, and therefore the technology of Patent Document 1 cannot be applied to images that have already been captured without using this means.

[0008] For these reasons, there is a demand for technology that can appropriately and easily evaluate whether an image captured of a part of a work machine that is subject to wear is suitable for measuring the amount of wear.

[0009] In view of the above circumstances, the present invention aims to simplify and streamline the work of measuring the amount of wear by appropriately and easily evaluating whether an image of a part of a work machine that is subject to wear is suitable for measuring the amount of wear. [Means for solving the problem]

[0010] In order to solve the above problem, the image evaluation device of the present invention is an image evaluation device that evaluates whether a captured image of a specific part of a work machine is suitable as a measurement image used to measure the amount of wear of the specific part, and is characterized in that it comprises: a learning image including a contour of the specific part and a plurality of feature points for identifying the contour of the specific part included in the learning image, which are machine-learned as teacher data; a feature point extraction unit that extracts feature points that identify the contour of the specific part included in the captured image; a reference value memory unit that stores a reference value that serves as a reference for evaluating the captured image, which is calculated from the feature points in the learning image captured from a direction suitable for measuring the amount of wear; an evaluation value calculation unit that calculates an evaluation value for evaluating whether the captured image is suitable as the measurement image based on the feature points in the captured image extracted by the feature point extraction unit and the reference value stored in the reference value memory unit; and an image judgment unit that judges whether the captured image can be used as the measurement image according to the evaluation value calculated by the evaluation value calculation unit. Effect of the Invention

[0011] According to the present invention, it is possible to appropriately and easily evaluate whether a captured image of a part of a work machine that is subject to wear is suitable for measuring the amount of wear, thereby simplifying and streamlining the work of measuring the amount of wear. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of a wear amount measuring system including an image evaluation device according to a first embodiment. [Diagram 2] FIG. 2 is a diagram showing a configuration of the work machine shown in FIG. 1. [Diagram 3] FIG. 2 is a block diagram showing a functional configuration of the image evaluation device shown in FIG. [Figure 4] 4 is a block diagram showing a detailed configuration of a reference value calculation unit and an evaluation value calculation unit shown in FIG. 3. [Diagram 5] FIG. 13 is a schematic diagram showing a front image suitable for measuring the amount of wear of a sprocket. [Figure 6]FIG. 4 is a diagram illustrating characteristic points that identify the contour of a sprocket. [Figure 7] 5 is a flowchart showing the operation of the image evaluation device in a reference value learning mode. [Figure 8] 5 is a flowchart showing the operation of the image evaluation device in an image extraction mode. [Figure 9] FIG. 11 is a block diagram showing the detailed configuration of a reference value calculation unit and an evaluation value calculation unit included in the image evaluation device of the second embodiment. [Figure 10] FIG. 4 is a diagram showing a schematic configuration of a tip portion of a bucket. [Figure 11] FIG. 13 is a schematic diagram showing an oblique image suitable for measuring the amount of wear of teeth. [Figure 12] FIG. 13 is a diagram illustrating characteristic points that identify the contours of the tooth and adapter. [Figure 13] FIG. 11 is a block diagram showing the functional configuration of an image evaluation device according to a third embodiment. [Figure 14] 5 is a flowchart showing the operation of the image evaluation device in an automatic imaging mode. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Components with the same reference numerals in each embodiment have the same functions in each embodiment unless otherwise specified, and description thereof will be omitted.

[0014] Using current AI technology, it is possible to automatically measure the amount of wear from a captured image 461 including parts of the work machine 6 that are subject to wear, such as the sprocket 63 (see FIG. 2) and the teeth 693 (see FIG. 10). However, high accuracy in the order of millimeters may be required for measuring the amount of wear. To ensure highly accurate measurement, the images used to measure the amount of wear (hereinafter also referred to as "measurement images") differ for each part of the work machine 6.

[0015] For example, the measurement image of the sprocket 63 is a front image captured by facing the imaging device 31 to the front 633 of the sprocket 63 (see FIG. 5). The sprocket 63 has a measurement gauge image that fits the teeth 631 and tooth roots 632 (see FIG. 5) of the sprocket 63 without any gaps before wear. The amount of wear of the sprocket 63 can be easily measured by measuring the size of the gap between the teeth 631 and tooth roots 632 included in the captured image 461 of the sprocket 63 and the measurement gauge image by image processing. In order to measure the amount of wear of the sprocket 63 with high accuracy from the two-dimensional captured image 461 and the measurement gauge image, it is important to fit the measurement gauge image to the front image that hardly shows any depth. Therefore, the measurement image of the sprocket 63 is a front image captured by facing the imaging device 31 to the front 633 of the sprocket 63.

[0016] On the other hand, the measurement image of the tooth 693 is an image (hereinafter also referred to as an "oblique image") captured by directing the image capture device 31 at a moderate angle with respect to the front 694 of the tooth 693 so that the front 694 and side 695 of the adapter 692 and the tooth 693 described later are captured (see FIG. 11). Since the tooth 693 does not have a plane that serves as a reference for the front, it is difficult to evaluate how close the captured image 461 of the tooth 693 is to a front image. Instead, information on the side 695 of the tooth 693 should be used. This is because, if the boundary 696 between the tooth 693 and the adapter 692 is used as the measurement reference, even if the bucket 69 is rotating, there is no problem in measuring the amount of wear. Through repeated experiments, the inventors have succeeded in proving that by using the boundary 696 as the measurement reference, the difference between the measured amount of wear from an image and the actual measured value can be made small. Therefore, the measurement image of tooth 693 is an oblique image captured from a moderately oblique direction with respect to front surface 694 of tooth 693 so that front surface 694 and side surface 695 of adapter 692 and tooth 693 are captured.

[0017] The image evaluation device 40 is a device that evaluates whether a captured image 461 including a worn part of the work machine 6 is suitable as a measurement image. The image evaluation device 40 performs processing appropriate for each part included in the captured image 461 to be evaluated, and can evaluate the captured images 461 of various parts.

[0018] [Embodiment 1] An image evaluation device 40 according to the first embodiment will be described with reference to FIGS.

[0019] In the first embodiment, when a part of a work machine 6 that wears is a sprocket 63, an image evaluation device 40 that evaluates whether a captured image 461 of the sprocket 63 is suitable as a measurement image will be described.

[0020] FIG. 1 is a diagram showing a schematic configuration of a wear amount measuring system 1 including an image evaluation device 40 according to the first embodiment.

[0021] The wear amount measurement system 1 is a system that measures the amount of wear of a part of a work machine 6 from an image of the part that wears. The wear amount measurement system 1 is a system in which a server device 2 that manages the wear amount measurement system 1 and a plurality of communication terminals 3 each equipped with an imaging device 31 are connected via a network so as to be able to wirelessly communicate with each other. In the wear amount measurement system 1 of this embodiment, the communication terminal 3 is provided with an image evaluation device 40, but all or part of the functions of the image evaluation device 40 may be provided in the server device 2.

[0022] The communication terminal 3 is a mobile communication terminal such as a smartphone, a tablet PC, etc. The communication terminal 3 includes an imaging device 31 capable of capturing moving images or still images, a display device 32 and an input device 33 constituting a touch panel display, and an image evaluation device 40.

[0023] The image evaluation device 40 includes an arithmetic processing device 401 including a CPU and the like, and a storage device 402 including a ROM and a RAM. The storage device 402 includes data storage areas including a teacher data storage unit 41, a reference value storage unit 45, and a captured image storage unit 46, which will be described later. The storage device 402 includes a program storage area for storing various programs for implementing functions of a learning unit 42, a feature point extraction unit 43, a reference value calculation unit 44, an evaluation value calculation unit 47, an image determination unit 48, and an image extraction unit 49, which will be described later. The arithmetic processing device 401 can implement various functions of the image evaluation device 40 by executing various programs stored in the program storage area of ​​the storage device 402 using various data stored in the data storage area of ​​the storage device 402.

[0024] FIG. 2 is a diagram showing the configuration of the work machine 6 shown in FIG.

[0025] The work machine 6 is not particularly limited, but may be, for example, a construction machine such as a hydraulic excavator. In this embodiment, the work machine 6 will be described as being a crawler-type hydraulic excavator.

[0026] The work machine 6 comprises a lower running body 61, an upper rotating body 65 rotatably attached to the lower running body 61 via a rotating device 651, and a multi-jointed front work machine 66 attached to the front of the upper rotating body 65.

[0027] The lower traveling body 61 includes an idler 62 provided at the front end of the lower traveling body 61, a sprocket 63 provided at the rear end of the lower traveling body 61 and driven by a hydraulic motor, and a belt-shaped crawler 64 stretched over the idler 62 and the sprocket 63. The sprocket 63 may wear out due to friction with the crawler 64 or friction with soil or sand that has entered between the sprocket 63 and the crawler 64.

[0028] The upper rotating body 65 includes a cab 652 provided at the front of the upper rotating body 65, and a machine room 653 provided at the rear of the upper rotating body 65 for accommodating a drive device of the work machine 6.

[0029] The front work implement 66 includes a boom 67 rotatably supported on the front part of the upper rotating body 65, an arm 68 rotatably supported on the tip of the boom 67, and a bucket 69 rotatably supported on the tip of the arm 68. The boom 67, arm 68, and bucket 69 are connected to a boom cylinder 671, an arm cylinder 681, and a bucket cylinder 691 via links, respectively, and rotate when these hydraulic cylinders extend and retract. Teeth 693 are attached to the tip 69a of the bucket 69 at intervals in the width direction. The teeth 693 may become worn due to friction with the soil and sand being excavated.

[0030] Fig. 3 is a block diagram showing the functional configuration of the image evaluation device 40 shown in Fig. 1. Fig. 4 is a block diagram showing the detailed configuration of the reference value calculation unit 44 and the evaluation value calculation unit 47 shown in Fig. 3. Fig. 5 is a schematic diagram showing a front image G1 suitable for measuring the amount of wear of the sprocket 63. Fig. 6 is a diagram explaining feature points that specify the contour of the sprocket 63.

[0031] The image evaluation device 40 extracts feature points that identify the contour of a part that wears, such as a sprocket 63, from a captured image 461 that includes the part, and calculates an evaluation value that evaluates how suitable the captured image 461 is as a measurement image. The image evaluation device 40 determines whether the captured image 461 can be used as a measurement image according to the calculated evaluation value. The feature point extraction function is constructed as a model generated by machine learning. The captured image 461 to be evaluated has already been captured and stored in advance in the captured image storage unit 46. The image evaluation device 40 can evaluate whether a plurality of captured images 461 stored in the captured image storage unit 46 are suitable as a measurement image, and extract a measurement image from the plurality of captured images 461.

[0032] The image evaluation device 40 includes a teacher data memory unit 41, a learning unit 42, a feature point extraction unit 43, a reference value calculation unit 44, a reference value memory unit 45, a captured image memory unit 46, an evaluation value calculation unit 47, an image judgment unit 48, and an image extraction unit 49.

[0033] The functions of the image evaluation device 40 can be broadly divided into functions used or executed in a learning stage in which machine learning is performed to generate a feature point extraction function and a reference value X described later is calculated, and functions used or executed in a utilization stage in which it is evaluated whether the captured image 461 is suitable as a measurement image.

[0034] First, the functions used or executed in the learning stage of the image evaluation device 40 will be described. The teacher data memory unit 41 stores a learning image 411 showing the contour of a part of the work machine 6 that is subject to wear, and feature point data 412 indicating the coordinates of a plurality of feature points contained in the learning image 411 for identifying the contour of that part, in association with each other.

[0035] The learning unit 42 performs machine learning to extract feature points using the learning image 411 and feature point data 412 stored in the teacher data storage unit 41 as teacher data, and generates the feature point extraction unit 43. The machine learning algorithm possessed by the learning unit 42 is not particularly limited and may be, for example, a random forest. When evaluating whether a captured image 461 stored in the captured image storage unit 46 is suitable as a measurement image, the machine learning-prepared feature point extraction unit 43 extracts feature points that identify the contour of a part included in the captured image 461.

[0036] The learning image 411 is an image including the contours of the front surface 633 and the back surface 634 of the tooth bottom 632 that forms the outer edge of the sprocket 63. The learning image 411 is acquired by capturing images of the teeth 631 and the tooth bottom 632 of the sprocket 63 from a plurality of imaging directions including a direction suitable for measuring the amount of wear. The learning image 411 may be an image captured by the imaging device 31 of the communication terminal 3, or may be an image captured by another imaging device different from the imaging device 31. The direction suitable for measuring the amount of wear is, for example, an imaging direction in which the optical axis of the imaging device 31 is perpendicular to a plane including the front surface 633 of the sprocket 63. The image captured from the direction suitable for measuring the amount of wear is a true-front image G1 as shown in FIG. 5. The image captured from the imaging direction other than the direction suitable for measuring the amount of wear is an image G2 as shown in FIG. 6. The image G2 shown in FIG. 6 is an image including the contours of the front surface 633 and the back surface 634 of the tooth bottom 632 of the sprocket 63.

[0037] The front surface 633 is a surface facing outward in the left-right direction of the work machine 6, the back surface 634 is a surface facing inward in the left-right direction of the work machine 6, and the side surface 635 is a surface that is continuous with the front surface 633 and the back surface 634 and runs along the circumferential direction of the sprocket 63.

[0038] As shown in FIG. 6, in an image including the contours of the tooth bottoms 632 of the sprocket 63, a front circle F approximating the contour of the front surface 633 of the tooth bottoms 632 and a back circle B approximating the contour of the back surface 634 of the tooth bottoms 632 can be drawn. In an image including the contours of the tooth bottoms 632 of the sprocket 63, the center Cf of the front circle F and the center Cb of the back circle B coinciding is almost equivalent to the image being a front image G1. In an image in which the contour of the back surface 634 is completely hidden, the center Cb of the back surface 634 is unknown. When an image including the contours of the tooth bottoms 632 of the sprocket 63 is acquired, if the posture of the imaging device is changed so that the center Cf of the front circle F and the center Cb of the back circle B overlap from a state in which the center Cf of the front circle F and the center Cb of the back circle B are captured, the two will approach each other. Eventually, the center Cf of the front circle F and the center Cb of the back circle B will coincide and the center Cb of the back circle B will disappear. The image captured at the moment the center Cb of the rear circle B disappears can be regarded as a front image G1.

[0039] Due to the principle of perspective, the radius Rb of the back circle B is smaller than the radius Rf of the front circle F. Therefore, it is impossible to overlap the contour of the front surface 633 of the bottom 632 with the contour of the back surface 634. In order to obtain a true front image G1 without using the technique of drawing the front circle F and the back circle B, the contour of the front surface 633 of the bottom 632 and the contour of the back surface 634 must be viewed simultaneously in parallel and captured intuitively while considering the balance of the deviation between the two. In this way, it is very difficult to obtain a true front image G1. However, by adopting the technique of drawing the front circle F and the back circle B, a more accurate true front image G1 can be obtained by a simple operation of simply overlapping the center Cf of the front circle F with the center Cb of the back circle B as described above.

[0040] For this reason, image evaluation device 40 of this embodiment sets the coordinates of points on each contour of tooth bottom 632 of sprocket 63 at which front circle F and back circle B can be drawn as feature point data 412. P1 to P4 in Figure 6 are feature points that specify the contour of the front side 633 of tooth bottom 632 of sprocket 63. P5 to P8 in Figure 6 are feature points that specify the contour of the back side 634 of tooth bottom 632 of sprocket 63.

[0041] The reference value calculation unit 44 calculates a reference value X that serves as a reference for evaluating whether the captured image 461 stored in the captured image storage unit 46 is suitable as a measurement image. As shown in FIG. 4, the reference value calculation unit 44 includes a center radius calculation unit 441 and a ratio calculation unit 442.

[0042] As described above, in an image including the contours of the tooth bottom 632 of the sprocket 63, the center Cf of the front circle F and the center Cb of the back circle B coincide with each other, which is almost equivalent to the image being a front image G1. However, since the radius Rf of the front circle F and the radius Rb of the back circle B are different, the points to be coincident are, to be precise, points closer to the tooth bottom 632 than the center Cf and points closer to the tooth bottom 632 than the center Cb. These points to be coincident are reference points (hereinafter also referred to as "reference points") for evaluating whether the captured image 461 to be evaluated is suitable for a measurement image. The point to be set on the reference point of the front circle F and the reference point of the back circle B to a point away from the center Cf and the center Cb on the tooth bottom 632 side by a distance calculated by multiplying the radius Rf and the radius Rb, respectively, depends on the width and thickness of the tooth bottom 632, and therefore depends on the type of the sprocket 63. In the image evaluation device 40 of this embodiment, before evaluating the captured image 461, the reference value calculation unit 44 calculates this ratio as a reference value X and stores it in the reference value storage unit 45 for each type of sprocket 63. In an image including each contour of the tooth bottom 632 of the sprocket 63, the fact that the reference point of the front circle F and the reference point of the back circle B coincide is completely equivalent to the image being a true front image G1.

[0043] For this reason, the image evaluation device 40 of this embodiment sets a point closer to the tooth bottom 632 than the center Cf of the front circle F as the reference point of the front circle F, and sets a point closer to the tooth bottom 632 than the center Cb of the back circle B as the reference point of the back circle B. Then, the image evaluation device 40 of this embodiment sets an image in which the reference point of the front circle F and the reference point of the back circle B coincide as a true front image G1.

[0044] When calculating the reference value X, the reference value calculation unit 44 uses feature points in the learning image 411 captured from a direction suitable for measuring the amount of wear. The learning image 411 used to calculate the reference value X may be, for example, the learning image 411 stored in the teacher data storage unit 41 or the learning image 411 newly captured by the imaging device 31 from a direction suitable for measuring the amount of wear in order to calculate the reference value X. The reference value calculation unit 44 inputs these learning images 411 to the feature point extraction unit 43 that has undergone machine learning. The feature point extraction unit 43 extracts feature points that specify the contour of the front surface 633 of the tooth bottom 632 and feature points that specify the contour of the back surface 634. The feature point extraction unit 43 specifies a front circle F and a back circle B from the extracted feature points.

[0045] The center radius calculation unit 441 calculates the center Cf and radius Rf of the front circle F identified by the feature point extraction unit 43, and the center Cb and radius Rb of the back circle B identified by the feature point extraction unit 43. The ratio calculation unit 442 of the reference value calculation unit 44 calculates a straight line that passes through the center Cf of the front circle F and the center Cb of the back circle B. The ratio calculation unit 442 calculates X on the calculated straight line that satisfies the following formula (1). Distance from center Cf of front circle F (radius Rf × X) = distance from center Cb of back circle B (radius Rb × X) (1)

[0046] The reference value calculation unit 44 determines a point on a line passing through the center Cf of the front circle F and the center Cb of the back circle B, which is a point away from the center Cf toward the tooth bottom 632 by a distance obtained by multiplying the radius Rf by X, as a reference point of the front circle F. The reference value calculation unit 44 determines a point on a line passing through the center Cf of the front circle F and the center Cb of the back circle B, which is a point away from the center Cb toward the tooth bottom 632 by a distance obtained by multiplying the radius Rb by X, as a reference point of the back circle B. The reference value calculation unit 44 stores the calculated X in the reference value storage unit 45 as the reference value X. At this time, the reference value storage unit 45 stores the reference value X calculated by the reference value calculation unit 44 in association with a part number that identifies the type of the sprocket 63.

[0047] In this way, the reference value storage unit 45 stores the reference value X, which is calculated from feature points in the learning image 411 captured from a direction suitable for measuring the amount of wear, and serves as a standard for evaluating the captured image 461.

[0048] Next, functions used or executed in the utilization stage of the image evaluation device 40 will be described. The captured image storage unit 46 stores a captured image 461. The captured image 461 stored in the captured image storage unit 46 is a moving image or a plurality of still images captured in advance. A moving image is composed of a plurality of frame images captured at a predetermined frame rate. When evaluating a moving image stored in the captured image storage unit 46, the image evaluation device 40 can evaluate each of the plurality of frame images that constitute the moving image.

[0049] When evaluating whether a captured image 461 stored in the captured image storage unit 46 is suitable as a measurement image, the feature point extraction unit 43 extracts a plurality of feature points for identifying an outline of a specific portion included in the captured image 461. Specifically, the captured image 461 stored in the captured image storage unit 46 is input to the feature point extraction unit 43. The feature point extraction unit 43 extracts feature points that identify the outline of a front surface 633 and feature points that identify the outline of a back surface 634 at the tooth bottom 632 of the sprocket 63 included in the captured image 461. The feature point extraction unit 43 identifies a front circle F and a back circle B from the extracted feature points.

[0050] The evaluation value calculation unit 47 calculates an evaluation value for evaluating whether the captured image 461 is suitable as a measurement image, based on the feature points in the captured image 461 extracted by the feature point extraction unit 43 and the reference value X stored in the reference value storage unit 45. As shown in FIG. 4, the evaluation value calculation unit 47 includes a reference value average calculation unit 471, a central radius calculation unit 472, a difference length calculation unit 473, and a difference ratio calculation unit 474.

[0051] The reference value average calculation unit 471 calculates the average value Xm of multiple reference values ​​X stored in the reference value storage unit 45 in association with the part number of the sprocket 63. The center radius calculation unit 472 calculates the center Cf and radius Rf of the front circle F identified by the feature point extraction unit 43, and the center Cb and radius Rb of the back circle B identified by the feature point extraction unit 43.

[0052] The differential length calculation unit 473 calculates a straight line passing through the center Cf of the front circle F and the center Cb of the back circle B. The differential length calculation unit 473 calculates a differential length D on the calculated straight line, which is expressed by the following equation (2). Differential length D = {point distant from the center Cf of the front circle F by a distance (radius Rf × Xm)} - {point distant from the center Cb of the back circle B by a distance (radius Rb × X)} (2)

[0053] The difference ratio calculation section 474 calculates the difference ratio Δ expressed by the following equation (3). Difference ratio Δ = difference length D / (radius Rf of front circle F + radius Rb of back circle B) (3)

[0054] The evaluation value calculation unit 47 sets the difference ratio Δ calculated by the difference ratio calculation unit 474 as an evaluation value for evaluating whether the captured image 461 is suitable as a measurement image. The evaluation value calculation unit 47 inputs the difference ratio Δ calculated as the evaluation value to the image determination unit 48.

[0055] The image determination unit 48 determines whether the captured image 461 can be used as a measurement image according to the evaluation value calculated by the evaluation value calculation unit 47. Specifically, the image determination unit 48 compares the difference ratio Δ calculated as the evaluation value with a threshold value T. The threshold value T is a threshold value for determining whether the captured image 461 can be used as a measurement image, and is determined in advance for each type of sprocket 63. The threshold value T is stored in advance in a threshold value storage unit (not shown) of the image evaluation device 40 in association with the part number of the sprocket 63. If the difference ratio Δ is smaller than the threshold value T, the image determination unit 48 determines that the captured image 461 can be used as a measurement image. If the difference ratio Δ is equal to or greater than the threshold value T, the image determination unit 48 determines that the captured image 461 cannot be used as a measurement image. The image determination unit 48 inputs the determination result to the image extraction unit 49.

[0056] The image extraction unit 49 extracts the measurement image from the captured images 461 stored in the captured image storage unit 46 according to the judgment result of the image judgment unit 48. Specifically, the image extraction unit 49 extracts the captured image 461 corresponding to the measurement image from the multiple frame images or multiple still images constituting the moving image stored in the captured image storage unit 46 according to the judgment result of the image judgment unit 48. When the multiple captured images 461 correspond to the measurement image in the judgment result of the image judgment unit 48, the image extraction unit 49 may extract all the captured images 461 corresponding to the measurement image, may extract only the captured image 461 with the smallest difference ratio Δ, or may extract only a predetermined number of captured images 461 (for example, the top five captured images 461) in order from the smallest difference ratio Δ. The image extraction unit 49 outputs the captured image 461 corresponding to the extracted measurement image to the display device 32 and causes the display device 32 to display the image.

[0057] After determining whether or not the captured image 461 stored in the captured image storage unit 46 can be used as a measurement image, the image evaluation device 40 may associate the determination result with the captured image 461 and store it in the captured image storage unit 46 or output it to the display device 32. In this case, the image evaluation device 40 may not need to extract the captured image 461 that corresponds to the measurement image from among the captured images 461 stored in the captured image storage unit 46. In this case, the image evaluation device 40 may omit the image extraction unit 49.

[0058] Furthermore, the image evaluation device 40 may only extract the captured image 461 having the smallest difference ratio Δ as the measurement image, without determining whether or not all of the captured images 461 stored in the captured image storage unit 46 can be used as the measurement image. In this case, the image evaluation device 40 may omit the image determination unit 48.

[0059] Next, a description will be given of the operation of the image evaluation device 40. The image evaluation device 40 can operate in a machine learning mode that generates an extraction function of feature points, a learning mode that calculates and stores a reference value X in advance, and an image extraction mode that extracts a measurement image from the captured images 461 stored in the captured image storage unit 46.

[0060] FIG. 7 is a flowchart showing the operation of the image evaluation device 40 in the reference value X learning mode.

[0061] In step S701, the image evaluation device 40 accepts the start of a learning mode for the reference value X in response to an input operation by the user on the input device 33.

[0062] In step S702, the image evaluation device 40 receives the part number of the sprocket 63 in response to an input operation by the user via the input device 33.

[0063] In step S703, the image evaluation device 40 judges whether or not erasure (initialization) of the reference value X has been accepted in response to an input operation by the user to the input device 33. If the image evaluation device 40 has not accepted the erasure of the reference value X, the image evaluation device 40 proceeds to step S705. If the image evaluation device 40 has accepted the erasure of the reference value X, the image evaluation device 40 proceeds to step S704.

[0064] In step S704, the image evaluation device 40 erases the reference value X. Specifically, the image evaluation device 40 erases the reference value X stored in the reference value storage unit 45, or the reference value X for which the error message was output in step S713.

[0065] In step S705, the image evaluation device 40 judges whether or not the end of the learning mode of the reference value X has been accepted in response to an input operation by the user to the input device 33. If the end of the learning mode of the reference value X has been accepted, the image evaluation device 40 ends this process shown in Fig. 7. If the end of the learning mode of the reference value X has not been accepted, the image evaluation device 40 proceeds to step S706.

[0066] In step S706, the image evaluation device 40 reads the learning image 411 used for calculating the reference value X.

[0067] In step S707, image evaluation device 40 extracts feature points specifying the contour of the front surface 633 of the tooth bottom 632 of the sprocket 63 and feature points specifying the contour of the back surface 634. Image evaluation device 40 specifies a front circle F and a back circle B from the extracted feature points.

[0068] In step S708, the image evaluation device 40 calculates the center Cf and radius Rf of the front circle F, and the center Cb and radius Rb of the back circle B.

[0069] In step S709, the image evaluation device 40 calculates the reference value X. Specifically, the image evaluation device 40 calculates the reference value X on a straight line passing through the center Cf of the front circle F and the center Cb of the back circle B, which satisfies the above formula (1).

[0070] In step S710, the image evaluation device 40 judges whether or not the reference value X has already been stored in the reference value storage unit 45. If the reference value X has not already been stored in the reference value storage unit 45, the image evaluation device 40 proceeds to step S713. If the reference value X has already been stored in the reference value storage unit 45, the image evaluation device 40 proceeds to step S711.

[0071] In step S711, the image evaluation device 40 judges whether or not the difference between the currently calculated reference value X and the reference value X already stored in the reference value storage unit 45 is within the allowable range. If the difference between the currently calculated reference value X and the reference value X already stored in the reference value storage unit 45 is within the allowable range, the image evaluation device 40 proceeds to step S713. If the difference between the currently calculated reference value X and the reference value X already stored in the reference value storage unit 45 is not within the allowable range, the image evaluation device 40 proceeds to step S712. Note that whether or not it is within the allowable range is based on a threshold value that is set and stored in advance. The threshold value can be set arbitrarily.

[0072] In step S712, the image evaluation device 40 outputs an error message to the display device 32, and the process proceeds to step S703.

[0073] In step S713, the image evaluation device 40 stores the currently calculated reference value X in the reference value storage unit 45 as a new reference value X.

[0074] In this manner, the image evaluation device 40 can calculate and store in advance the reference value X that serves as a reference for evaluating the captured image 461.

[0075] FIG. 8 is a flowchart showing the operation of the image evaluation device 40 in the image extraction mode.

[0076] In step S801, the image evaluation device 40 accepts the start of an image extraction mode in response to an input operation by the user via the input device 33.

[0077] In step S802, the image evaluation device 40 receives the part number of the sprocket 63 in response to an input operation by the user via the input device 33.

[0078] In step S803, the image evaluation device 40 judges whether or not the reference value X is stored in the reference value storage unit 45. If the reference value X is stored in the reference value storage unit 45, the image evaluation device 40 proceeds to step S805. If the reference value X is not stored in the reference value storage unit 45, the image evaluation device 40 proceeds to step S804.

[0079] In step S804, the image evaluation device 40 outputs a message to the display device 32 to the effect that the reference value X is not stored. The image evaluation device 40 then ends this process shown in FIG.

[0080] In step S805, the image evaluation device 40 calculates the average value Xm of the reference values ​​X stored in the reference value storage unit 45.

[0081] In step S806, the image evaluation device 40 reads the captured image 461 stored in the captured image storage unit .

[0082] In step S807, image evaluation device 40 extracts feature points specifying the contour of the front surface 633 of the tooth bottom 632 of the sprocket 63 and feature points specifying the contour of the back surface 634. Image evaluation device 40 specifies a front circle F and a back circle B from the extracted feature points.

[0083] In step S808, the image evaluation device 40 calculates the center Cf and radius Rf of the front circle F, and the center Cb and radius Rb of the back circle B.

[0084] In step S809, the image evaluation device 40 calculates the differential length D on the straight line passing through the center Cf of the front circle F and the center Cb of the rear circle B, which is expressed by the above formula (2).

[0085] In step S810, the image evaluation device 40 calculates the difference ratio Δ expressed by the above formula (3).

[0086] In step S811, the image evaluation device 40 determines whether the calculated difference ratio Δ is smaller than a predetermined threshold value T, thereby determining whether the currently read captured image 461 can be used as a measurement image.

[0087] In step S812, the image evaluation device 40 judges whether or not the judgment result in step S811 indicates that the image can be used as a measurement image. If the judgment result does not indicate that the image can be used as a measurement image, the image evaluation device 40 proceeds to step S814. If the judgment result indicates that the image can be used as a measurement image, the image evaluation device 40 proceeds to step S813.

[0088] In step S813, the image evaluation device 40 extracts the currently read captured image 461 as a measurement image.

[0089] In step S814, the image evaluation device 40 judges whether or not all the captured images 461 stored in the captured image storage section 46 have been evaluated. If the image evaluation device 40 has not evaluated all the captured images 461 stored in the captured image storage section 46, the image evaluation device 40 proceeds to step S806. If the image evaluation device 40 has evaluated all the captured images 461 stored in the captured image storage section 46, the image evaluation device 40 proceeds to step S815.

[0090] In step S815, the image evaluation device 40 judges whether or not a measurement image is extracted. If a measurement image is extracted, the image evaluation device 40 proceeds to step S817. If a measurement image is not extracted, the image evaluation device 40 proceeds to step S816.

[0091] In step S816, the image evaluation device 40 outputs a message to the effect that no measurement image was extracted to the display device 32. The image evaluation device 40 then ends this process shown in FIG.

[0092] In step S817, the image evaluation device 40 outputs the extracted measurement image to the display device 32. The image evaluation device 40 ends this process shown in FIG.

[0093] As described above, the image evaluation device 40 of the first embodiment is an image evaluation device that evaluates whether the captured image 461 of a specific part (a part that wears, for example, the sprocket 63) of the work machine 6 is suitable as a measurement image used to measure the amount of wear. The feature point extraction unit 43 machine-learns the learning image 411 including the contour of the specific part and the feature point data 412 indicating the coordinates of a plurality of feature points for identifying the contour of the specific part included in the learning image 411 as teacher data, and extracts feature points that identify the contour of the specific part included in the captured image 461. The reference value storage unit 45 stores the reference value X that is calculated from the feature points in the learning image 411 captured from a direction suitable for measuring the amount of wear and serves as a reference for evaluating the captured image 461. The evaluation value calculation unit 47 calculates an evaluation value that evaluates whether the captured image 461 is suitable as a measurement image based on the feature points in the captured image 461 extracted by the feature point extraction unit 43 and the reference value X stored in the reference value storage unit 45. The image determination section 48 determines whether or not the captured image 461 can be used as a measurement image, depending on the evaluation value calculated by the evaluation value calculation section 47.

[0094] As a result, the image evaluation device 40 of the first embodiment can evaluate with high accuracy whether the captured image 461 of a specific part of the work machine 6 is suitable for measuring the amount of wear, without using a special means such as the planar measurement means disclosed in Patent Document 1. Furthermore, the image evaluation device 40 of the first embodiment can evaluate with high accuracy whether the captured image 461 is suitable for measuring the amount of wear, regardless of whether the captured image 461 has already been captured. Therefore, the image evaluation device 40 of the first embodiment can appropriately and easily evaluate whether the captured image 461 is suitable for measuring the amount of wear, even if the captured image 461 is an image of a part that is subject to wear that is casually captured by a user using a smartphone or the like. By introducing the image evaluation device 40, the user can measure the amount of wear from the two-dimensional captured image 461, without using an expensive means such as a three-dimensional shape measuring machine that acquires a three-dimensional point cloud of the target part. The image evaluation device 40 of the first embodiment can simplify and improve the efficiency of the measurement work of the amount of wear.

[0095] Furthermore, the image evaluation device 40 of the first embodiment includes an image extraction unit 49 that extracts a measurement image. The captured image 461 to be evaluated in the first embodiment is a moving image or a plurality of still images captured in advance and stored in the captured image storage unit 46. The image extraction unit 49 extracts a measurement image from the plurality of frame images or the plurality of still images constituting the moving image stored in the captured image storage unit 46 according to the determination result of the image determination unit 48.

[0096] This allows the image evaluation device 40 of the first embodiment to automatically extract images suitable for measuring the amount of wear from a collection of moving images or still images that have already been captured. In particular, the image evaluation device 40 of the first embodiment can automatically extract images suitable for measuring the amount of wear even from a collection of moving images or still images that have been captured without any purpose. Thus, the image evaluation device 40 of the first embodiment can easily obtain images suitable for measuring the amount of wear, further simplifying and improving the efficiency of the work of measuring the amount of wear.

[0097] In the above-described wear amount measurement system 1, the communication terminal 3 has all the functions of the image evaluation device 40, but the server device 2 may have all or part of the functions of the image evaluation device 40. For example, the server device 2 may have the teacher data storage unit 41 and the learning unit 42, and the communication terminal 3 may have the feature point extraction unit 43 that has been trained by machine learning. Alternatively, the server device 2 may also have the feature point extraction unit 43 that has been trained by machine learning, and the communication terminal 3 may receive the extraction result of the feature point extraction unit 43 transmitted from the server device 2. Alternatively, the server device 2 may have the teacher data storage unit 41, the learning unit 42, the feature point extraction unit 43, the reference value calculation unit 44, the reference value storage unit 45, the evaluation value calculation unit 47, the image determination unit 48, and the image extraction unit 49, and the communication terminal 3 may only have the captured image storage unit 46. Alternatively, the server device 2 may also have the captured image storage unit 46, and the communication terminal 3 may receive the extraction result of the image extraction unit 49 transmitted from the server device 2 and output it to the display device 32.

[0098] [Embodiment 2] The image evaluation device 40 of the second embodiment will be described with reference to Figures 9 to 12. In the image evaluation device 40 of the second embodiment, the description of the same configuration and operation as in the first embodiment will be omitted.

[0099] In the second embodiment, when a part of a work machine 6 that wears is a tooth 693, an image evaluation device 40 that evaluates whether a captured image 461 of the tooth 693 is suitable as a measurement image will be described.

[0100] Fig. 9 is a block diagram showing detailed configurations of the reference value calculation unit 44 and the evaluation value calculation unit 47 included in the image evaluation device 40 of the second embodiment. Fig. 10 is a diagram showing a schematic configuration of the tip 69a of the bucket 69. Fig. 11 is a diagram showing a schematic perspective image G3 suitable for measuring the amount of wear of the tooth 693. Fig. 12 is a diagram explaining characteristic points that specify the contours of the tooth 693 and the adapter 692. Note that the dashed lines shown in Figs. 11 and 12 indicate the contour of the tooth 693 before wear.

[0101] As shown in Fig. 10, the teeth 693 are attached to adapters 692 provided at the tip 69a of the bucket 69 at intervals in the width direction. The adapter 692 is a portion that is not easily worn out. In this embodiment, the length and width of the adapter 692 are fixed. A front surface 694 of the adapter 692 and the tooth 693 shown in Fig. 11 is a surface that continues with the inner surface of the bucket 69. A side surface 695 of the adapter 692 and the tooth 693 shown in Fig. 11 is a surface that continues with the back surface that is continuous with the outer surface of the bucket 69 and the front surface 694, and is a surface that runs along the dump / crowd direction of the bucket 69.

[0102] The amount of wear (wear rate) of tooth 693 can be measured as follows. That is, as shown in Fig. 11, the length of worn tooth 693 on the image is Lt, and the length of adapter 692 on the image is La. The actual length of adapter 692 is la (fixed value), and the length of tooth 693 before wear on the image is Lt0. In this case, the actual length lt of worn tooth 693 can be expressed as in the following formula (4). The amount of wear (wear rate) of tooth 693 can be expressed as in the following formula (5). Actual length lt of worn tooth 693 = (length Lt of worn tooth 693 on the image) × (length La of adapter 692 on the image) / (actual length la of adapter 692) (4) Wear amount of tooth 693 (wear rate) = 1 - {(length of worn tooth 693 on the image Lt) / (length of tooth 693 on the image before wear Lt0)} (5)

[0103] However, in an image close to being captured from the front 694 of the tooth 693, the area of ​​the adapter 692 and the side 695 of the tooth 693 is narrow. Therefore, in an image close to being captured from the front 694 of the tooth 693, the length La of the adapter 692 on the image and the lengths Lt and Lt0 of the tooth 693 on the image cannot be accurately measured. On the other hand, in an image captured from an excessively oblique direction with respect to the front 694 of the tooth 693, the adapter 692 and the tooth 693 on the front side of the image overlap with the adapter 692 and the tooth 693 on the back side of the image, and a part of the side 695 of the adapter 692 and the tooth 693 on the back side of the image is not captured. Therefore, in an image captured from an excessively oblique direction with respect to the front 694 of the tooth 693, the length La of the adapter 692 on the image and the lengths Lt and Lt0 of the tooth 693 on the image cannot be accurately measured. Therefore, an image captured from a direction suitable for measuring the amount of wear is an image captured from a direction that allows accurate measurement of the length La on the image of the adapter 692 and the lengths Lt and Lt0 on the image of the tooth 693. The image captured from a direction suitable for measuring the amount of wear is an oblique image G3 captured from a moderately oblique direction with respect to the front 694 of the tooth 693, as shown in FIG.

[0104] The oblique image G3 is an image including the contours of the front 694 and side 695 of the adapter 692 and the contours of the front 694 and side 695 of the tooth 693. In the oblique image G3, the ratio of the width Wf of the front 694 to the width Ws of the side 695 at the boundary 696 between the adapter 692 and the tooth 693 can be defined as a predetermined value (at least within a predetermined range) as a length on the image. In an image including the contours of the front 694 and side 695 of the adapter 692 and the contours of the front 694 and side 695 of the tooth 693, the ratio of the width Wf of the front 694 to the width Ws of the side 695 at the boundary 696 being a predetermined value (at least within a predetermined range) is equivalent to the image being the oblique image G3.

[0105] For this reason, in the image evaluation device 40 of the second embodiment, the coordinates of the vertices that specify the contours of the front surface 694 and the side surface 695 of the adapter 692 and the coordinates of the vertices that specify the contours of the front surface 694 and the side surface 695 of the tooth 693 are set as the feature point data 412. The image G4 shown in FIG. 12 is an image including the contours of the front surface 694 and the side surface 695 of the adapter 692 and the contours of the front surface 694 and the side surface 695 of the tooth 693. P11 to P14 in FIG. 12 are feature points that specify the contour of the front surface 694 of the tooth 693. P12, P14, and P15 in FIG. 12 are feature points that specify the contour of the side surface 695 of the tooth 693. P13, P14, P16, and P17 in FIG. 12 are feature points that specify the contour of the front surface 694 of the adapter 692. P14, P15, P17, and P18 in FIG. 12 are feature points that identify the contour of the side surface 695 of the adapter 692.

[0106] In the image evaluation device 40 of the second embodiment, an image including each contour of the front surface 694 and the side surface 695 of the adapter 692 and each contour of the front surface 694 and the side surface 695 of the tooth 693 is set as a learning image 411. The learning image 411 is acquired by capturing images of the front surface 694 and the side surface 695 of each of the adapter 692 and the tooth 693 from a plurality of imaging directions including a direction suitable for measuring the amount of wear.

[0107] In the image evaluation device 40 of the second embodiment, the ratio of the width Wf of the front surface 694 to the width Ws of the side surface 695 at the boundary 696 between the adapter 692 and the tooth 693 is set as a reference value X.

[0108] Specifically, the reference value calculation unit 44 of the second embodiment inputs the learning image 411 to the machine-learned feature point extraction unit 43. The feature point extraction unit 43 of the second embodiment extracts feature points specifying the contours of the front 694 and side 695 of the adapter 692 included in the learning image 411, and feature points specifying the contours of the front 694 and side 695 of the tooth 693. The feature point extraction unit 43 specifies the width Wf of the front 694 and the width Ws of the side 695 at the boundary 696 between the adapter 692 and the tooth 693 from the extracted feature points.

[0109] 9, the reference value calculation unit 44 of the second embodiment includes a width ratio calculation unit 443. The width ratio calculation unit 443 calculates a ratio (Wf / Ws) between a width Wf of a front surface 694 and a width Ws of a side surface 695 at a boundary portion 696 identified by the feature point extraction unit 43. The width ratio calculation unit 443 stores the calculated ratio (Wf / Ws) in the reference value storage unit 45 as a reference value X. At this time, the reference value storage unit 45 stores the reference value X calculated by the reference value calculation unit 44 in association with a part number that identifies the type of the tooth 693.

[0110] When evaluating whether a captured image 461 stored in the captured image storage unit 46 is suitable as a measurement image, the feature point extraction unit 43 of the second embodiment extracts feature points specifying the contours of the front surface 694 and side surface 695 of the adapter 692 included in the captured image 461 and feature points specifying the contours of the front surface 694 and side surface 695 of the tooth 693. The feature point extraction unit 43 specifies the width Wf of the front surface 694 and the width Ws of the side surface 695 at the boundary portion 696 between the adapter 692 and the tooth 693 from the extracted feature points.

[0111] 9, the evaluation value calculation unit 47 of the second embodiment includes a reference value average calculation unit 471, a width ratio calculation unit 475, and a width ratio difference calculation unit 476. The reference value average calculation unit 471 calculates an average value Xm of a plurality of reference values ​​X associated with the part number of the tooth 693 and stored in the reference value storage unit 45, as in the first embodiment. The width ratio calculation unit 475 calculates a ratio X (=Wf / Ws) between a width Wf of a front surface 694 and a width Ws of a side surface 695 at a boundary portion 696 identified by the feature point extraction unit 43. The width ratio difference calculation unit 476 calculates a difference value |X-Xm| between the ratio X calculated by the width ratio calculation unit 475 and the average value Xm calculated by the reference value average calculation unit 471.

[0112] The evaluation value calculation unit 47 of the second embodiment uses the difference value |X-Xm| calculated by the width ratio difference calculation unit 476 as an evaluation value for evaluating whether the captured image 461 is suitable as a measurement image. The evaluation value calculation unit 47 inputs the difference value |X-Xm| calculated as the evaluation value to the image determination unit 48.

[0113] The image determination unit 48 of the second embodiment, like the first embodiment, determines whether or not the captured image 461 can be used as a measurement image, depending on the evaluation value calculated by the evaluation value calculation unit 47. The image extraction unit 49 of the second embodiment extracts a measurement image from the captured images 461 stored in the captured image storage unit 46, depending on the determination result of the image determination unit 48, like the first embodiment.

[0114] As described above, the image evaluation device 40 of the second embodiment can appropriately and easily evaluate whether the captured image 461 is suitable for measuring the amount of wear, similar to the first embodiment, even when there is no directly frontal reference plane for a specific part of the work machine 6, such as the tooth 693. The image evaluation device 40 of the second embodiment can automatically extract images suitable for measuring the amount of wear from a collection of captured moving images or still images, similar to the first embodiment.

[0115] [Embodiment 3] The image evaluation device 40 of the third embodiment will be described with reference to Figures 13 and 14. In the image evaluation device 40 of the third embodiment, the description of the same configuration and operation as in the first embodiment will be omitted. FIG. 13 is a block diagram showing the functional configuration of an image evaluation device 40 according to the third embodiment.

[0116] The imaging device 31 has a live view function that displays an image of a subject on the display device 32 in real time when the image is captured. The image evaluation device 40 of the third embodiment does not evaluate a captured image 461 that has already been captured and that is stored in the captured image storage unit 46, but evaluates a captured image 461 (hereinafter also referred to as a "live view image 501") acquired by the live view function of the imaging device 31. The image evaluation device 40 of the third embodiment guides the position or attitude of the imaging device 31 via the display device 32 so that, when a user uses the imaging device 31 to capture an image of a part of the work machine 6 that is subject to wear, the imaging device 31 can capture a measurement image suitable for measuring the amount of wear.

[0117] As shown in FIG. 13, the image evaluation device 40 of the third embodiment includes a live view image acquisition section 50 instead of the captured image storage section 46, and an imaging guide section 51 instead of the image extraction section 49.

[0118] The live view image acquisition unit 50 acquires a live view image 501 acquired by the live view function of the imaging device 31 from the imaging device 31, and inputs it to the learned feature point extraction unit 43. When the image determination unit 48 determines that the live view image 501 cannot be used as a measurement image, the imaging guidance unit 51 outputs a message to the display device 32 to guide the position or attitude of the imaging device 31 so that the live view image 501 can be used as a measurement image. The imaging guidance unit 51 may output the message to a speaker (not shown) of the communication terminal 3. When the image determination unit 48 determines that the live view image 501 can be used as a measurement image, the imaging guidance unit 51 automatically captures the live view image 501 and stores it in a predetermined storage unit. The time for which the automatic imaging is continued is arbitrary and continues as long as the live view image 501 is suitable as a measurement image, but it may be stopped at any timing by an operation instructing "stop". The imaging time may be changed according to the progress of wear. In this case, a reference value for judging the progress of wear may be set in advance, and if the reference value is exceeded, it may be assumed that wear is progressing, and automatic imaging may be repeated at least two times, with the operator being asked to select the best imaging result. This makes it possible to grasp the progress of wear more reliably and with higher accuracy.

[0119] In this way, the image evaluation device 40 of the third embodiment can operate in an automatic imaging mode in which the live view image 501 is automatically captured when it is suitable as a measurement image.

[0120] FIG. 14 is a flowchart showing the operation of the image evaluation device 40 in the automatic imaging mode.

[0121] In step S1401, the image evaluation device 40 accepts the start of the automatic imaging mode in response to an input operation by the user via the input device 33.

[0122] In step S1402, the image evaluation device 40 determines whether or not a change to the threshold T has been accepted in response to an input operation by the user via the input device 33. The threshold T is a threshold for determining whether or not the live view image 501 can be used as a measurement image, and is stored in advance in a threshold storage unit (not shown) of the image evaluation device 40. If the image evaluation device 40 has not accepted a change to the threshold T, the image evaluation device 40 proceeds to step S1404. If the image evaluation device 40 has accepted a change to the threshold T, the image evaluation device 40 proceeds to step S1403.

[0123] In step S1403, the image evaluation device 40 resets the received threshold value T.

[0124] In step S1404, the image evaluation device 40 receives the part number of the sprocket 63 in response to an input operation by the user via the input device 33.

[0125] In step S1405, the image evaluation device 40 determines whether or not the reference value X is stored in the reference value storage unit 45. If the reference value X is stored in the reference value storage unit 45, the image evaluation device 40 proceeds to step S1407. If the reference value X is not stored in the reference value storage unit 45, the image evaluation device 40 proceeds to step S1406.

[0126] In step S1406, the image evaluation device 40 outputs a message to the display device 32 to the effect that the reference value X is not stored. The image evaluation device 40 then ends this process shown in FIG.

[0127] In step S 1407 , the image evaluation device 40 calculates the average value Xm of the reference values ​​X stored in the reference value storage unit 45 .

[0128] In step S1408, the image evaluation device 40 reads the live view image 501 acquired by the live view image acquisition unit 50.

[0129] In steps S1409 to S1413, the image evaluation device 40 performs the same processes as those in steps S807 to S811 shown in FIG.

[0130] In step S1414, the image evaluation device 40 judges whether or not the judgment result in step S1413 indicates that the image can be used as a measurement image. If the judgment result does not indicate that the image can be used as a measurement image, the image evaluation device 40 proceeds to step S1417. If the judgment result indicates that the image can be used as a measurement image, the image evaluation device 40 proceeds to step S1415.

[0131] In step S1415, the image evaluation device 40 captures the currently read live view image 501 and stores it as a measurement image.

[0132] In step S1416, the image evaluation device 40 outputs the extracted measurement image to the display device 32. The image evaluation device 40 ends this process shown in FIG.

[0133] In step S1417, the image evaluation device 40 outputs to the display device 32 a message guiding the position or attitude of the image capture device 31 so that the live view image 501 can be used as a measurement image. For example, when the reference point of the front circle F calculated in step S1410 is diagonally above and to the right of the reference point of the rear circle B, the image evaluation device 40 outputs to the display device 32 a message guiding the user to "move the image capture device 31 further diagonally above and to the right and capture an image from that position."

[0134] In step S1418, the image evaluation device 40 determines whether or not the position or attitude of the imaging device 31 has changed. The image evaluation device 40 can determine whether or not the position or attitude of the imaging device 31 has changed by checking a change in the live view image 501 or checking a detection signal from an attitude sensor of the communication terminal 3. If the position or attitude of the imaging device 31 has changed, the image evaluation device 40 proceeds to step S1408. If the position or attitude of the imaging device 31 has not changed, the image evaluation device 40 proceeds to step S1419.

[0135] In step S1419, the image evaluation device 40 determines whether or not the end of the automatic imaging mode has been accepted in response to an input operation by the user via the input device 33. If the end of the automatic imaging mode has not been accepted, the image evaluation device 40 proceeds to step S1418. If the end of the automatic imaging mode has been accepted, the image evaluation device 40 ends this process shown in FIG.

[0136] As described above, the image evaluation device 40 of the third embodiment includes the imaging guidance unit 51 that provides guidance on the position or attitude of the imaging device 31 that captures images of parts of the work machine 6 that are subject to wear. The captured image 461 to be evaluated in the third embodiment is a live view image 501 acquired by the live view function of the imaging device 31. When the image determination unit 48 determines that the live view image 501 cannot be used as a measurement image, the imaging guidance unit 51 outputs a message that provides guidance on the position or attitude of the imaging device 31 so that the live view image 501 can be used as a measurement image.

[0137] As a result, the image evaluation device 40 of the third embodiment can easily enable a user who is about to capture an image of a worn part of the work machine 6 to capture an image suitable for measuring the amount of wear. Therefore, the image evaluation device 40 of the third embodiment can further simplify and improve the efficiency of the work of measuring the amount of wear. Also, when capturing an image, due to the characteristics of the subject to be captured, there are cases where object recognition is not possible due to the presence of mud, etc. This is an obstacle to accurate measurement of the amount of wear. By capturing an image of the part to be captured in real time using a live view image, it is possible to obtain a more accurate image suitable for measuring the amount of wear.

[0138] Although the third embodiment has been described taking as an example a case where the part of the work machine 6 that wears is the sprocket 63, the image evaluation device 40 of the third embodiment is naturally also applicable to a case where the part of the work machine 6 that wears is the tooth 693. Also, the image evaluation device 40 of the third embodiment is provided with a live view image acquisition unit 50 instead of the captured image storage unit 46, and an image pickup guide unit 51 instead of the image extraction unit 49. However, the image evaluation device 40 may be provided with both the captured image storage unit 46 and the live view image acquisition unit 50, and may be provided with both the image extraction unit 49 and the image pickup guide unit 51.

[0139] [Other embodiments] The wear amount measuring system 1 measures the amount of wear using the measurement image extracted by the image evaluation device 40. The measurement of the amount of wear may be performed by the communication terminal 3 or by the server device 2. The wear amount measuring system 1 can identify the level of the measured amount of wear, create a wear amount indicator image including an indication according to the identified level, and output it to the display device 32 of the communication terminal 3.

[0140] The wear level may be, for example, a first level where the worn portion has reached an amount of wear that requires replacement, a second level where the worn portion is likely to reach an amount of wear that requires replacement after a predetermined amount of work has been performed, and a third level where the worn portion has not yet reached an amount of wear that requires replacement even after a predetermined amount of work has been performed.

[0141] The indicator image of the wear amount is created for each part by changing at least one of the shape, size, color, and pattern according to the level of the wear amount. For example, if the level of the wear amount is the first level, the indicator image of the wear amount may be created using red, if the level of the wear amount is the second level, the indicator image of the wear amount may be created using yellow, and if the level of the wear amount is the third level, the indicator image of the wear amount may be created using green. Note that the levels and colors are not limited to three levels or three colors, and may be set arbitrarily. Alternatively, for example, the indicator image of the wear amount may be created so that the length of the indicator image of the wear amount is the longest when the level of the wear amount is the first level, the length of the indicator image of the wear amount is medium when the level of the wear amount is the second level, and the length of the indicator image of the wear amount is the shortest when the level of the wear amount is the third level. That is, the indicator image may be created so that the length of the indicator image becomes longer as the replacement recommendation level of the worn part becomes higher.

[0142] Furthermore, the wear amount measurement system 1 can predict the future change in the amount of wear from the current operating status of the work machine 6. Then, the wear amount measurement system 1 can create an image in which the predicted change in the amount of wear is superimposed on the indicator image, and output it to the display device 32. Alternatively, the wear amount measurement system 1 can create an animation in which the length of the indicator image changes according to the predicted change in the amount of wear, and output it to the display device 32.

[0143] Furthermore, the wear amount measurement system 1 can collect the wear amounts of parts that wear in a plurality of work machines 6 in the server device 2, and identify a plurality of work machines 6 that have similar wear amounts. Then, the wear amount measurement system 1 can transmit information on a plurality of work machines 6 that have similar wear amounts to the users' communication terminals 3, allowing the information to be shared among the users.

[0144] With this configuration, the wear amount measurement system 1 can notify the user of information such as the wear amount level, part replacement timing, and predicted change in the wear amount in a format that is easy to intuitively understand.

[0145] Although the embodiments of the present invention have been described in detail above, the present invention is not limited to the above embodiments, and various modifications can be made without departing from the spirit of the present invention described in the claims. In the present invention, the configuration of one embodiment can be added to the configuration of another embodiment, the configuration of one embodiment can be replaced with the configuration of another embodiment, or a part of the configuration of one embodiment can be deleted. [Explanation of symbols]

[0146] 1...wear amount measuring system, 2...server device, 3...communication terminal, 31...imaging device, 32...display device, 33...input device, 40...image evaluation device, 401...arithmetic processing device, 402...storage device, 41...teacher data storage unit, 411...learning image, 412...feature point data, 42...learning unit, 43...feature point extraction unit, 44...reference value calculation unit, 441...center radius calculation unit, 442...ratio calculation unit, 443...width ratio calculation unit, 45...reference value storage unit, 46...captured image storage unit, 461...captured image, 47...evaluation value calculation unit, 471...reference value average calculation unit, 472...center radius calculation unit, 473...difference length calculation unit, 474...difference ratio calculation unit, 475...width ratio calculation unit, 476...width ratio difference calculation unit , 48...image determination unit, 49...image extraction unit, 50...live view image acquisition unit, 501...live view image, 51...imaging guide unit, 6...work machine, 61...lower running body, 62...idler, 63...sprocket, 631...tooth, 632...tooth bottom, 633...front, 634...rear, 635...side, 64...crawler, 65...upper rotating body, 651...rotating device, 652...cab, 653...machine room, 66...front working machine, 67...boom, 671...boom cylinder, 68...arm, 681...arm cylinder, 69...bucket, 69a...tip, 691...bucket cylinder, 692...adapter, 693...tooth, 694...front, 695...side, 696...boundary

Claims

1. 1. An image evaluation device for evaluating whether a captured image of a specific portion of a work machine is suitable as a measurement image used for measuring an amount of wear of the specific portion, a feature point extraction unit that extracts feature points that specify the contour of the specific part included in the captured image, the feature points being machine-trained as teacher data based on a learning image including the contour of the specific part and a plurality of feature points for specifying the contour of the specific part included in the learning image; a reference value storage unit configured to store a reference value that is a reference for evaluating the captured image and that is calculated from the feature points in the learning image captured from a direction suitable for measuring the amount of wear; and an evaluation value calculation unit that calculates an evaluation value for evaluating whether the captured image is suitable as the measurement image based on the feature points in the captured image extracted by the feature point extraction unit and the reference value stored in the reference value storage unit; an image determination unit that determines whether or not the captured image can be used as the measurement image according to the evaluation value calculated by the evaluation value calculation unit; An image evaluation device comprising:

2. An image extraction unit that extracts the measurement image, The captured image is a moving image or a plurality of still images captured in advance, The image extraction unit extracts the measurement image from among a plurality of frame images constituting the moving image or the plurality of still images according to a determination result of the image determination unit.

2. The image evaluation device according to claim 1.

3. An imaging guide unit that guides a position or a posture of an imaging device that images the specific part, the captured image is a live view image acquired by the imaging device, When the image determination unit determines that the live view image cannot be used as the measurement image, the imaging guidance unit outputs a message providing guidance on a position or attitude of the imaging device so that the live view image can be used as the measurement image.

2. The image evaluation device according to claim 1.

Images