Missing tooth determination system and missing tooth determination method

Abstract

In the present invention, a processor acquires a captured image showing a bucket having teeth. An area in which the bucket appears is extracted by removing the background portion from the captured image. The processor detects, from the area in which the bucket appears, a plurality of protruding points protruding from the body of the bucket, and determines the presence or absence of a missing tooth on the basis of the plurality of protruding points.

Description

Tooth deficiency determination system and tooth deficiency determination method

[0001] The present disclosure relates to a tooth deficiency determination system and a tooth deficiency determination method. The present application claims priority based on Japanese Patent Application No. 2025-053031 filed in Japan on March 27, 2025, the content of which is incorporated herein by reference.

[0002] As disclosed in Patent Document 1, working machines having a bucket such as wheel loaders and hydraulic excavators are known.

[0003] Japanese Unexamined Patent Publication No. 2021-127601

[0004] A plurality of teeth for penetrating into earth and sand are attached to the tip of a bucket included in a working machine. The teeth of the bucket may be damaged due to loads caused by work such as excavation of earth and sand. An object of the present disclosure is to provide a tooth deficiency determination system and a tooth deficiency determination method capable of determining whether there is a tooth deficiency.

[0005] According to one aspect of the present invention, a tooth deficiency determination system includes a processor, wherein the processor acquires a captured image capturing a bucket having teeth, extracts a region capturing the bucket by removing a background portion from the captured image, detects a plurality of protruding points protruding from the bucket body from the region capturing the bucket, and determines whether there is a tooth deficiency based on the plurality of protruding points.

[0006] According to the above aspect, it is possible to determine whether there is a tooth deficiency.

[0007] FIG. 1 is a side view of the working machine according to the first embodiment. FIG. 2 is a schematic block diagram showing the configuration of a control device of the working machine according to the first embodiment. FIG. 3 is a flowchart showing tooth deficiency determination processing by the control device according to the first embodiment. FIG. 4 is a diagram showing an example of the progress of the tooth deficiency determination processing according to the first embodiment.

[0008] <First Embodiment> <Configuration of the Working Machine> The embodiments will be described in detail below with reference to the drawings. Figure 1 is a side view of the working machine according to the first embodiment. The working machine 100 according to the first embodiment is a wheel loader. The working machine 100 comprises a body 110, a working machine 120, and wheels 130.

[0009] The vehicle body 110 is provided with a driver's cab 111 where the operator sits. Inside the driver's cab 111 is an operating device 112 for operating the work machine 100. The operating device 112 includes an accelerator pedal, a brake pedal, a steering wheel, a forward / reverse selector switch, a shift switch, a boom lever, and a bucket lever.

[0010] The vehicle body 110 consists of a front vehicle body 110a and a rear vehicle body 110b. The front vehicle body 110a and the rear vehicle body 110b are rotatably connected around a steering axis that extends vertically across the vehicle body 110. A front wheel 130a, which is a wheel 130, is provided at the lower part of the front vehicle body 110a. A rear wheel 130b, which is a wheel 130, is provided at the lower part of the rear vehicle body 110b. A steering cylinder 113 is provided between the front vehicle body 110a and the rear vehicle body 110b. The steering cylinder 113 is a hydraulic cylinder. The base end of the steering cylinder 113 is attached to the rear vehicle body 110b, and the tip end is attached to the front vehicle body 110a. The steering cylinder 113 expands and contracts with the help of hydraulic fluid, thereby defining the angle between the front vehicle body 110a and the rear vehicle body 110b. In other words, the steering angle of the front wheels 130a is determined by the extension and retraction of the steering cylinder 113.

[0011] The work machine 120 is used for excavating and transporting materials such as soil and sand. The work machine 120 is installed at the front of the vehicle body 110. The work machine 120 comprises a boom 121, a bucket 122, a bell crank 123, a lift cylinder 124, and a bucket cylinder 125.

[0012] The base end of the boom 121 is attached to the front of the front body 110a via a pin. The bucket 122 comprises a plurality of teeth 1221 for excavating the workpiece and a container for transporting the excavated workpiece. The base end of the bucket 122 is attached to the tip of the boom 121 via a pin. The plurality of teeth 1221 are arranged at equal intervals in the width direction at the tip of the bucket 122. The bell crank 123 transmits power from the bucket cylinder 125 to the bucket 122. The first end of the bell crank 123 is attached to the bottom of the bucket 122 via a link mechanism. The second end of the bell crank 123 is attached to the tip of the bucket cylinder 125 via a pin.

[0013] The lift cylinder 124 is a hydraulic cylinder. The base end of the lift cylinder 124 is attached to the front of the front body 110a. The tip end of the lift cylinder 124 is attached to the boom 121. The boom 121 is driven in the upward or downward direction by the extension and contraction of the lift cylinder 124 by hydraulic fluid. The bucket cylinder 125 is a hydraulic cylinder. The base end of the bucket cylinder 125 is attached to the front of the front body 110a. The tip end of the bucket cylinder 125 is attached to the bucket 122 via a bell crank 123. The bucket 122 swings in the tilt direction or dump direction by the extension and contraction of the bucket cylinder 125 by hydraulic fluid. The lift cylinder 124 is provided with a lift stroke sensor 1241 for measuring the stroke length. The bucket cylinder 125 is provided with a bucket stroke sensor 1251 for measuring the stroke length. The posture of the work machine 120 can be determined from the stroke lengths of the lift cylinder 124 and the bucket cylinder 125.

[0014] 《Imaging Device》 An imaging device 114 is provided on the front of the front body 110a. The imaging device 114 captures images of the area in front of the body 110. The imaging range of the imaging device 114 includes the work machine 120. In particular, when the work machine 120 is in a position to dump soil, the teeth 1221 of the bucket 122 are included in the imaging range. The conditions for the position of the work machine 120 in which the teeth 1221 are included in the imaging range are specified in advance. Note that the installation location of the imaging device 114 is not limited to the location shown in Figure 1. For example, the imaging device 114 may be provided near the front lights of the work machine 100, or it may be provided in the driver's cab. Also, imaging devices 114 may be provided on both the left and right sides of the work machine 100. When imaging devices 114 are provided on both the left and right sides of the work machine 100, the entire bucket 122 may be captured by combining the images captured by each imaging device 114.

[0015] Control device: The work machine 100 is equipped with a control device 300 for controlling the work machine 100 according to the amount of operation of the operating device 112.

[0016] Figure 2 is a schematic block diagram showing the configuration of the control device for the work machine according to the first embodiment. The control device 300 is a computer comprising a processor 310, main memory 330, storage 350, and interface 370.

[0017] The storage 350 is a tangible, non-temporary storage medium. Examples of the storage 350 include HDDs (Hard Disk Drives), SSDs (Solid State Drives), magnetic disks, magneto-optical disks, CD-ROMs (Compact Disc Read Only Memory), DVD-ROMs (Digital Versatile Disc Read Only Memory), and semiconductor memory. The storage 350 may be an internal medium directly connected to the bus of the control device 300, or an external medium connected to the control device 300 via an interface 370 or a communication line. The storage 350 stores a program for controlling the work machine 100.

[0018] The program may be for implementing a part of the functions to be performed by the control device 300. For example, the program may perform functions in combination with other programs already stored in the storage 350, or in combination with other programs implemented in other devices. In other embodiments, the control device 300 may include a custom LSI (Large Scale Integrated Circuit) such as a PLD (Programmable Logic Device) in addition to or instead of the above configuration. Examples of PLDs include PAL (Programmable Array Logic), GAL (Generic Array Logic), CPLD (Complex Programmable Logic Device), and FPGA (Field Programmable Gate Array). In this case, some or all of the functions implemented by the processor 310 may be implemented by the integrated circuit.

[0019] When a program is distributed to the control device 300 via a communication line, the receiving control device 300 may expand the program into the main memory 330 and execute the above processing. Furthermore, the program may be for the purpose of realizing a part of the functions described above. Moreover, the program may be a so-called differential file (differential program) that realizes the functions described above in combination with other programs already stored in the storage 350.

[0020] The control device 300 according to the first embodiment, in addition to controlling the work machine 100, detects whether or not there is a missing tooth 1221 in the bucket 122. If there is a missing tooth 1221, the control device 300 notifies the operator of the defect. This prevents the tooth 1221 that has fallen out of the bucket 122 from entering the inside of other machines such as a crusher and causing a malfunction. In other words, the control device 300 according to the first embodiment is an example of a tooth defect detection system.

[0021] The processor 310, by executing a program, includes an image acquisition unit 311, a condition determination unit 312, a distance image generation unit 313, an extraction unit 314, a missing image determination unit 315, and a notification unit 316.

[0022] The image acquisition unit 311 acquires an image from the imaging device 114. The condition determination unit 312 determines whether or not to determine whether or not the tooth 1221 is missing. The control device 300 cannot determine whether or not the tooth 1221 is missing if it is not visible in the image, so the condition determination unit 312 determines whether or not the tooth 1221 is visible in the image. In the first embodiment, the condition determination unit 312 determines whether or not to determine whether or not the tooth 1221 is missing based on the measurement data of the lift stroke sensor 1241 and the bucket stroke sensor 1251. The conditions for the posture of the work machine 120 so that the tooth 1221 is included in the imaging range are known, and the posture of the work machine 120 can be identified from the measurement data of the lift stroke sensor 1241 and the bucket stroke sensor 1251, so the condition determination unit 312 can determine whether or not to determine whether or not the tooth 1221 is missing based on the measurement data of the lift stroke sensor 1241 and the bucket stroke sensor 1251.

[0023] The condition determination unit 312 can directly determine whether or not tooth 1221 is missing based on whether or not the measured data values ​​from the lift stroke sensor 1241 and the bucket stroke sensor 1251 are within a predetermined range. Alternatively, the condition determination unit 312 may determine whether or not tooth 1221 is missing by identifying the position of the tip of the bucket 122 from the measured data values ​​from the lift stroke sensor 1241 and the bucket stroke sensor 1251, and determining whether or not the identified position is included in a known imaging range.

[0024] The depth image generation unit 313 generates a depth image from the captured image using a monocular depth estimation model, which is a pre-trained machine learning model. A depth image is image data that has pixel values ​​representing the depth of the imaging device 114. The monocular depth estimation model may be composed of a neural network such as a CNN (Convolutional Neural Network). An example of a monocular depth estimation model is Depth Anything (https: / / depth-anything-v2.github.io / , accessed October 25, 2024). The monocular depth estimation model is trained to output a depth image representing the depth of the subject in a color image when a color image is input, using a training dataset consisting of a combination of a color image of the subject captured by the imaging device and a depth image having pixel values ​​representing the depth of the subject. Note that the training dataset used to train the monocular depth estimation model does not necessarily have to be an image of bucket 122 or tooth 1221. In other words, the monocular depth estimation model does not need to be specifically trained for the work machine 100, as long as it is capable of generating a depth image from a color image. In other embodiments, a general-purpose semantic segmentation model may be used instead of the monocular depth estimation model. An example of a semantic segmentation model is SAM (https: / / segment-anything.com / , accessed March 7, 2025).

[0025] The extraction unit 314 removes the background portion from the depth image generated by the depth image generation unit 313 and generates a binarized image in which the working machine 120, which is the subject, is captured. Specifically, the extraction unit 314 binarizes the depth image using a threshold distance that is greater than the furthest distance related to the movable range of the working machine 120. For example, the extraction unit 314 generates a binarized image in which pixels representing distances shorter than the threshold distance in the depth image are white, and pixels representing distances longer than the threshold distance are black. In other embodiments, the extraction unit 314 may extract the area in which the working machine 120, which is the subject, is captured by, for example, identifying the portion where the distance difference between adjacent pixels exceeds a predetermined threshold as an outline.

[0026] The missing image detection unit 315 determines whether or not there are missing teeth 1221 based on the binarized image generated by the extraction unit 314. The missing image detection unit 315 obtains a determination curve that passes through the teeth 1221 based on the binarized image and identifies the portion where the white area of ​​the binarized image (the area where the work machine 120 is visible) and the determination curve overlap as the portion where the teeth 1221 are visible. The missing image detection unit 315 determines whether or not there are missing teeth 1221 based on the number and spacing of the portions where the teeth 1221 are visible.

[0027] The notification unit 316 notifies the operator of the tooth 1221 defect when the defect detection unit 315 determines that there is a defect in the tooth 1221. For example, the notification unit 316 may output an alert sound to indicate the defect in the tooth 1221 from a speaker (not shown). The notification unit 316 may also display the defective portion of the tooth 1221 on a monitor (not shown).

[0028] <Operation of Control Device 300> Figure 3 is a flowchart showing the tooth absence determination process by the control device according to the first embodiment. The control device 300 performs the tooth absence determination process shown in Figure 3 at predetermined calculation cycles while the work machine 100 is in operation. Figure 4 is a diagram showing an example of the progress of the tooth absence determination process according to the first embodiment.

[0029] The image acquisition unit 311 of the control device 300 acquires an image G1 (Figure 4) from the imaging device 114 (step S1). The condition determination unit 312 acquires measurement data from the lift stroke sensor 1241 and the bucket stroke sensor 1251 (step S2). The condition determination unit 312 determines whether or not it is possible to determine the absence of tooth 1221 based on the measurement data (step S3). If it is not possible to determine the absence of tooth 1221 (step S3: NO), the tooth absence determination process in that calculation cycle is skipped.

[0030] If it is possible to determine the absence of tooth 1221 (step S3: YES), the depth image generation unit 313 generates a depth image G2 (Figure 4) from the captured image G1 using a monocular depth estimation model (step S4). Next, the extraction unit 314 generates a binarized image G3 (Figure 4) from the depth image G2 (step S5). In the binarized image G3, pixels representing distances shorter than the threshold distance are white (pixel value = 1), and pixels representing distances longer than the threshold distance are black (pixel value = 0).

[0031] The missing image detection unit 315 extracts contour lines G4 (Figure 4) of the white and black regions from the binarized image G3 (step S6). Next, the missing image detection unit 315 identifies the envelope G5 (Figure 4) of the convex hull that encloses the pixels constituting the extracted contour lines G4 (step S7). The envelope G5 is a polyline connecting the tips of the teeth 1221. The missing image detection unit 315 extracts pixels G6 (Figure 4) from the subject area (white region) of the binarized image that overlap with the envelope G5 (step S8). The extracted pixels G6 are pixels corresponding to the tips of the teeth 1221.

[0032] The defect detection unit 315 determines a first curve G71 (Figure 4) to fit the extracted pixels G6 (step S9). The first curve G71 may be represented by, for example, a quadratic function or an elliptic function. The tip of the bucket 122 often has a curve to easily penetrate the soil. Therefore, the defect detection unit 315 according to the first embodiment determines the first curve G71 as a curve such as a quadratic function. Next, the defect detection unit 315 determines a second curve G72 (Figure 4), which is a curve obtained by translating the first curve G71 and passes through the base of the tooth 1221 of the bucket 122 (step S10). The second curve G72 is the curve that is closest to the first curve among the curves that have a continuous white area overlapping the binarized image G3 without interruption. The defect detection unit 315 identifies a curve obtained by translating the first curve G71, which is located between the first curve G71 and the second curve G72, as the determination curve G73 (Figure 4) to be used for determining the defect of the tooth 1221 (step S11). The determination curve G73 may be positioned, for example, in the middle of the first curve G71 and the second curve G72. The second curve G72 is a curve that passes through the base of the tooth 1221 of the bucket 122. The first curve G71 is a curve that passes near the tip of the bucket body. Therefore, pixels that overlap with the determination curve G73 can be said to represent protruding points that protrude from the bucket body. The protruding points are usually teeth 1221.

[0033] The missing data detection unit 315 determines the distribution of pixel values ​​G8 (Figure 4) of the binarized image G3 passing through the determination curve G73 along the determination curve G73 (step S12). The missing data detection unit 315 identifies a central point for each continuous white area (pixel value = 1) from the distribution G8 and determines the interval between adjacent central points (step S13). The missing data detection unit 315 determines whether or not an abnormal value that is more than a predetermined threshold away from the average is included in the interval between the central points (step S14).

[0034] The missing data detection unit 315 determines that if an abnormal value is found in the interval between the central points (step S14: YES), there is a missing tooth 1221 in the portion corresponding to the abnormal value. If the notification unit 316 determines that there is a missing tooth 1221, it notifies the operator of the occurrence of the missing tooth 1221 (step S15). On the other hand, if the missing data detection unit 315 does not find an abnormal value in the interval between the central points (step S14: NO), it determines that there is no missing tooth 1221.

[0035] 《Operation and Effects》 As described above, the control device 300 according to the first embodiment performs the following processing. The image acquisition unit 311 acquires an image G1 in which the bucket 122 having teeth 1221 is visible. The extraction unit 314 extracts the region in which the bucket 122 is visible by removing the background portion from the image G1. The defect determination unit 315 detects a plurality of protruding points that protrude from the bucket body from the region in which the bucket 122 is visible, and determines whether or not there is a defect in the teeth 1221 based on the plurality of protruding points. The control device 300 can determine whether or not there is a defect in the teeth 1221 without learning using label data of the teeth 1221 by removing the background portion from the image G1 and identifying the protruding points.

[0036] In particular, the control device 300 according to the first embodiment generates a depth image G2 from the captured image G1 and extracts the area in which the bucket is captured based on the depth image G2. In the captured image G1, the hue and brightness of the background and the bucket 122 may be similar depending on the imaging conditions. In such cases, it may be difficult to directly remove the background portion from the captured image G1. In contrast, the control device 300 according to the first embodiment can clearly separate the bucket 122 from the background by using the distance image G2, rather than by hue or brightness.

[0037] In the first embodiment, the control device 300 determines the presence or absence of missing teeth 1221 based on white pixels G6 that touch the determination curve G73 in the binarized image G3, rather than white pixels G6 that touch the envelope G5 of the convex hull in the binarized image G3. As described above, the tip of the bucket 122 often forms a curve, and in some cases, the absence of teeth 1221 can be determined by the white pixels G6 that touch the envelope G5. On the other hand, since the teeth 1221 wear down with use, there is variation in the length of each tooth 1221. Therefore, depending on the wear situation, even if there is no defect, teeth 1221 may not touch the envelope G5 due to wear. In contrast, by using a determination curve offset towards the bucket body side from the envelope G5, as in the first embodiment, it is possible to detect teeth 1221 that have been shortened due to wear. As a result, the control device 300 in the first embodiment can suppress false detection of missing teeth 1221. On the other hand, in other embodiments, the control device 300 may use the envelope G5 instead of the determination curve G73 to determine whether a defect exists.

[0038] Furthermore, the determination curve G73 according to the first embodiment is determined from a first curve G71 that fits to the tip of the tooth 1221 and a second curve G72 that fits to the base of the tooth 1221 (the tip of the bucket body). This allows the control device 300 to set a determination curve G73 that passes through the tooth 1221 at a constant rate, regardless of the type of bucket 122 or tooth 1221. On the other hand, the determination curve G73 according to other embodiments is not limited to this, and may be determined, for example, at a position offset by a predetermined distance from the first curve G71, and may be determined without relying on the second curve G72.

[0039] <Other Embodiments> Although one embodiment has been described in detail above with reference to the drawings, the specific configuration is not limited to that described above, and various design changes can be made. That is, in other embodiments, the order of the above-described processes may be changed as appropriate. Also, some processes may be executed in parallel. The control device 300 according to the above embodiment may be composed of a single computer, or the configuration of the control device 300 may be divided among multiple computers, and the multiple computers may cooperate with each other to function as the control device 300. In this case, some of the computers constituting the control device 300 may be mounted inside the work machine, and the other computers may be provided outside the work machine.

[0040] The missing image detection unit 315 of the control device 300 according to the above embodiment identifies the envelope G5 of the convex hull for the binarized image G3 generated by the extraction unit 314, but is not limited to this. In addition to the bucket 122, other parts of the work machine 100, such as the boom 121, may be visible in the binarized image G3. Since the boom 121 is located in front of the bucket 122, the part of the binarized image G3 where the boom 121 is visible will be white. If the boom 121 is visible below the tooth 1221 in the binarized image G3, it may not be possible to obtain an appropriate envelope G5 due to the boom 121. For this reason, the missing image detection unit 315 according to other embodiments may identify the envelope G5 after trimming the part of the binarized image G3 where the bucket 122 is not present. The part where the bucket 122 is not present may be determined by a trained model such as a CNN, calculated from the posture of the work machine 120, or identified in advance by experimentation or the like.

[0041] The condition determination unit 312 of the control device 300 according to the above embodiment determines whether or not a tooth 1221 is missing based on the posture of the work machine 120 identified from the measurement data of the stroke sensor, but is not limited to this. For example, the condition determination unit 312 according to another embodiment may determine whether or not a tooth 1221 is missing based on the captured image G1 or the distance image G2. For example, the condition determination unit 312 according to another embodiment may determine whether or not a tooth 1221 is missing using a trained model that takes the captured image G1 or the distance image G2 as input and calculates whether or not a tooth 1221 is missing. Such a trained model is trained in advance using a training dataset consisting of a combination of the captured image G1 or the distance image G2 and label data representing whether or not a tooth 1221 is missing from the image. Furthermore, in another embodiment, the condition determination unit 312 may determine whether or not a tooth 1221 is missing based on whether or not the number of pixels whose difference from the average or median value of the pixel values ​​of the distance image G2 is within a predetermined threshold is greater than or equal to a predetermined number.

[0042] The control device 300 according to the above embodiment generates a distance image G2 from the captured image G1 using a monocular depth estimation model, but is not limited to this. For example, if the imaging device 114 according to another embodiment is a TOF (Time of Flight) camera, a laser scanner, or a LiDAR device, the control device 300 may directly acquire the distance image G2 without going through the captured image G1, which is a color image. Also, if the imaging device 114 according to another embodiment is a stereo camera, the control device 300 may acquire the distance image G2 from the stereo image.

[0043] Although the work machine 100 in the above-described embodiment is a wheel loader, the work machine 100 in other embodiments is not limited to this and may be other work machines 100 having a bucket 122, such as a hydraulic excavator (backhoe, loading shovel).

[0044] The control device 300 according to the above-described embodiment generates a binarized image from which a background is removed using a distance image, but the present invention is not limited thereto. For example, in another embodiment, a segmentation image may be generated using a general-purpose semantic segmentation model instead of a monocular depth estimation model. Examples of semantic segmentation models include SAM (https: / / segment-anything.com / , searched on March 7, 2025). The control device 300 may generate a binarized image obtained by extracting a segment including pixels in which the bucket 122 is necessarily captured, from the segmentation image.

[0045] According to the above aspect of the present disclosure, the presence or absence of a tooth defect can be determined.

[0046] 100…Working machine 110…Vehicle body 120…Working implement 130…Wheels 110a…Front vehicle body 110b…Rear vehicle body 113…Steering cylinder 114…Imaging device 130a…Front wheels 130b…Rear wheels 111…Operator's cab 112…Operating device 121…Boom 122…Bucket 123…Bell crank 124…Lift cylinder 125…Bucket cylinder 1241…Lift stroke sensor 1251…Bucket stroke sensor 1221…Tooth 300…Control device 310…Processor 330…Main memory 350…Storage 370…Interface 311…Captured image acquisition unit 312…Condition determination unit 313…Distance image generation unit 314…Extraction unit 315…Defect determination unit 316…Notification unit

Claims

1. A tooth defect detection system comprising a processor, wherein the processor acquires an image showing a bucket having teeth, extracts a region showing the bucket by removing the background portion from the image, detects a plurality of protruding points protruding from the bucket body within the region showing the bucket, and determines whether or not there are tooth defects based on the plurality of protruding points.

2. The tooth defect determination system according to claim 1, wherein the processor generates a depth image from the captured image and extracts a region in which the bucket is visible based on the distance image.

3. The tooth defect detection system according to claim 1, wherein the processor performs semantic segmentation of the captured image, generates a segmented image, and extracts a region in which the bucket is visible based on the segmented image.

4. The tooth loss detection system according to claim 1, wherein the processor extracts the region in which the bucket is depicted by a binarization process that divides the background portion and the region in which the bucket is depicted into different pixel values.

5. The tooth loss detection system according to claim 4, wherein the processor extracts a region in which the bucket is visible by trimming the portion of the binarized image in which the bucket does not exist.

6. The tooth defect determination system according to claim 1, wherein the processor identifies a second line obtained by offsetting a first line fitted to the plurality of protruding points toward the bucket body, and identifies the overlapping portion between the second line and the area in which the bucket is reflected, thereby determining whether or not there is a tooth defect.

7. The tooth defect detection system according to claim 1, wherein the processor determines that there is a tooth defect and notifies the system of the tooth defect.

8. A tooth defect determination method comprising: acquiring an image of a bucket having teeth; extracting a region in which the bucket is visible by removing the background portion from the image; and detecting a plurality of protruding points protruding from the bucket body from the region in which the bucket is visible, and determining whether or not there is a tooth defect based on the plurality of protruding points.

9. The tooth defect determination method according to claim 8, further comprising the step of generating a depth image from the captured image, wherein the step of extracting the region in which the bucket is visible is to extract the region in which the bucket is visible based on the distance image.

10. A tooth defect determination method according to claim 8, comprising the steps of performing semantic segmentation on the captured image and generating a segmentation image, wherein the step of extracting a region in which the bucket is visible is to extract a region in which the bucket is visible based on the segmentation image.

11. The tooth loss detection method according to claim 8, wherein in the step of extracting the area in which the bucket is visible, the area in which the bucket is visible is extracted by a binarization process that separates the background portion and the area in which the bucket is visible into different pixel values.

12. The tooth loss detection method according to claim 11, wherein in the step of extracting the region in which the bucket is visible, the region in which the bucket is visible is extracted by trimming the portion in which the bucket is not visible from the binarized image.

13. A tooth defect determination method according to claim 8, comprising the steps of: identifying a first line fitted to the plurality of protruding points; identifying a second line offset toward the bucket body; and identifying an overlapping portion between the second line and the area in which the bucket is captured, wherein the presence or absence of tooth defects is determined based on the identified overlapping portion in the step of extracting the area in which the bucket is captured.

14. The tooth defect detection method according to claim 8, further comprising the step of notifying the tooth defect when it is determined that a tooth defect exists.

Images