Image processing device and image processing method

Abstract

To improve the reliability of estimation results for engine pistons.SOLUTION: An image processing device 100 of the present invention includes: a learning device 5 trained using image data of an engine piston and training data associated with surface conditions of a piston ring or cleanliness of a ring land of the piston; a recognition unit 2 configured to input image data obtained by imaging an engine piston to be tested to the learning device 5 and recognize the piston to be tested; and an estimation unit 1 configured to estimate, based on recognition results of the recognition unit 2, surface conditions of the piston ring of the piston to be tested or cleanliness of the ring land of the piston to be tested.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image processing device and an image processing method. [Background technology]

[0002] In recent years, technological development for managing infrastructure structures, large facilities, large vehicles, etc. has been active, and related inventions have been made public. For example, Patent Document 1 discloses an inspection support device for inspecting structures. The inspection support device of Patent Document 1 includes a processor that organizes images containing damage to the structure. The processor acquires image data containing information including a structural diagram of the target structure on a medium and damage identification information and photographed image identification information related to damage added by a user to the medium. The processor also recognizes the damage identification information and photographed image identification information from the acquired image data through image recognition. The processor also associates the damage identification information and photographed image identification information for the same damage. The processor also acquires photographed images corresponding to the photographed image identification information and associates the damage identification information with the photographed images. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2023-57591 Summary of the Invention [Problem to be solved by the invention]

[0004] In conventional techniques, including the invention of Patent Document 1, inspections are performed using images of an inspection object placed on-site. However, imaging conditions at the site are not necessarily good, and images of the inspection object captured on-site are not necessarily clear. This limits the accuracy of image recognition of the inspection object, posing a problem in that it is not possible to improve the reliability of estimation results using images (e.g., estimation of the surface condition of the inspection object). In particular, when the inspection object is a piston of a diesel engine mounted on a ship's hull, there is a need to improve the false recognition of the piston in images of the piston in order to estimate the piston's surface condition, etc. From this perspective, an object of the present invention is to improve the reliability of estimation results for engine pistons. [Means for solving the problem]

[0005] The present invention, which solves the above problem, is an image processing device that includes a learning device trained using teacher data that associates image data of an engine piston with the surface condition of the piston rings of the piston or the cleanliness of the ring land of the piston; a recognition unit that inputs image data obtained by photographing an engine piston to be inspected into the learning device and recognizes the engine piston to be inspected; and an estimation unit that estimates the surface condition of the piston rings of the engine piston to be inspected or the cleanliness of the ring land of the engine piston to be inspected based on the recognition result of the recognition unit. The present invention also provides an image processing method in which an image processing device includes a learning device trained using teacher data relating image data of an engine piston and the surface condition of a piston ring of the piston or the cleanliness of a ring land of the piston, and executes a recognition step of inputting image data obtained by photographing an engine piston to be inspected into the learning device and recognizing the piston, and an estimation step of estimating the surface condition of a piston ring of the piston to be inspected or the cleanliness of a ring land of the piston to be inspected based on the recognition result of the recognition step. This configuration enables image recognition to identify the number of piston rings and the number of stages of ring lands for the piston under inspection, thereby reducing misrecognition of the image of the piston under inspection. When making an estimation using a learning device, the reliability of the estimation result can be improved by using the recognition result of the piston under inspection.

[0006] It is also preferable that the recognition unit obtains, as the recognition result, an arrangement order of ring lands detected from image data of the piston to be inspected. According to this configuration, when estimation is performed by the learning device, the order of the ring lands detected from the image data is used, thereby making it possible to further improve the reliability of the estimation result.

[0007] It is also preferable that the recognition unit performs a clustering process on the ring lands detected from the image data of the piston to be inspected, recognizes the number of stages of the ring lands of the piston to be inspected, and determines the arrangement order according to the recognized number of stages. According to this configuration, the clustering process can be used to recognize ring lands that fit within the angle of view of the image data as ring lands of the same piston. This improves the accuracy of recognizing the number of stages of the ring lands of the piston being inspected. Along with the improved accuracy of recognizing the number of stages of the ring lands, the accuracy of recognizing the order of the ring lands can also be improved. As a result, by using the order, the reliability of the estimation results can be further improved.

[0008] It is preferable that the recognition unit adopts a sorting order according to the recognition results or clustering results for each piston ring or ring land. According to this configuration, by using the sorting order, it is possible to further improve the reliability of the estimation result.

[0009] Preferably, the inspection apparatus further comprises an extraction unit that extracts feature quantities of the inspection target piston from image data obtained by photographing the inspection target piston. This configuration makes it easy to identify the piston rings and ring lands of the piston based on the feature extraction results. Therefore, when making an estimation using a learning device, the reliability of the estimation result can be further improved by also using the feature extraction results.

[0010] Preferably, the image data of the piston to be inspected is image data obtained by photographing a part of the piston to be inspected through a scavenging port. This configuration enables image recognition to identify the number of rings and the number of stages of the ring lands even for a piston that is only partially visible through the scavenging port. Therefore, when making an estimation using a learning device, the reliability of the estimation result can be improved by using the recognition result of the piston that is being inspected.

[0011] It is also preferable that the image data obtained by photographing the piston to be inspected is image data that shows at least three of the piston rings and four stages of the ring lands. This configuration increases the probability of distinguishing between a piston with three piston rings and four ring lands and a piston with four piston rings and five ring lands, thereby further improving the reliability of the estimation results.

[0012] Preferably, the estimation unit simultaneously estimates the surface condition of the piston ring of the inspection target piston and the cleanliness of the ring land of the inspection target piston. According to this configuration, a comprehensive evaluation of the piston ring and the ring land can be performed.

[0013] It is also preferable that the estimation unit estimates the image quality of the image data simultaneously with the surface condition of the piston ring of the piston to be inspected, or estimates the image quality of the image data simultaneously with the cleanliness of the ring land of the piston to be inspected. According to this configuration, it is possible to exclude estimation of the surface condition of the piston ring and the cleanliness of the ring land from images that are unclear, thereby further improving the reliability of the estimation results. [Effects of the Invention]

[0014] According to the present invention, it is possible to improve the reliability of the estimation results for the pistons of the engine. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a functional configuration diagram of an image processing apparatus according to an embodiment of the present invention. [Figure 2] This is an overall view of a piston with four rings and five ring lands. [Figure 3] This is an overall view of a piston with three rings and four ring lands. [Figure 4] This is an example of image data of a piston with four rings and five ring lands, photographed through the scavenging port. [Figure 5] This is an example of image data of a piston with three rings and four ring lands, photographed through the scavenging port. [Figure 6] This is an example of image data of a portion of a piston with four rings and five ring lands, photographed through the scavenging port. [Figure 7] 10 is an example of a flowchart showing a process for estimating the surface condition of the ring of the inspected piston and the cleanliness of the ring land of the inspected piston. [Figure 8] 10 is a flowchart showing a process of determining the order of ring lands; [Figure 9] 10 is a flowchart showing a clustering process. [Figure 10] FIG. 10 is an explanatory diagram of erroneous recognition of a piston with three rings and four ring lands in a comparative example. [Figure 11] 10 is an explanatory diagram of the recognition of a piston with three rings and four ring lands according to this embodiment. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Each drawing is merely a schematic illustration to allow a sufficient understanding of the present invention. Therefore, the present invention is not limited to the illustrated examples. In each drawing, common or similar components are designated by the same reference numerals, and redundant explanations thereof will be omitted. The image processing device and image processing method of the present invention are not particularly limited as long as the inspection object is an engine piston, but in this embodiment, a piston of a diesel engine mounted on a ship's hull will be described as an example.

[0017] [composition] FIG. 1 is a functional configuration diagram of the image processing apparatus of this embodiment. The image processing device 100 is a computer equipped with hardware such as an input unit, an output unit, a control unit, and a storage unit. For example, if the control unit is configured with a CPU (Central Processing Unit), information processing by a computer including the control unit is realized by program execution processing by the CPU. Furthermore, a storage unit included in the computer stores various programs for realizing the functions of the computer in response to instructions from the CPU. This realizes collaboration between software and hardware. The programs can be recorded on a recording medium or provided via a network. The storage unit may also be implemented as a cloud. A console can be communicatively connected to the image processing device 100, and the console can display the processing details of the image processing device 100. For example, the console can display image data generated by image processing by the image processing device 100 on a screen.

[0018] 1, the image processing device 100 includes an estimation unit 1, a recognition unit 2, an extraction unit 3, and a display control unit 4. The image processing device 100 also stores a learning device 5 and a teacher DB 6.

[0019] The estimation unit 1 estimates the surface condition of the piston rings (also simply referred to as "rings") of the piston to be inspected or the cleanliness of the ring land of the piston to be inspected based on image data obtained by photographing the piston in an engine. For example, the estimation unit 1 can estimate the state of scuffing (scratches) as the surface condition of the ring. The scuffing state can be classified into two levels, for example, normal (no scratches) and scuffed (scratches present). Furthermore, for example, the estimation unit 1 can estimate the state of the ring land with soot attached as the cleanliness of the ring land. The amount of attached soot can be classified into four levels, for example, excessive (large amount of soot attached), light carbon (small amount of soot attached), discolor (discoloration of attached soot), and normal (no attachment).

[0020] The piston to be inspected may be photographed, for example, by an image capturing device, but is not limited to this. The image capturing device may be, for example, a digital camera, a digital video camera, or a tablet terminal with a camera function, but is not limited to these. Furthermore, multiple image capturing devices may capture images of the same piston to be inspected. In this case, the imaging angles of the multiple image capturing devices may be the same or different.

[0021] The recognition unit 2 recognizes the inspection target piston from image data obtained by photographing the engine inspection target piston. For example, the recognition unit 2 can recognize the inspection target piston using a learning device 5. Specifically, image data obtained by photographing the engine inspection target piston may be input to the learning device 5, and the prediction results of the inspection target piston (e.g., the position and number of stages of the ring land, cleanliness, position and number of stages of the ring, surface condition, etc.) output from the learning device 5 may be used as the recognition result of the recognition unit 2. The recognition unit 2 may also be implemented as a predetermined image analysis algorithm. The image quality of the image portion of the ring land or the image portion of the ring in the image data may also be used as the recognition result of the recognition unit 2. For example, the SNR (Signal to Noise Ratio) of the target image portion can be used as an index of image quality, but is not limited to this. The estimation unit 1 estimates the surface condition of the ring of the piston to be inspected or the cleanliness of the ring land of the piston to be inspected based on the recognition result of the recognition unit 2.

[0022] The extraction unit 3 extracts feature quantities of the inspection target piston from image data obtained by photographing the inspection target piston. For example, the extraction unit 3 can extract the boundary line (edge) between the ring and ring land of the inspection target piston as a feature quantity. The feature amount extracted by the extraction unit 3 may be included in the recognition result of the recognition unit 2, and the extraction unit 3 and the recognition unit 2 do not need to be separate units.

[0023] The display control unit 4 controls the display of predetermined display content. Examples of predetermined display content include, but are not limited to, image data obtained by photographing the piston to be inspected and estimation results by the estimation unit 1 (surface condition of the ring, cleanliness of the ring land, etc.). For example, the display control unit 4 can cause the display content to be displayed on a display unit serving as an output unit of the image processing device 100. Furthermore, the display control unit 4 can cause the display content to be displayed on a console capable of communicating with the image processing device 100.

[0024] The learning device 5 is a machine learning algorithm trained using image data of the engine piston and training data that associates the surface condition of the piston ring or the cleanliness of the piston ring land. The training data can be created, for example, using image data obtained by photographing a piston whose ring surface condition and ring land cleanliness are known. Specifically, the image data is divided into an image region of the ring, an image region of the ring land, and other image regions. The division method may be a well-known method. A surface condition label indicating the surface condition of the ring, a ring stage number label indicating the number of stages in the ring, and an image quality label indicating the image quality can be assigned to the image portion of the ring. For example, "normal" and "scuffing" can be prepared as surface condition labels. For example, "1" to "4" (the numbers indicate the order when arranged vertically) can be prepared as ring stage labels. For example, "good" (clear) and "poor image quality" (unclear) can be prepared as image quality labels. One or more types of labels can be assigned to the same image portion of the ring. A cleanliness label indicating the cleanliness of the ring land, a ring land stage label indicating the number of stages in the ring, and an image quality label indicating the image quality can be assigned to the image portion of the ring land. For example, cleanliness labels can be "excessive," "light carbon," "discolor," and "normal." For example, ring land stage labels can be "1" to "5" (the numbers indicate the order when arranged vertically). For example, image quality labels can be "good" (clear) or "poorly photographed" (unclear). One or more types of labels can be assigned to the same image portion of the ring land. The training data can be configured in such a manner that one or more types of labels are assigned to various image partial regions of the image data of the piston. The machine learning algorithm can be, for example, but not limited to, k-nearest neighbors, logistic regression, support vector machine, neural network, Gaussian mixture model, and the like.

[0025] The teacher DB6 is a database that stores teacher data. Furthermore, the estimation results estimated by the estimation unit 1 for the piston to be inspected can be diverted as teacher data, and the diverted teacher data can be stored in the teacher DB6. The learning device 5 can be trained using the diverted teacher data. In this case, parameters (tuning parameters, hyperparameters) determined by the learning device 5, to which image data of the piston to be inspected is input, at the time of prediction may be included in the diverted teacher data.

[0026] [Piston configuration] The diesel engine mounted on the hull may be, for example, a large uniflow scavenging two-cycle diesel engine, which is the main propulsion engine for the ship. Two-cycle diesel engines are also called two-stroke engines. The pistons used in such diesel engines are either one with four rings and five ring lands, or one with three rings and four ring lands.

[0027] Fig. 2 is an overall view of a piston with four rings and five ring lands. As shown in Fig. 2, the piston 20 is generally cylindrical and slides up and down within a cylindrical cylinder 10 that extends vertically. Note that a diesel engine is equipped with multiple cylinders 10 and multiple pistons 20, but Fig. 2 shows only one of each.

[0028] The piston 20 has annular grooves formed in the circumferential direction, and annular rings 21-24 are fitted onto each groove. Therefore, five ring lands 25-29 are formed as areas of the entire circumferential surface of the piston 20 where no grooves are formed. The piston 20 is attached to a pedestal 12 that forms the upper end of the piston rod 11. The lower side of the fifth ring land 29 forms the pedestal 12.

[0029] In the space within the cylinder 10 above the piston 20, soot is generated when fuel burns in accordance with the sliding of the piston 20. Most of the generated soot is exhausted from the top of the cylinder 10 in accordance with the opening and closing of an exhaust valve (not shown), but some of it adheres to the ring lands 25 to 29. In particular, a relatively large amount of soot tends to adhere to the first-stage ring land 25.

[0030] The cylinder 10 has a plurality of scavenging ports 13 on the circumferential surface of the lower part. The scavenging ports 13 are openings for supplying air, which is supplied via a scavenging pipe (not shown) large enough for a person to enter, into the cylinder 10. When the piston 20 slides inside the cylinder 10, it crosses the scavenging ports 13 in the vertical direction.

[0031] For example, a photographer such as a crew member or maintenance worker of a ship approaches the bottom of the cylinder 10 when the engine is stopped and photographs the piston 20 inside the cylinder 10 through the scavenging ports 13. The vertical position of the piston 20 when the engine is stopped can be adjusted as appropriate. When photographing, the piston 20 is adjusted to be located at a predetermined inspection position. For example, the predetermined inspection position can be a position where the piston 20 can be seen through the scavenging ports 13. Specifically, the predetermined inspection position can be, but is not limited to, the lower limit position of the movement range of the piston 20 or a position a predetermined distance above the lower limit position.

[0032] FIG. 3 is an overall view of a piston with three rings and four stages of ring lands. As shown in FIG. 3, piston 30 is generally cylindrical and slides up and down inside a cylindrical cylinder 10 that extends vertically. Piston 30 has annular grooves formed in the circumferential direction, and annular rings 31 to 33 are fitted into each groove. Therefore, four stages of ring lands 34 to 37 are formed in the areas of the entire circumferential surface of piston 30 where no grooves are formed. The assembly structure of piston 30 inside cylinder 10 is the same as that of piston 20 described with reference to FIG. 2, and therefore will not be described here.

[0033] [Photographed image data] FIG. 4 shows an example of image data of a piston with four rings and five stages of ring lands, photographed through a scavenging port. The display control unit 4 can display image data 40. The image data 40 includes at least one scavenging port 13 and a portion of the piston 20 visible through the scavenging port 13. In FIG. 4, three scavenging ports 13 are shown. For convenience of illustration, parts of the scavenging port 13 that cannot be seen (the piston 20, the piston rod 11, and the base portion 12) are shown with dashed lines. The scavenging port 13 is an elongated hole extending in the sliding direction of the piston 20. As shown in FIG. 4, the piston 20 seen through the scavenging port 13 has rings 21 to 24 and ring lands 25 to 29 alternately arranged in the vertical direction. The rings 21 to 24 and the ring lands 25 to 29 are visible as a roughly rectangular shape, an upper semicircular shape, and a lower semicircular shape. The scavenging port 13 is tapered so that the hole diameter gradually decreases toward the inside in the radial direction of the cylinder 10, and this shape improves scavenging performance. Therefore, the hole wall of the scavenging port 13 can be seen in the image data 40.

[0034] 5 is an example of image data of a piston with three rings and four stages of ring lands photographed through a scavenging port. The display control unit 4 can display image data 50. As in the case of FIG. 4, in the image data 50, the piston 30 seen through the scavenging port 13 is visually recognized with rings 31 to 33 and ring lands 34 to 37 alternately arranged in the vertical direction.

[0035] FIG. 6 is an example of image data of a portion of a piston with four rings and five stages of ring lands, photographed through a scavenging port. The display control unit 4 can display image data 60. The image data 60 corresponds to image data photographed at a position where the piston 20 has moved upward by a predetermined distance with respect to the image data of FIG. 4. As shown in FIG. 6, the first stage ring land 25 and the first ring 21, which are located above the top of the scavenging port 13, cannot be seen through the scavenging port 13, but the rings 22 to 24 and ring lands 26 to 29 can be seen lined up alternately in the vertical direction.

[0036] 5, the image data 60 is similar to the image data 50 in that three rings and four stages of ring lands are visible through the scavenging ports 13. For this reason, it is not easy to distinguish from the image data 60 whether it is a part of the piston 20 with four rings and five stages of ring lands, or the entire piston 30 with three rings and four stages of ring lands. Similarly, it is not easy to distinguish from the image data 50 whether it is a part of the piston 20 with four rings and five stages of ring lands, or the entire piston 30 with three rings and four stages of ring lands. The present invention makes it possible to make such distinctions.

[0037] [process] The following describes the processing of the image processing device 100 of this embodiment. Fig. 7 is an example of a flowchart showing the processing for estimating the surface condition of the ring of the piston to be inspected and the cleanliness of the ring land of the piston to be inspected.

[0038] First, the recognition unit 2 inputs image data of the piston to be inspected to the learning device 5 (step A1). The image data is image data captured through the scavenging port 13, such as image data 40 to 60 shown in FIGS. 4 to 6. The recognition unit 2 obtains the prediction result of the piston to be inspected output from the learning device 5 and sets it as the recognition result of the recognition unit 2. Next, the extraction unit 3 performs edge extraction processing on the image data of the piston to be inspected (step A2). Specifically, the extraction unit 3 extracts the boundary line (edge) between the ring and ring land of the piston to be inspected as a feature. The feature extracted by the extraction unit 3 is included in the recognition result of the recognition unit 2.

[0039] Next, the recognition unit 2 detects the ring lands of the piston to be inspected from the recognition result of the piston to be inspected (step A3). For example, in the case of image data 40 (FIG. 4), the image partial area of ​​the ring lands visible through the scavenging ports 13 is identified. At least three scavenging ports 13 are photographed in the image data 40, and five stages of ring lands 25 to 29 are visible for each scavenging port 13. Therefore, a total of 15 ring land portions are visible. If all 15 ring land portions are photographed clearly, the recognition unit 2 can recognize all of them, and a total of 15 image partial areas of the ring lands can be detected. The process of FIG. 7 can also be performed on image data of an inspection target piston in which two or less scavenging ports 13 are shown, or on image data of an inspection target piston in which four or more scavenging ports 13 are shown.

[0040] Next, the recognition unit 2 performs an order determination process on the detected ring lands (step A4). The order determination process will be described later. As a result, the recognition unit 2 can determine whether the piston being inspected has four rings and five ring lands (see Figure 2), or three rings and four ring lands (see Figure 3). The recognition unit 2 can also determine the order of the ring lands, i.e., the order of the ring lands. The determination result of step A4 is included in the recognition result of the recognition unit 2. In addition, since the rings of the piston under inspection are located between the ring lands, if the ring lands are detected, the rings can also be detected. Therefore, the recognition unit 2 can determine the order of the detected rings. The determination result of the ring order is also included in the recognition result of the recognition unit 2. This can also be reversed. In other words, the rings can be detected and the space between them can be considered as the ring lands, or both can be detected at the same time.

[0041] Next, the estimation unit 1 outputs the cleanliness of each stage of the ring land of the inspected piston based on the recognition result of the recognition unit 2 (step A5). Specifically, the estimation unit 1 inputs image data of the inspected piston to the learning device 5 and uses the prediction result, feature values ​​(edges), and the order of the ring lands output from the learning device 5 to output the cleanliness of each detected ring land, which is used as the estimation result of the estimation unit 1. For example, the estimation result for the ring land can be in a form in which a cleanliness label, a ring land stage number label, and an image quality label are assigned to each image partial region of the detected ring land. The estimation unit 1 can estimate the cleanliness of the ring land of the inspected piston and the image quality of the image data at the same time. Therefore, it is possible to exclude the estimation of the ring land cleanliness for unclear images, etc., thereby further improving the reliability of the estimation results. "Clear" refers to, for example, the area to be judged being too small, out of focus, camera shake, low resolution, insufficient light, or not cleaned. Unclearness can be determined using machine learning or image processing.

[0042] Next, the estimation unit 1 outputs the surface condition of each of the rings of the piston under inspection based on the recognition result of the recognition unit 2 (step A6). Specifically, the estimation unit 1 inputs image data of the piston under inspection into the learning device 5 and uses the prediction result, feature values ​​(edges), and ring arrangement order output from the learning device 5 to output the surface condition of each of the detected rings, as the estimation result of the estimation unit 1. For example, the estimation result for the rings can be in a form in which a surface condition label, a ring stage number label, and an image quality label are assigned to each of the detected image partial regions of the rings. The estimation unit 1 can estimate the surface condition of the ring of the inspected piston and the image quality of the image data at the same time. Therefore, as in the case of the cleanliness of the ring land, estimation of the surface condition of the piston ring can be excluded for unclear images, etc., and the reliability of the estimation result can be further improved. This completes the processing in FIG.

[0043] (Details of step A4: sort order determination process) FIG. 8 is a flowchart showing the process of determining the arrangement order of ring lands. First, the recognition unit 2 performs clustering processing on the ring lands of the piston being inspected detected in step A3 (step B1). The clustering processing will be described later. As a result, the number of stages of the ring lands detected in step A3 (FIG. 7) can be recognized. For example, in the case of image data 40 (FIG. 4), the recognition unit 2 performs clustering processing on the image subregions in which a total of 15 ring lands are detected.

[0044] Next, the recognition unit 2 determines whether the number of stages of the ring lands recognized by the clustering process is five (step B2). If there are five stages (Yes in step B2), the recognition unit 2 performs a numbering process on the ring lands (step B3). In this case, the recognition unit 2 determines that the piston to be inspected has four rings and five stages of ring lands (see FIG. 2), and recognizes ring lands 25 to 29. In the numbering process, the recognition unit 2 assigns numbers 1 to 5 to the recognized ring lands 25 to 29 (numbering from top to bottom). This allows the recognition unit 2 to recognize the ring lands 25 to 29 as the first stage to the fifth stage, respectively. After the numbering process (step B3), the process of FIG. 8 ends. The numbering process may be performed using the stage number label, which is the prediction result output from the learning device 5. In this case, if the result of the numbering process is compared with the output stage number label and is the same, the stage number label may be used as is. In this embodiment, the stage number indicated by the result of the numbering process is used.

[0045] On the other hand, if the number of ring lands is not five (No in step B2), the recognition unit 2 determines whether the number of ring lands recognized by the clustering process is three or less (step B4). If the number of ring lands is three or less (Yes in step B4), the recognition unit 2 performs an exclusion process (step B5). That is, if the piston under test is located at a fairly high position, the image area of ​​the piston under test visible through the scavenging port 13 in the image data is too small, and the recognition unit 2 can no longer determine whether the piston under test has four rings and five ring lands (see FIG. 2) or three rings and four ring lands (see FIG. 3). Such image data is not used for estimation by the learning unit 5. That is, steps A5 and A6 (FIG. 7) are not performed. After the exclusion process (step B5), the process of FIG. 8 ends.

[0046] On the other hand, if the number of ring lands is more than three (No in step B4), the number of ring lands recognized by the clustering process is four. In this case, the recognition unit 2 calculates the likelihood LHH of the recognized top ring land (step B6). The "top ring land" refers to the ring land located highest in the vertical direction (the direction of piston movement) among the ring lands detected from the two-dimensional image data. The likelihood LHH is a parameter indicating the likelihood that the "top ring land" is the first-stage ring land 25 when the piston under test has four rings and five stages of ring lands (see FIG. 2). The likelihood LHH is a parameter indicating the likelihood that the "top ring land" is the first-stage ring land 34 when the piston under test has three rings and four stages of ring lands (see FIG. 3). The likelihood calculation may be, for example, a method of obtaining the output of a softmax function, but is not limited to this. The first-stage ring lands 25, 34 have a distinctive shape among the overall shapes of the piston, and are also easily visible through the upper semicircle at the top of the scavenging port 13, so they can be effectively used for judgment using the likelihood LHH. The likelihood is calculated in the learning device 5. The likelihood LHH can be obtained as a prediction result of the learning device 5. When outputting a stage number label for each ring land, the learning device 5 can also output the likelihood at the same time.

[0047] Next, the recognition unit 2 calculates the likelihood LHL of the recognized bottom ring land (step B7). The "bottom ring land" refers to the ring land located at the bottom in the vertical direction among the ring lands detected from the two-dimensional image data. The likelihood LHL is a parameter indicating the likelihood that the "bottom ring land" is the fifth-stage ring land 29 when the piston to be inspected has four rings and five stages of ring lands (see FIG. 2). The likelihood LHL is a parameter indicating the likelihood that the "bottom ring land" is the fourth-stage ring land 37 when the piston to be inspected has three rings and four stages of ring lands (see FIG. 3). The fifth-stage ring land 29 and the fourth-stage ring land 37 are easily visible through the lower semicircle at the bottom of the scavenging port 13, and therefore can be effectively utilized for determination using the likelihood LHL. The likelihood LHL can be obtained as a prediction result of the learning device 5.

[0048] Next, the recognition unit 2 determines whether the likelihoods LHH and LHL are approximately the same (step B8). The "approximately" criterion is merely a design consideration. If they are approximately the same (Yes in step B8), the recognition unit 2 identifies the piston under test as a piston with three rings and four ring lands (see FIG. 3) (step B9). In this case, the number of recognized ring lands is four, and there is a fairly high possibility that the recognized "top ring land" is the topmost ring land, and there is a fairly high possibility that the recognized "bottom ring land" is the bottommost ring land. Therefore, if ring lands 34 to 37 in the first to fourth ring lands of a piston with three rings and four ring lands are recognized, a match is achieved. In the numbering process, the recognition unit 2 assigns numbers 1 to 4 to the recognized four ring lands. As a result, the recognition unit 2 estimates that the ring lands numbered 1 to 4 are the first to fourth ring lands 34 to 37 in the piston with three rings and four ring lands. After the identification (step B9), the processing of FIG. 8 ends. If the amount of training data (teaching data) for the recognized four-stage ring land is insufficient, there is a possibility that the recognition using the stage number label output by the learning device 5 will be erroneous. In such a case, it is effective to use the process of step B8.

[0049] On the other hand, if the likelihoods LHH and LHL are not substantially equal (No in step B8), the recognition unit 2 determines whether the likelihood LHH is sufficiently large (step B10). The criterion for determining whether it is "sufficient" is merely a design consideration. If it is sufficiently large (Yes in step B10), the recognition unit 2 performs a numbering process on the ring lands in ascending order (step B11). In this case, the number of recognized stages is four, and the recognized "topmost ring land" is highly likely to be the topmost ring land, while the recognized "bottommost ring land" is highly unlikely to be the bottommost ring land. Therefore, if we assume that the first to fourth ring lands 25 to 28 of the piston with four rings and five ring lands have been recognized (the fifth ring land 29 was not visible through the scavenging port 13), a match is achieved. In the numbering process, the recognition unit 2 assigns numbers 1 to 4 to the recognized four ring lands in ascending order from top to bottom. As a result, the recognition unit 2 estimates that the ring lands numbered 1 to 4 are the first to fourth ring lands 25 to 28 in the piston with four rings and five ring lands. After the numbering process (step B11), the process in FIG. 8 ends.

[0050] On the other hand, if the likelihood LHH is not sufficiently large (No in step B10), the recognition unit 2 determines whether the likelihood LHL is sufficiently large (step B12). The criterion of "sufficient" is merely a design consideration. If it is sufficiently large (Yes in step B12), the recognition unit 2 assigns numbers to the ring lands in descending order (step B13). In this case, the number of recognized stages is four, and the recognized "top ring land" is extremely unlikely to be the top ring land, and the recognized "bottom ring land" is extremely likely to be the bottom ring land. Therefore, if we assume that the second to fifth ring lands 26 to 29 of the piston with four rings and five ring lands have been recognized (the first ring land 25 was not visible through the scavenging port 13), a match is achieved. In the numbering process, the recognition unit 2 assigns numbers 5 to 2 to the recognized four ring lands, starting from the bottom. As a result, the recognition unit 2 estimates that the ring lands numbered 5 to 2 are ring lands 29 to 26 in the fifth to second stages of the piston with four rings and five ring lands. After the numbering process (step B13), the process in FIG. 8 ends.

[0051] On the other hand, if the likelihood LHL is not sufficiently large (No in step B12), the recognition unit 2 performs an exclusion process (step B5). In this case, since there is little basis for determining the shape of the piston in the image data, such image data is not used for estimation by the learning device 5. In other words, steps A5 and A6 (FIG. 7) are not performed. This completes the processing in FIG. After the process of FIG. 8 is completed, the process proceeds to step A5 (FIG. 7).

[0052] (Clustering process: details of step B1) FIG. 9 is a flowchart showing the clustering process. First, the recognition unit 2 calculates the centers of gravity of all ring lands of the piston to be inspected detected in step A3 (step Step C1). For example, in the case of two-dimensional image data 40 (Fig. 4), a total of 15 ring land portions visible through the scavenging port 13 are detected. The recognition unit 2 calculates the center of gravity (two-dimensional coordinates) of each image partial area surrounding the ring land portion. Note that the method of setting the coordinate axes of the two-dimensional coordinates in the image data is a design matter. Note that if the image of the ring land portion is unclear, it may be excluded from the calculation of the center of gravity.

[0053] Next, the recognition unit 2 selects two of the calculated centroids and performs the loop process of steps C3 to C5 for each of the two selected centroids (step C2). For example, if 15 ring land portions are detected in the two-dimensional image data 40 (FIG. 4) and 15 centroids are calculated, the options for the two centroids are 105 (= 15 In other words, the recognition unit 2 performs the loop process of step C2 105 times.

[0054] Next, the recognition unit 2 calculates the distance between a line passing through the two selected centroids and each of the non-selected centroids in the image data (step C3). For example, suppose 15 centroids are calculated and two are randomly selected from the 15. In this case, the recognition unit 2 finds the equation of the line passing through the two selected centroids. Furthermore, for the 13 non-selected centroids, the recognition unit 2 calculates a total of 13 distances between the centroids and the lines using the two-dimensional coordinates of the centroids and the equation of the line.

[0055] Next, the recognition unit 2 extracts and registers non-selected centroids whose calculated distances are equal to or less than a predetermined threshold (step C4). The registration is temporary, and a suitable memory can be prepared for the registration. For example, in the case of two-dimensional image data 40 ( FIG. 4 ), the centroids of 15 ring land portions are calculated, two centroids are selected to obtain a line, and the distances between each of the 13 non-selected centroids and the line are calculated. In this case, if the calculated distances are equal to or less than the threshold, all 13 non-selected centroids are registered. However, if there is a non-selected centroid whose distance exceeds the threshold, that centroid is not registered and is not used in subsequent calculations. The threshold is preferably set to a value that can ensure, for example, that ring lands that fit within the angle of view of the image data are ring lands visible from the same scavenging port 13. If multiple centroids of image partial regions surrounding the ring land portions are obtained from the image data, and one of the obtained centroids provides a distance exceeding the threshold, it is possible that a ring land of a piston other than the inspected piston of another scavenging port 13 fits within the angle of view of the image data. Such centroids will not be used in further calculations.

[0056] Next, the recognition unit 2 calculates the RMS (Root Mean Square): Lc of the distance for the registered non-selected centroids (step C5). For example, for two-dimensional image data 40 (FIG. 4), the centroids of 15 ring land portions are calculated, two points passing through a straight line are selected, and all 13 non-selected centroids are registered. In this case, 13 distances corresponding to the 13 non-selected centroids are calculated. Furthermore, Lc is calculated from the calculated 13 distances. Note that Lc can be an example of a parameter that indicates an error contained in the recognition result of the piston to be inspected. As a result of the loop processing (step C2), 105 Lc values ​​are calculated.

[0057] After the loop process (step C2), the recognition unit 2 assigns the threshold value selected in step C4 to the variable Lmin (step C6). Next, the recognition unit 2 selects the combination of centroids that minimizes the Lc calculated in step C5 (step C7). For example, assume that the centroids of 15 ring land portions are calculated for the two-dimensional image data 40 (FIG. 4). In this case, the recognition unit 2 selects the combination of centroids that minimizes the Lc from the 105 calculated Lc (the combination of the two centroids selected to draw the line and the 13 unselected centroids).

[0058] Next, the recognition unit 2 determines whether Lc selected in step C7 is smaller than Lmin (step C8). If it is not smaller (No in step C8), the recognition unit 2 performs an exclusion process (step C11). In other words, even if the smallest value of the multiple calculated Lc values ​​exceeds the threshold value prepared in step C4, the error contained in the recognition result of the inspected piston for the image data cannot be ignored. Such image data is not used for estimation by the learning device 5. After the exclusion process (step C11), the processing of FIG. 9 ends.

[0059] On the other hand, if it is smaller (Yes in step C8), the recognition unit 2 registers the image partial region surrounding the multiple ring land portions corresponding to the combination of centroids selected in step C7 as a cluster candidate (step C9). Next, the recognition unit 2 analyzes the registered cluster candidates and recognizes the number of stages of the ring lands detected in step A3 (FIG. 7) (step C10). The analysis of clusters is an analysis of geometric information of image subregions surrounding multiple ring lands, but since this is well known, detailed explanation will be omitted.

[0060] This completes the processing in Figure 9. Then, proceed to step B2 (Figure 8). Of course, this clustering algorithm is just one example, and it is also possible to perform clustering using the k-means method or the like, using the center of gravity coordinates and sizes of the rings and ring lands as feature quantities. By the clustering process (step B1), it is possible to recognize that ring lands that fall within the angle of view of the image data obtained by photographing belong to the same piston.

[0061] [Specific example] Fig. 10 is an explanatory diagram of the erroneous recognition of a piston with three rings and four stages of ring lands in a comparative example. Fig. 10 corresponds to the visualization by the display control unit 4 of the recognition result of the recognition unit 2 in the case where the arrangement order determination process (step A4) is not performed on the image data 50 of Fig. 5 in the estimation process shown in Fig. 7. The numerical values ​​enclosed by the rectangular lines shown for each of the ring lands 34 to 37 indicate the stage in which the ring land portion detected by the recognition unit 2 is located. Note that the portion that could not be detected as a ring land is not displayed with a boxed numerical value. Also, displaying the boxed numerical value is optional.

[0062] For example, as shown in Figure 10, the third ring land 36 is assigned a number "4" enclosed in a rectangular box, and the recognition unit 2 erroneously recognizes it as the fourth ring land. Furthermore, although the piston 30 only has four ring land stages, the fourth ring land 37 is assigned a number "5" enclosed in a rectangular box, and the recognition unit 2 erroneously recognizes it as the fifth ring land, and erroneously recognizes the piston under test as having four rings and five ring lands. In this case, the reliability of the estimation result by the estimation unit 1 is low because the configuration of the piston itself, which is the premise of the estimation, is incorrectly recognized.

[0063] FIG. 11 is an explanatory diagram of the recognition of a piston with three rings and four stages of ring lands according to this embodiment. FIG. 11 corresponds to the visualization by the display control unit 4 of the recognition results of the recognition unit 2 when the arrangement order determination process (step A4) is performed on the image data 50 of FIG. 5 in the estimation process shown in FIG. 7. Compared to FIG. 10, the numbers "1" to "4" enclosed in rectangular boxes are assigned to the first to fourth stages of ring lands 34 to 37, indicating that they have been correctly recognized. Furthermore, the estimation results by the estimation unit 1 generally agree with the results of observations separately conducted by the applicant, and the reliability of the estimation results is high.

[0064] [effect] According to this embodiment, it is possible to improve the reliability of the estimation results for the pistons of the engine. More specifically, image recognition is possible to identify the number of rings and the number of stages of the ring lands for the piston under inspection, reducing misrecognition of the image of the piston under inspection. When making an estimation using a learning device, the reliability of the estimation result can be improved by using the recognition results of the piston under inspection. Furthermore, when making an estimation using a learning device, the reliability of the estimation results can be further improved by using the order of the ring lands detected from the image data. Furthermore, clustering processing can be used to recognize ring lands that fit within the angle of view of the image data as ring lands of the same piston. This improves the accuracy of recognizing the number of stages of the ring lands of the piston being inspected. Along with the improved accuracy of recognizing the number of stages of the ring lands, the accuracy of recognizing the order of the ring lands can also be improved. As a result, using the order of the stages can further improve the reliability of the estimation results. Furthermore, piston rings and ring lands can be easily identified based on the feature extraction results. Therefore, when making an estimation using a learning device, the reliability of the estimation results can be further improved by also using the feature extraction results. Furthermore, even if only a portion of the piston under inspection is visible through the scavenging port, image recognition is possible to identify the number of rings and the number of stages of the ring lands. Therefore, when making an estimation using a learning device, the reliability of the estimation result can be improved by using the recognition results of the piston under inspection. Furthermore, it is possible to increase the probability of distinguishing between a piston with three piston rings and four ring lands and a piston with four piston rings and five ring lands, thereby further improving the reliability of the estimation results. In addition, a comprehensive evaluation of the piston ring and ring land can be performed. Furthermore, the reliability of the evaluation of the piston ring and the evaluation of the ring land can be improved. In addition, for unclear images, estimation of the surface condition of the piston ring and the cleanliness of the ring land can be excluded, further improving the reliability of the estimation results.

[0065] [others] (a): The engine using the piston to be inspected is not limited to a diesel engine mounted on a ship, but may be a diesel engine mounted on other types of mobile bodies. Also, it is not limited to a diesel engine, but may be a gasoline engine or a hybrid engine. (b): In this embodiment, the arrangement order determination process for the ring lands (see FIG. 8) has been described. However, an arrangement order determination process for the rings may also be introduced. In this case, the recognition unit 2 detects the rings from the image data of the piston to be inspected, performs an arrangement order determination process on the detected rings, and recognizes the number of stages and order. The recognition result for the rings can be used for the estimation by the estimation unit 1. (c) As training data, image data of the piston taken through the scavenging port may be prepared in association with the surface condition of the piston ring or the cleanliness of the piston ring land. (d): In this embodiment, the learning device 5 has been described as a machine learning algorithm that predicts the surface condition of the piston ring and the cleanliness of the piston ring land. However, the learning device that predicts the surface condition of the piston ring and the learning device that predicts the cleanliness of the piston ring land may be implemented separately. (e): When the recognition unit 2 recognizes the search target piston, separate models for recognizing the image quality, surface condition, position, and number of stages of the rings of the search target piston may be prepared, but it is preferable to use the same model. Also, separate models for recognizing the image quality, cleanliness, position, and number of stages of the ring lands of the search target piston may be prepared, but it is preferable to use the same model. (f): The estimation unit 1 may simultaneously estimate the surface condition of the piston ring of the inspected piston and the cleanliness of the ring land of the inspected piston. In other words, the processes of steps A6 and A7 in the process of FIG. 7 may be performed simultaneously. This allows for a comprehensive evaluation of the piston ring and the ring land. (g): In this embodiment, the recognition result of the recognition unit 2 includes the image quality of the image portion of the ring land and the image portion of the ring in the image data of the search target piston. In this way, image quality is incorporated into the estimation result of the estimation unit 1, allowing for a certain degree of quantitative evaluation of the reliability of the estimation result. However, for example, an exclusion step based on image quality may be introduced at the stage when the image data of the inspection target piston is photographed and obtained. In other words, image data with low image quality can be avoided from the beginning without being input to the learning device 5. For example, the exclusion step can be executed by an algorithm implemented as the recognition unit 2. As a result, the overall workload of the image processing device 100 can be reduced. The method described in (g), in which the exclusion step based on image quality is first applied, makes estimation impossible for the entire image. In contrast, this embodiment estimates image quality simultaneously with the estimation of the cleanliness of the ring land (by assigning an image quality label), or estimates image quality simultaneously with the estimation of the surface condition of the ring (by assigning an image quality label). Therefore, estimation can be performed on a ring land or ring basis, resulting in more detailed estimation results.

[0066] (h): It is also possible to realize a technology that appropriately combines the various technologies described in this embodiment. (i) The software described in this embodiment can be realized as hardware, and vice versa. (j) In addition, the constituent elements of the present invention can be appropriately modified within the scope of the present invention. [Explanation of symbols]

[0067] 100 Image processing device 1 Estimation part 2 Recognition section 3 Extraction part 4 Display control section 5. Learning Unit 6 Teacher DB 10 cylinders 11 Piston rod 12 Base 13 Scavenging port 20 (4 rings, 5 ring lands) piston 21~24 Rings (piston rings) 25~29 Ringland 30 (3 rings, 4 ring lands) piston 31~33 Rings (piston rings) 34~37 Ringland 40, 50, 60 image data

Claims

1. a learning machine trained using teacher data relating image data of an engine piston to a surface condition of a piston ring of the piston or a cleanliness of a ring land of the piston; a recognition unit that inputs image data obtained by photographing an engine piston to be inspected into the learning device and recognizes the piston; an estimation unit that estimates a surface condition of a piston ring of the inspection target piston or a cleanliness of a ring land of the inspection target piston based on a recognition result of the recognition unit.

2. The image processing device according to claim 1 , wherein the recognition unit obtains, as the recognition result, an arrangement order of piston rings or ring lands detected from the image data of the inspection target piston.

3. 3. The image processing device according to claim 2, wherein the recognition unit performs a clustering process on the piston rings or ring lands detected from the image data of the inspection target piston, recognizes the number of stages of the piston rings or ring lands of the inspection target piston, and determines the arrangement order according to the recognized number of stages.

4. The image processing device according to claim 3 , wherein the recognition unit adopts a sorting order according to the recognition results and clustering results for each individual piston ring or ring land.

5. The image processing device according to claim 1 , further comprising an extracting unit that extracts a feature amount of the inspection target piston from image data obtained by photographing the inspection target piston.

6. The image processing device according to claim 1 , wherein the image data of the piston to be inspected is image data obtained by photographing a part of the piston to be inspected through a scavenging port.

7. 2. The image processing device according to claim 1, wherein the image data obtained by photographing the piston to be inspected includes at least three of the piston rings and four stages of the ring lands.

8. The image processing device according to claim 1 , wherein the estimation unit simultaneously estimates the surface condition of the piston ring of the inspection target piston and the cleanliness of the ring land of the inspection target piston.

9. 2. The image processing device according to claim 1, wherein the estimation unit estimates the image quality of the image data simultaneously with the surface condition of a piston ring of the piston to be inspected, or estimates the image quality of the image data simultaneously with the cleanliness of a ring land of the piston to be inspected.

10. an image processing device including a learning device trained using teacher data relating image data of an engine piston and a surface condition of a piston ring of the piston or a cleanliness of a ring land of the piston, a recognition step of inputting image data obtained by photographing an engine piston to be inspected into the learning device and recognizing the piston to be inspected; an estimation step of estimating the surface condition of a piston ring of the inspection target piston or the cleanliness of a ring land of the inspection target piston based on the recognition result of the recognition step.

Images