Inspection apparatus, inspection method, and program

Abstract

To improve the accuracy of inspection using a gray scale image.SOLUTION: The inspection device 1 includes an image data acquisition unit 151 that acquires grayscale image data generated by capturing an image of the inspection target W, an image data generation unit 152 that generates color image data for inspection by converting a plurality of grayscale pixel values included in the grayscale image data into a plurality of color pixel values based on conversion data indicating a correspondence relationship between a plurality of grayscale pixel values and a plurality of color pixel values, and an inspection unit 153 that outputs a result of inspection of the inspection target included in the color image data for inspection based on a result of determination output by the machine learning model trained using the color image data for training.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an inspection device, an inspection method, and a program. [Background technology]

[0002] Conventionally, an inspection device that inspects an object to be inspected using a grayscale image generated by an X-ray imaging device is known (see, for example, Patent Document 1). Also known is a system that inspects an object to be inspected by inputting an image of the object to a machine learning model (see, for example, Patent Document 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2023-103351 [Patent Document 2] Japanese Patent Application Laid-Open No. 2024-67360 Summary of the Invention [Problem to be solved by the invention]

[0004] Depending on the type of inspection object, the contrast of a grayscale image generated by a monochrome sensor may be greater than the contrast of a color image, making it suitable for inspection. However, because the grayscale image generated by a monochrome sensor has less data than a color image with the same resolution, when a grayscale image generated by a monochrome sensor is input into a machine learning model capable of inspection based on a color image, the inspection accuracy may be lower than when a color image is input.

[0005] The present invention has been made in view of these points, and has as its object to improve the accuracy of inspections using grayscale images. [Means for solving the problem]

[0006] An inspection device according to a first aspect of the present invention includes an image data acquisition unit that acquires grayscale image data generated by photographing an object to be inspected; an image data generation unit that generates inspection color image data by converting a plurality of grayscale pixel values ​​included in the grayscale image data into a plurality of color pixel values ​​based on conversion data indicating a correspondence between a plurality of grayscale pixel values ​​and a plurality of color pixel values; and an inspection unit that inputs the inspection color image data into a machine learning model that has trained using the training color image data, and outputs an inspection result for the object to be inspected that is included in the inspection color image data based on a determination result output by the machine learning model.

[0007] The image data acquisition unit may acquire the grayscale image data that is an integration of a plurality of provisional grayscale image data that are generated by photographing the object to be inspected with different exposure times.

[0008] The image data acquisition unit may acquire first grayscale image data, second grayscale image data, and third grayscale image data, each of which is generated by photographing the object to be inspected under conditions in which the exposure times are different from each other, and the image data generation unit may generate the inspection color image data by converting the plurality of grayscale pixel values ​​included in the first grayscale image data into a plurality of pixel values ​​of an R channel of the inspection color image data, converting the plurality of grayscale pixel values ​​included in the second grayscale image data into a plurality of pixel values ​​of a G channel of the inspection color image data, and converting the plurality of grayscale pixel values ​​included in the third grayscale image data into a plurality of pixel values ​​of a B channel of the inspection color image data.

[0009] The inspection device may further include a receiving unit that receives a setting operation for the plurality of color pixel values ​​corresponding to the plurality of grayscale pixel values, and a conversion data creating unit that creates the conversion data based on the relationship between the plurality of grayscale pixel values ​​and the plurality of color pixel values ​​indicated by the setting operation.

[0010] The conversion data generating device may further include a storage unit that stores a plurality of pixel value data used to generate the conversion data, a receiving unit that receives an operation to select one pixel value data from the plurality of pixel value data, and a conversion data generating unit that generates the conversion data based on the one pixel value data.

[0011] The image data generation unit generates color image data for setting by converting a plurality of grayscale pixel values ​​included in gray image data for setting generated by imaging the object to a plurality of color pixel values ​​using the conversion data based on one or more pixel value data of the plurality of pixel value data, and the inspection device may further have a display processing unit that displays the color image data for setting on a display unit, and the receiving unit may receive an operation to select the one pixel value data from the plurality of pixel value data after the display processing unit displays the color image data for setting on the display unit.

[0012] The inspection device may further include a memory unit that stores a plurality of pixel value data used to create the conversion data in association with a type of inspection object, a reception unit that receives the type of inspection object, and a conversion data creation unit that creates the conversion data by selecting the pixel value data stored in the memory unit in association with the type received by the reception unit.

[0013] The inspection device may further include a receiving unit that receives designation of an inspection target area in the grayscale image data generated by photographing the inspection target object having an abnormal area, and the abnormal area included in the inspection target area, and the image data generating unit may generate the inspection color image data by converting the plurality of grayscale pixel values ​​to the plurality of color pixel values ​​based on the conversion data so that a color difference between a first color scale pixel value corresponding to the inspection target area and a second color scale pixel value corresponding to the abnormal area is equal to or greater than a threshold.

[0014] The learning color image data is, for example, data created by converting a plurality of grayscale pixel values ​​included in the learning grayscale image data into a plurality of color pixel values ​​based on the conversion data.

[0015] An inspection method according to a second aspect of the present invention includes the steps of: acquiring grayscale image data generated by photographing an object to be inspected, executed by a computer; generating inspection color image data by converting a plurality of grayscale pixel values ​​included in the grayscale image data into a plurality of color pixel values ​​based on conversion data indicating a correspondence between a plurality of grayscale pixel values ​​and a plurality of color pixel values; and inputting the inspection color image data into a machine learning model trained using the training color image data, and outputting an inspection result for the object to be inspected included in the inspection color image data based on a determination result output by the machine learning model.

[0016] A third aspect of the present invention provides a program for causing a computer to execute the following steps: acquiring grayscale image data generated by photographing an object to be inspected; generating inspection color image data by converting a plurality of grayscale pixel values ​​included in the grayscale image data into a plurality of color pixel values ​​based on conversion data indicating a correspondence between a plurality of grayscale pixel values ​​and a plurality of color pixel values; and inputting the inspection color image data into a machine learning model trained using the training color image data, and outputting an inspection result of the object to be inspected included in the inspection color image data based on a determination result output by the machine learning model. [Effects of the Invention]

[0017] The present invention provides an advantage in that it is possible to improve the accuracy of inspection using grayscale images. [Brief explanation of the drawings]

[0018] [Figure 1] FIG. 1 is a diagram for explaining an overview of an inspection system S. [Figure 2] FIG. 2 is a diagram for explaining an outline of the processing performed by the inspection device 1. [Figure 3] 1 is a diagram showing the configuration of an inspection device 1. FIG. [Figure 4] FIG. 1 is a diagram illustrating an overview of the relationship between grayscale pixel values ​​and color pixel values. [Figure 5] FIG. 2 is a diagram illustrating the relationship between grayscale image data and grayscale pixel values. [Figure 6] FIG. 10 is a diagram illustrating an example of color pixel values ​​to which grayscale pixel values ​​are converted. [Figure 7] FIG. 10 is a diagram showing an example of an operation screen for setting color pixel values. [Figure 8] 3 is a flowchart showing the flow of processing in the inspection device 1. [Figure 9] 10 is a flowchart showing the flow of processing for creating conversion data. [Figure 10] FIG. 10 is a diagram for explaining a method for generating one piece of test color image data based on a plurality of pieces of grayscale image data. DETAILED DESCRIPTION OF THE INVENTION

[0019] [Outline of Inspection System S] Fig. 1 is a diagram illustrating an overview of an inspection system S. The inspection system S is a system for inspecting an inspection object W, and includes an inspection device 1 and an imaging device 2. Fig. 1 shows a plurality of inspection objects W moving while being placed on a belt conveyor C, but the inspection objects W may also be placed at the inspection positions by a person.

[0020] The imaging device 2 is a camera, line scanner, line sensor, or area sensor having a monochrome sensor that generates grayscale image data by capturing an image of the inspection object W. The imaging device 2 may be a camera that generates grayscale image data that shows the appearance of the inspection object W, or may be a camera (e.g., an X-ray camera, a visible light camera, or an infrared camera) that generates grayscale image data that shows the internal state of the inspection object W. The imaging device 2 transmits the generated grayscale image data to the inspection device 1.

[0021] The inspection device 1 is a device that inspects whether or not there is an abnormality in the inspection object W by analyzing the grayscale image data received from the imaging device 2, and includes, for example, a computer. Grayscale image data often shows stronger contrast or edges than color image data, and is therefore considered to be advantageous for detecting abnormalities such as stains, scratches, or dents in the inspection object W. However, although the machine learning model used to analyze the image data is configured to be able to analyze color image data, the amount of data in the grayscale image data is smaller than the amount of data in the color image data, and therefore, when the grayscale image data is input into the machine learning model, the performance of the machine learning model cannot be fully utilized.

[0022] Specifically, pixel values ​​(i.e., brightness values) of grayscale image data are generally represented by 8 bits, while pixel values ​​of color image data are generally represented by 8 bits for R (red), 8 bits for G (green), and 8 bits for B (blue), for a total of 24 bits. Therefore, when grayscale image data is input into a machine learning model, it is thought that 16 bits of the 24 bits of data will not be utilized.

[0023] Therefore, the inspection device 1 converts the grayscale image data into color image data for inspection based on predetermined rules, and inputs the color image data for inspection into a machine learning model to inspect the inspection target W. This makes it possible to take full advantage of the grayscale image data, which has the advantage of strongly expressing contrast or edges, while maximizing the performance of the machine learning model, thereby improving the accuracy of the inspection.

[0024] FIG. 2 is a diagram for explaining an overview of the processing performed by the inspection device 1. FIG. 2(a) is a schematic diagram of an example of grayscale image data. The dotted areas in FIG. 2(a) indicate areas where the surface of the inspection object W is normal, and the shaded areas indicate areas where dirt is attached to the surface of the inspection object W. If the color difference between the color of the normal area and the color of the dirt on the inspection object W is small, it is expected that dirt cannot be detected based on the grayscale image data. Therefore, the inspection device 1 converts the grayscale image data into color image data for inspection so that the color difference between the color of the normal area and the color of the abnormal area is as large as possible.

[0025] FIG. 2(b) is a schematic diagram of inspection color image data generated by converting the grayscale image data shown in FIG. 2(a). The white areas in FIG. 2(b) are, for example, yellow, and the black areas in FIG. 2(b) are blue, which has a relatively large color difference from yellow. That is, the dotted areas in FIG. 2(a) are converted to yellow, and the shaded areas are converted to blue. By inputting this color image data into a machine learning model, the accuracy of the inspection is improved.

[0026] The color difference is expressed by the distance in a color difference diagram in the Lab color space, for example. The greater the difference in lightness between the colors, the greater the color difference, and the greater the difference in saturation between the colors. The configuration and operation of the inspection device 1 will be described in detail below.

[0027] [Configuration of inspection device 1] 3 is a diagram showing the configuration of the inspection device 1. The inspection device 1 has a first communication unit 10, a second communication unit 11, an operation unit 12, a display unit 13, a storage unit 14, and a control unit 15. The control unit 15 has an image data acquisition unit 151, an image data generation unit 152, an inspection unit 153, a reception unit 154, a display processing unit 155, and a conversion data creation unit 156.

[0028] The first communication unit 10 has a communication interface for receiving grayscale image data from the imaging device 2. The first communication unit 10 is, for example, a USB (Universal Serial Bus), but may have another communication interface. The second communication unit 11 has a communication interface for transmitting the results of an inspection to determine whether or not there is an abnormality in the inspection target W to an external device. The second communication unit 11 is, for example, a LAN controller, but may have another communication interface.

[0029] The operation unit 12 is a device for accepting user operations, and is, for example, a keyboard, a mouse, or a touch panel. The display unit 13 is a device for displaying information, such as a display.

[0030] The storage unit 14 has storage media such as a ROM (Read Only Memory), a RAM (Random Access Memory), and an SSD (Solid State Drive). The storage unit 14 stores programs executed by the control unit 15. The storage unit 14 temporarily stores grayscale image data generated by the imaging device 2. The storage unit 14 also stores conversion data used to convert the grayscale image data into color image data. The storage unit 14 may store, as a program executed by the control unit 15, a machine learning model program for inspecting the inspection target W based on the inspection color image data.

[0031] 4 to 6 are diagrams for explaining the conversion data. FIG. 4 is a diagram showing an outline of the relationship between pixel values ​​of grayscale image data (hereinafter sometimes referred to as "grayscale pixel values") and pixel values ​​of color image data (hereinafter sometimes referred to as "color pixel values"). The conversion data is data that specifies which of the 24-bit values ​​of the color pixel values ​​each of the 8-bit values ​​of the grayscale pixel values ​​should be converted to. As shown in FIG. 4, the grayscale image pixel values ​​range from 0 to 255 (2 8 ) represent the color pixel values ​​from 0 to 2 24 is converted to one of the values

[0032] Fig. 5 is a diagram showing the relationship between grayscale image data and grayscale pixel values. Fig. 5(a) is grayscale image data of the inspection object W shown in Fig. 2(a), and shows a normal area A and a dirty area B of the inspection object W. Fig. 5(b) shows that the pixel value of area A is 60 and the pixel value of area B is 50.

[0033] Figure 6 shows examples of color pixel values ​​converted from the grayscale pixel values ​​shown in Figure 5(b). Figure 6(a) shows the color pixel value converted from the grayscale pixel value 60 in Figure 5(b), which corresponds to yellow, R255:G255:B0. Figure 6(b) shows the color pixel value converted from the grayscale pixel value 50 in Figure 5(b), which corresponds to blue, R0:G0:B255.

[0034] In this case, in the conversion data, the grayscale pixel value 60 is associated with the color pixel values ​​R255:G255:B0. Also, the grayscale pixel value 50 is associated with the color pixel values ​​R0:G0:B255. By configuring the conversion data in this way, the inspection device 1 can use a machine learning model to generate inspection color image data that makes it easier to detect abnormalities in the inspection object W.

[0035] 3, the configuration of the control unit 15 will be described. The control unit 15 has, for example, a CPU (Central Processing Unit) or a GPU (Graphic Processing Unit). The control unit 15 executes programs stored in the storage unit 14, thereby functioning as an image data acquisition unit 151, an image data generation unit 152, an inspection unit 153, a reception unit 154, a display processing unit 155, and a conversion data creation unit 156.

[0036] The image data acquisition unit 151 acquires grayscale image data generated by photographing the inspection object W. The image data acquisition unit 151 acquires the grayscale image data from, for example, the imaging device 2, but may also acquire the grayscale image data via another device. The image data acquisition unit 151 inputs the acquired grayscale image data to the image data generation unit 152.

[0037] The image data generation unit 152 generates test color image data by converting multiple grayscale pixel values ​​included in the grayscale image data into multiple color pixel values ​​based on conversion data that indicates the correspondence between multiple grayscale pixel values ​​and multiple color pixel values. The image data generation unit 152 reads the conversion data stored in the storage unit 14 and identifies, in the conversion data, color pixel values ​​that correspond to each of the multiple grayscale pixel values ​​included in the grayscale image data. The image data generation unit 152 generates test color image data by converting the grayscale pixel value of each of the multiple pixels in the grayscale image data into the identified color pixel value.

[0038] The inspection unit 153 inspects whether or not there is an abnormality in the inspection object W by analyzing the inspection color image data generated by the image data generation unit 152. Specifically, the inspection unit 153 inputs the inspection color image data into a machine learning model that has been trained using the learning color image data, and outputs the inspection result of the inspection object included in the inspection color image data based on the determination result output by the machine learning model.

[0039] The determination result output by the machine learning model is, for example, a numerical map indicating the probability of an abnormality associated with each of multiple pixels included in the inspection color image data. The inspection unit 153 outputs, for example, the numerical map output by the machine learning model. The inspection unit 153 may output an inspection result indicating the position of pixels in the numerical map where the numerical value (i.e., the abnormality probability) is equal to or greater than a threshold. The inspection unit 153 may output an inspection result indicating an area where a predetermined number or more of consecutive pixels have a probability indicated by the determination result that is equal to or greater than a threshold. The inspection unit 153 may output an inspection result indicating that an abnormality exists in the inspection target W, provided that the numerical value map has a predetermined number or more of pixels where the numerical value is equal to or greater than the threshold.

[0040] The inspection unit 153 outputs the inspection result indicating whether the inspection object W is normal or abnormal to an external device, for example, via the second communication unit 11. The inspection unit 153 may cause the display unit 13 to display the inspection result via the display processing unit 155.

[0041] The machine learning model is stored in, for example, the storage unit 14, but may be stored in a device different from the inspection device 1. The training color image data used for training the machine learning model is data created by converting multiple grayscale pixel values ​​included in the training grayscale image data into multiple color pixel values ​​based on the conversion data. The training grayscale image data is grayscale image data generated by photographing an inspection object W in which an abnormality has occurred. The training grayscale image data may also be grayscale image data generated by photographing a normal inspection object W.

[0042] The learning grayscale image data may be associated with the type of inspection object (for example, object detection, classification from the entire image, or pixel-by-pixel region detection (segmentation)). In this case, a different machine learning model is created for each type of inspection object, and the inspection unit 153 inspects the inspection object W by inputting the inspection color image data into the machine learning model corresponding to the type of inspection object.

[0043] The reception unit 154 receives various operations by a user who uses the inspection device 1 via the operation unit 12. For example, the reception unit 154 receives an operation for creating conversion data or an operation for specifying the type of inspection target.

[0044] The display processing unit 155 displays various types of information on the display unit 13. The display processing unit 155 displays on the display unit 13, for example, a screen for setting conversion data.

[0045] The conversion data creation unit 156 creates conversion data. The conversion data creation unit 156 creates conversion data based on, for example, the contents indicated by the setting operation performed by the user of the inspection device 1. Below, several examples of methods for creating conversion data will be described.

[0046] [How to create conversion data 1] The accepting unit 154 accepts, for example, an operation for setting a plurality of color pixel values ​​corresponding to a plurality of grayscale pixel values ​​so that the conversion data creating unit 156 can create conversion data. The specific method of the operation for setting a plurality of color pixel values ​​corresponding to a plurality of grayscale pixel values ​​is arbitrary. As one example, the accepting unit 154 causes the display unit 13 to display, via the display processing unit 155, a screen for inputting color pixel values ​​corresponding to a plurality of grayscale pixel values ​​0 to 255, and accepts input of a plurality of color pixel values ​​from the user.

[0047] Conversion data creation unit 156 creates conversion data based on the relationship between the multiple grayscale pixel values ​​and the multiple color pixel values ​​indicated by the setting operation accepted by acceptance unit 154. Conversion data creation unit 156 creates conversion data indicating the relationship between the multiple grayscale pixel values ​​and the multiple color pixel values, and stores the created conversion data in storage unit 14.

[0048] [How to create conversion data 2] It takes a long time for the user to input a large number of color pixel values. Therefore, a plurality of pixel value data used to create conversion data may be stored in the storage unit 14, and the conversion data creation unit 156 may create conversion data based on one pixel value data selected by the user from the plurality of pixel value data. In this case, the reception unit 154 displays, for example, a plurality of pixel value data indicating color pixel values ​​corresponding to a plurality of grayscale pixel values ​​0 to 255 on the display unit 13, and receives an operation to select one pixel value data from the plurality of pixel value data. The conversion data creation unit 156 creates conversion data based on the selected one pixel value data.

[0049] To make it easier for the user to select one pixel value data, the display processing unit 155 may display on the display unit 13 the setting color image data created by the image data generation unit 152 using conversion data based on pixel value data selected by the user from the plurality of pixel value data. The image data generation unit 152 creates the setting color image data by, for example, converting a plurality of grayscale pixel values ​​included in the setting gray image data generated by capturing an image of the inspection object into a plurality of color pixel values. The display processing unit 155 may display a plurality of setting color image data simultaneously on the display unit 13, or may switch the setting color image data to be displayed in response to a user operation.

[0050] The accepting unit 154 accepts an operation to select one piece of pixel value data from the plurality of pieces of pixel value data after the display processing unit 155 displays the color image data for settings on the display unit 13. The accepting unit 154 may accept an operation to select one piece of color image data for settings from the plurality of pieces of color image data for settings as an operation to select one piece of pixel value data corresponding to the selected color image data for settings.

[0051] 7 is a diagram showing an example of an operation screen for setting color pixel values. The operation screen shown in FIG. 7 includes areas R1, R2, and R3.

[0052] Region R1 is an area where a grayscale image of the inspection target W is displayed. Region R1 displays a grayscale image including an abnormality, for example, designated by a user who performed an operation to create conversion data. Although one grayscale image is displayed in region R1 in FIG. 7, multiple grayscale images may be displayed in region R1.

[0053] Area R2 is an area for selecting pixel value data. Area R2 displays a grayscale map GS indicating grayscale pixel values ​​and a color map CM (CM1, CM2, CM3, ...) which is a plurality of pixel value data. The grayscale map GS displays the grayscale pixel values ​​contained in the grayscale image displayed in area R1.

[0054] The multiple color maps CM each have a different color pixel value corresponding to a grayscale pixel value. A box for selecting one of the color maps CM is displayed to the right of the multiple color maps CM. The color pixel values ​​in the color map CM correspond to the grayscale pixel values ​​at the same horizontal positions in the grayscale map GS. The conversion data creation unit 156 creates conversion data that associates the color pixel values ​​in the color map CM selected by the user with the grayscale pixel values ​​at the same horizontal positions.

[0055] Area R3 is an area where a color image based on color image data generated by converting the grayscale image data displayed in area R1 using conversion data based on a color map CM selected by the user is displayed. The user can determine whether the selected color map CM is appropriate by checking the color image displayed in area R3. If the user determines that the selected color map CM is appropriate, the user presses "OK" and the conversion data creation unit 156 creates conversion data based on the selected color map CM. If the user determines that the selected color map CM is inappropriate, the user can select another color map CM and check the color image based on the selected color map CM.

[0056] [How to create conversion data 3] It is expected that colors that the machine learning model is likely to detect abnormalities in will differ depending on the type of inspection target, such as the type of inspection object W or the type of inspection content. Therefore, the storage unit 14 may store multiple pixel value data used to create the conversion data in association with the type of inspection target, and the receiving unit 154 may receive the type of inspection target. In this case, the conversion data creating unit 156 creates the conversion data by selecting pixel value data stored in the storage unit 14 in association with the type received by the receiving unit 154.

[0057] The type of inspection object is specified by the name or model name of the inspection object W, the shape of the inspection object W, the content of the inspection, or a combination of these. The content of the inspection is, for example, an inspection for surface scratches, an inspection for outer shape, or an inspection for color.

[0058] 7 on the display unit 13 in response to the reception unit 154 receiving the type of inspection object. In this case, the display processing unit 155 displays one or more color maps CM, which are one or more pixel value data associated with the type of inspection object, on the pixel value data selection screen (area R2). In this way, by allowing different pixel value data to be selected depending on the type of inspection object, the image data generation unit 152 can generate inspection color image data suitable for the type of inspection object, thereby improving the accuracy of the inspection.

[0059] [How to create conversion data 4] In order to minimize the number of setting operations by the user, the image data generating unit 152 may generate conversion data without the user having to input pixel values ​​or select pixel value data (color map). For example, the image data generating unit 152 generates conversion data so that the color difference between a normal area and an abnormal area included in a grayscale image is maximized.

[0060] To achieve this, the receiving unit 154 receives designation of an inspection target area in grayscale image data generated by photographing the inspection target W having an abnormal area, and an abnormal area included in the inspection target area. The receiving unit 154 instructs the display processing unit 155 to display a setting grayscale image such as that shown in area R1 in Fig. 7 on the display unit 13, and receives an operation to designate the inspection target area A and the abnormal area B in the displayed setting grayscale image.

[0061] The conversion data creation unit 156 creates conversion data such that the color difference between the first color scale pixel values ​​corresponding to the grayscale pixel values ​​of the inspection target region and the second color scale pixel values ​​corresponding to the grayscale pixel values ​​of the abnormal region is equal to or greater than a threshold. As an example, conversion data is created such that the color pixel values ​​of the inspection target region A in the setting grayscale image shown in Figure 7 become yellow and the color pixel values ​​of the abnormal region B become blue.

[0062] Based on the created conversion data, the image data generation unit 152 generates inspection color image data by converting the multiple grayscale pixel values ​​into multiple color pixel values ​​so that the color difference between the first color scale pixel values ​​corresponding to the inspection target region and the second color scale pixel values ​​corresponding to the abnormal region is equal to or greater than a threshold. When the image data generation unit 152 generates such inspection color image data, the color difference between the inspection color image data corresponding to the normal region and the abnormal region included in the grayscale image data becomes greater than the threshold, thereby improving the accuracy of the inspection.

[0063] [Processing flow in inspection device 1] Fig. 8 is a flowchart showing the flow of processing in the inspection device 1. The flowchart shown in Fig. 8 starts from the point in time when the inspection device 1 receives an instruction to inspect the inspection object W.

[0064] First, the image data acquisition unit 151 acquires grayscale image data from the imaging device 2 (S1). The image data generation unit 152 converts the grayscale image data into inspection color image data based on the conversion data (S2). The inspection unit 153 inputs the inspection color image data into a machine learning model (S3) and acquires a determination result from the machine learning model. The inspection unit 153 outputs the acquired determination result (S4).

[0065] 9 is a flowchart showing the flow of processing for creating conversion data. First, the image data acquisition unit 151 acquires setting grayscale image data (S21). The setting grayscale image data is grayscale image data generated by, for example, photographing an inspection object W having an abnormality.

[0066] The display processing unit 155 displays the setting grayscale image in a manner such as that shown in Fig. 7 (S22). In this state, the receiving unit 154 receives the setting of an abnormal area (for example, area B in Fig. 7) in the grayscale image data (S23).

[0067] Next, the conversion data creation unit 156 creates provisional conversion data that maximizes the color difference between the color pixel values ​​of the set abnormal area and the color pixel values ​​of other areas (S24). The image data generation unit 152 converts the setting grayscale image data into color image data using the provisional conversion data (S25), and the display processing unit 155 displays the color image data on the display unit 13 (S26).

[0068] If the accepting unit 154 accepts an operation indicating that the provisional conversion data corresponding to the displayed color image data will be used (YES in S27), the conversion data creating unit 156 determines the provisional conversion data as the conversion data (S28) and stores the determined conversion data in the storage unit 14. If the accepting unit 154 accepts an operation indicating that the provisional conversion data corresponding to the displayed color image data will not be used (NO in S27), the process returns to S24, where the conversion data creating unit 156 creates other provisional conversion data, and the inspection device 1 repeats the processes from S24 to S27.

[0069] [Example of using multiple grayscale images] In the above description, an example has been described in which the inspection device 1 inspects the inspection object W using one grayscale image data, but the inspection device 1 may also inspect the inspection object W using multiple grayscale image data. The multiple grayscale image data are grayscale image data generated by photographing one inspection object W multiple times, or grayscale image data generated by photographing the inspection object W with multiple imaging devices 2.

[0070] In this case, the image data generation unit 152 generates a plurality of inspection color image data based on a plurality of grayscale image data, for example, and the inspection unit 153 inspects the inspection object W by sequentially inputting the plurality of inspection color image data into a machine learning model. If an abnormality is found in one or more of the plurality of inspection color image data, the inspection unit 153 determines that an abnormality exists in the inspection object W. In this way, the inspection device 1 inspects the inspection object W using a plurality of grayscale image data, thereby improving the accuracy of the inspection.

[0071] If the inspection object W has unevenness, there may be areas that are not clearly visible when photographed using a specific exposure time. Therefore, the imaging device 2 may generate multiple grayscale image data by photographing the inspection object W using different exposure times, and the inspection device 1 may inspect the inspection object W using the multiple grayscale image data thus generated.

[0072] However, if the image data generation unit 152 generates multiple pieces of test color image data based on multiple pieces of grayscale image data and the inspection unit 153 inputs each of the multiple pieces of test color image data into a machine learning model, the inspection time will be long. Therefore, the image data generation unit 152 may generate one piece of test color image data based on multiple pieces of grayscale image data.

[0073] As an example, the image data acquisition unit 151 acquires grayscale image data that is an integration of multiple pieces of provisional grayscale image data that have been generated by photographing the inspection object W with different exposure times. The image data acquisition unit 151 may integrate the multiple acquired pieces of provisional grayscale image data to generate one piece of grayscale image data. This grayscale image data is configured by selecting, for each of multiple regions of the inspection object W, provisional grayscale image data whose grayscale pixel values ​​are within a predetermined range between black and white, and assigning the grayscale pixel values ​​of the selected provisional grayscale image data to each pixel. Provisional grayscale image data whose grayscale pixel values ​​are within the predetermined range between black and white within a predetermined region is image data that does not have blown-out highlights or blackouts within that region.

[0074] The image data generation unit 152 generates color image data for inspection based on such grayscale image data, and the inspection unit 153 inspects the object W to be inspected using the color image data for inspection, thereby improving the accuracy of inspection when the object W to be inspected has unevenness.

[0075] When multiple pieces of provisional grayscale image data are integrated to generate one piece of grayscale image data, information contained in the unselected pieces of provisional grayscale image data is not used for inspection. Therefore, the image data generating unit 152 may generate inspection color image data using grayscale pixel values ​​of the multiple pieces of provisional grayscale image data.

[0076] 10 is a diagram illustrating a method for generating one inspection color image data based on multiple grayscale image data. The image data acquisition unit 151 acquires first grayscale image data, second grayscale image data, and third grayscale image data generated by photographing the inspection object W with different exposure times.

[0077] The image data generation unit 152 converts the multiple grayscale pixel values ​​included in the first grayscale image data into multiple pixel values ​​in the R channel of the test color image data. Similarly, the image data generation unit 152 converts the multiple grayscale pixel values ​​included in the second grayscale image data into multiple pixel values ​​in the G channel of the test color image data, and converts the multiple grayscale pixel values ​​included in the third grayscale image data into multiple pixel values ​​in the B channel of the test color image data. The image data generation unit 152 generates the test color image data by converting the grayscale pixel values ​​of the multiple grayscale image data into pixel values ​​in each of the RGB channels in this way.

[0078] That is, the image data generation unit 152 integrates the three grayscale image data based on conversion data that indicates that each of the three 8-bit grayscale image data should be assigned to 8-bit data corresponding to red (R), 8-bit data corresponding to green (G), and 8-bit data corresponding to blue (B) in the test color image data.

[0079] The inspection color image data thus generated is color image data that expresses the characteristics contained in the three grayscale image data captured with different exposure times. Therefore, by inspecting the inspection target W based on this inspection color image data, the inspection unit 153 can improve the accuracy of the inspection compared to when an inspection is performed based on a single piece of grayscale image data.

[0080] [Effects of Inspection Device 1] As described above, the image data generation unit 152 generates inspection color image data by converting multiple grayscale pixel values ​​included in the grayscale image data into multiple color pixel values ​​based on conversion data indicating the correspondence between multiple grayscale pixel values ​​and multiple color pixel values. The inspection unit 153 then inputs the generated inspection color image data into a machine learning model capable of analyzing color image data, thereby inspecting the presence or absence of abnormalities in the inspection target W. This configuration of the inspection device 1 makes it possible to maximize the performance of the machine learning model while taking advantage of the grayscale image data acquired by a monochrome sensor, which has the advantage of strongly expressing contrast or edges, thereby improving the accuracy of the inspection.

[0081] The present invention has been described above using embodiments, but the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. For example, all or part of the device can be configured by functionally or physically distributing or integrating any unit. Furthermore, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effects of the new embodiments resulting from the combination also have the effects of the original embodiments.

[0082] For example, the above description illustrates a case where a machine learning model detects an abnormality in an inspection target W, but the present invention can also be applied to cases where the machine learning model has a function to detect an object (Object Detection), a function to classify an entire image (Classification), or a function to detect an area in pixel units (Segmentation). Furthermore, the grayscale image data is not limited to data generated by the imaging device 2, but may be image data generated by a generation AI, or image data in which an abnormality has been added using image editing software to image data generated by photographing a normal product. [Explanation of symbols]

[0083] 1. Inspection equipment 2. Imaging device 10. First Communications Department 11 Second Communications Department 12 Control section 13 Display section 14 Storage section 15 Control Unit 151 Image data acquisition unit 152 Image data generation unit 153 Inspection Department 154 Reception Department 155 Display processing unit 156 Conversion Data Creation Unit

Claims

1. an image data acquisition unit that acquires grayscale image data generated by photographing an object to be inspected; an image data generating unit that generates test color image data by converting a plurality of grayscale pixel values ​​included in the grayscale image data into a plurality of color pixel values ​​based on conversion data that indicates a correspondence relationship between a plurality of grayscale pixel values ​​and a plurality of color pixel values; an inspection unit that outputs an inspection result of the object to be inspected that is included in the inspection color image data based on a judgment result output by a machine learning model that has been trained using the learning color image data by inputting the inspection color image data into the machine learning model; An inspection device having the above structure.

2. the image data acquisition unit acquires the grayscale image data obtained by integrating a plurality of provisional grayscale image data generated by photographing the object to be inspected with different exposure times, The inspection device according to claim 1 .

3. the image data acquisition unit acquires first grayscale image data, second grayscale image data, and third grayscale image data generated by photographing the inspection object with different exposure times, the image data generation unit converts the plurality of grayscale pixel values ​​included in the first grayscale image data into a plurality of pixel values ​​of an R channel of the test color image data, converts the plurality of grayscale pixel values ​​included in the second grayscale image data into a plurality of pixel values ​​of a G channel of the test color image data, and converts the plurality of grayscale pixel values ​​included in the third grayscale image data into a plurality of pixel values ​​of a B channel of the test color image data, thereby generating the test color image data. The inspection device according to claim 1 .

4. a receiving unit that receives a setting operation of the plurality of color pixel values ​​corresponding to the plurality of grayscale pixel values; a conversion data creating unit that creates the conversion data based on the relationship between the plurality of grayscale pixel values ​​and the plurality of color pixel values ​​indicated by the setting operation; The inspection device according to claim 1 .

5. a storage unit that stores a plurality of pixel value data used to create the conversion data; a receiving unit that receives an operation to select one pixel value data from the plurality of pixel value data; a conversion data creation unit that creates the conversion data based on the one pixel value data; further comprising The inspection device according to claim 1 .

6. the image data generation unit generates color image data for setting by converting a plurality of grayscale pixel values ​​included in the gray image data for setting generated by imaging the object to a plurality of color pixel values ​​using the conversion data based on one or more pixel value data of the plurality of pixel value data; the inspection device further includes a display processing unit that displays the setting color image data on a display unit; the accepting unit accepts an operation to select the one pixel value data from the plurality of pixel value data after the display processing unit has displayed the setting color image data on the display unit. The inspection device according to claim 5 .

7. a storage unit that stores a plurality of pixel value data used to create the conversion data in association with a type of inspection object; a reception unit that receives the type of the inspection target; a conversion data creation unit that creates the conversion data by selecting the pixel value data stored in the storage unit in association with the type accepted by the acceptance unit; further comprising The inspection device according to claim 1 .

8. a receiving unit that receives designation of an inspection target area in the grayscale image data generated by photographing the inspection target having an abnormal area and the abnormal area included in the inspection target area, the image data generation unit generates the inspection color image data by converting the plurality of grayscale pixel values ​​into the plurality of color pixel values ​​based on the conversion data so that a color difference between a first color scale pixel value corresponding to the inspection target region and a second color scale pixel value corresponding to the abnormal region is equal to or greater than a threshold value. The inspection device according to claim 1 .

9. the learning color image data is data created by converting a plurality of grayscale pixel values ​​included in the learning grayscale image data into a plurality of color pixel values ​​based on the conversion data; The inspection device according to claim 1 .

10. The computer executes acquiring grayscale image data generated by photographing an object to be inspected; generating test color image data by converting a plurality of grayscale pixel values ​​included in the grayscale image data into a plurality of color pixel values ​​based on conversion data indicating a correspondence relationship between a plurality of grayscale pixel values ​​and a plurality of color pixel values; a step of inputting the inspection color image data into a machine learning model trained using the learning color image data, and outputting an inspection result of the object to be inspected that is included in the inspection color image data based on a judgment result output by the machine learning model; An inspection method having the following.

11. On the computer, acquiring grayscale image data generated by photographing an object to be inspected; generating test color image data by converting a plurality of grayscale pixel values ​​included in the grayscale image data into a plurality of color pixel values ​​based on conversion data indicating a correspondence relationship between a plurality of grayscale pixel values ​​and a plurality of color pixel values; a step of inputting the inspection color image data into a machine learning model trained using the learning color image data, and outputting an inspection result of the object to be inspected that is included in the inspection color image data based on a judgment result output by the machine learning model; A program to execute.

Images