Inspection system, and inspection method

Abstract

To provide an inspection system capable of improving the accuracy of determining the surface color of an object, and an inspection method.SOLUTION: An inspection system 1 includes a determination unit F13. The determination unit F13 determines the surface condition of an object 100. The determination unit F13 is configured to acquire images of multiple portions where specular reflection components that are obtained by picking up an object 100 under imaging conditions that maintain the continuity of the object 100 are dominant, generate a composite image from the images of multiple portions so as to maintain the continuity of the object 100 and determine the surface condition of the object from the composite image. The images of multiple portions are acquired by picking up images of the surface of the object 100 while displacing multiple cameras relative to the object 100.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an inspection system and an inspection method, and more particularly, to an inspection system and an inspection method for making a judgment regarding a surface of an object based on an image. [Background technology]

[0002] Patent Document 1 discloses a color inspection device. The color inspection device of Patent Document 1 includes a camera having three spectral sensitivities linearly converted equivalent to the CIEXYZ color matching function, a calculation processing device that acquires and calculates color data obtained by converting the image having the three spectral sensitivities acquired by the camera into tristimulus values ​​X, Y, and Z in the CIEXYZ color system, and a lighting unit that illuminates an automobile, which is an example of a measurement target, and inspects the color by calculating a color distribution coincidence index that indicates the overlapping ratio of two xyz chromaticity histogram distributions of the inspection object and the reference object. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2015-155892 A Summary of the Invention

[0004] The object is to provide an inspection system and an inspection method that can improve the accuracy of determining the color of the surface of an object.

[0005] An inspection system according to an aspect of the present disclosure includes a determination unit. The determination unit determines a surface condition of an object. The determination unit acquires a plurality of partial images in which specular reflection components are dominant and are obtained by imaging the object under imaging conditions that maintain the continuity of the object. The determination unit generates a composite image from the plurality of partial images such that the continuity of the object is maintained. The determination unit determines the surface condition of the object from the composite image. The plurality of partial images are obtained by imaging the surface of the object by displacing a plurality of cameras relative to the object.

[0006] An inspection method according to one aspect of the present disclosure is an inspection method for determining a surface condition of an object. The inspection method includes a first step, a second step, and a third step. The first step is a step of acquiring a plurality of partial images obtained by imaging the object under imaging conditions that maintain the continuity of the object, in which the specular reflection component is dominant. The second step is a step of generating a composite image from the plurality of partial images so that the continuity of the object is maintained. The third step is a step of determining the surface condition of the object from the composite image. The plurality of partial images are obtained by imaging the surface of the object by displacing a plurality of cameras relative to the object.

[0007] A program according to one embodiment of the present disclosure is a program for causing one or more processors to execute the above-described inspection method.

[0008] A storage medium according to one aspect of the present disclosure is a non-transitory storage medium readable by a computer, and stores the above-mentioned program. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram of an inspection system according to one embodiment. [Diagram 2] FIG. 2 is an explanatory diagram of the above-mentioned inspection system. [Diagram 3] FIG. 3 is a flowchart of the setting process of the above-mentioned inspection system. [Figure 4] FIG. 4 is a flowchart of the color judgment process of the above-mentioned inspection system. [Diagram 5] FIG. 5 is an explanatory diagram of the first separated image. [Figure 6] FIG. 6 is an explanatory diagram of the second separated image. [Figure 7] FIG. 7 is an explanatory diagram of the separation process of the above-mentioned inspection system. [Figure 8] FIG. 8 is an explanatory diagram of another separation process of the above inspection system. [Figure 9]FIG. 9 is an explanatory diagram of the synthesis process of the above-mentioned inspection system. [Figure 10] FIG. 10 is another explanatory diagram of the synthesis process of the above inspection system. [Figure 11] FIG. 11 is an explanatory diagram of the color matching process of the above-mentioned inspection system. [Figure 12] FIG. 12 is a flowchart of the painting process including the painting judgment process of the above-mentioned inspection system. [Figure 13] FIG. 13 is an explanatory diagram of the coating process of the above inspection system. [Figure 14] FIG. 14 is another explanatory diagram of the coating process of the above inspection system. [Figure 15] FIG. 15 is a flowchart of the texture determination process of the above-mentioned inspection system. [Figure 16] FIG. 16 is an explanatory diagram of the texture determination process of the above inspection system. [Figure 17] FIG. 17 is an explanatory diagram of an imaging system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] 1. Embodiment 1.1 Overview 1 shows an inspection system 1 according to an embodiment. The inspection system 1 according to the present embodiment includes an acquisition unit F11 and a determination unit F13. The acquisition unit F11 acquires an image of the surface of the object 100. The determination unit F13 performs a color determination process. The color determination process is a process for determining the color of the surface of the object 100 based on a plurality of reflection states having different ratios of specular reflection components and diffuse reflection components on the surface of the object 100, which are obtained from the image of the surface of the object 100 acquired by the acquisition unit F11.

[0011] In the inspection system 1, the color of the surface of the object 100 is determined from multiple viewpoints, not from a single viewpoint. In particular, the color of the surface of the object 100 is determined by utilizing multiple reflection states in which the ratio of the specular reflection component to the diffuse reflection component differs from one another on the surface of the object 100. The specular reflection component reflects the surface condition of the object 100 better than the diffuse reflection component, and the diffuse reflection component reflects the surface color of the object 100 itself better than the specular reflection component. This makes it possible to determine the surface color of the object 100 by taking into account not only the color of the surface of the object 100 itself, but also the surface condition of the object 100. As a result, the inspection system 1 can improve the accuracy of determining the surface color of the object 100.

[0012] 1.2 Details The inspection system 1 will be described in more detail below with reference to the drawings. The inspection system 1 is a system for inspecting an object 100. The inspection system 1 has a function as a coloring inspection device. In this embodiment, the inspection system 1 inspects the color, paint condition, and texture of the surface of the object 100. The inspection system 1 can also paint the object 100. The inspection system 1 can paint the object 100 according to the inspection result, thereby obtaining the object 100 with the desired paint.

[0013] The object 100 may be any object having a surface. In this embodiment, the object 100 is an automobile. In particular, the surface of the object 100 is the outer surface of the body of the automobile. The object 100 is not limited to an automobile. For example, the object 100 may be a moving object other than an automobile, or may not be a moving object. Examples of moving objects include motorcycles, trains, drones, aircraft, construction machinery, and ships. The object 100 may also be electrical equipment, tableware, containers, furniture, clothing, building materials, and the like. In short, the object 100 may be any object having a surface. In particular, if the object 100 is an object to be painted, the inspection system 1 of this embodiment can be suitably used.

[0014] As shown in FIG. 1, the inspection system 1 includes a determination system 10, a lighting system 20, an imaging system 30, and a coating system 40.

[0015] The lighting system 20 is a system for irradiating light onto the surface of the object 100. As shown in FIG. 2, the lighting system 20 includes a plurality of lamps 21 (four in FIG. 2) that irradiate the object 100 with light. The lamps 21 are, for example, LED lamps. The lamps 21 emit white light. In the lighting system 20, the number of lamps 21 is not particularly limited, and the type of the lamps 21 may be a light source other than an LED. The light emission color of the lamps 21 is not limited to white. The light emission color of the lamps 21 may be appropriately set in consideration of the color of the object 100 and the color detectable by the imaging system 30. The wavelength of the light emitted by the lighting system 20 may be changeable. Although the inspection system 1 of this embodiment includes the lighting system 20, if the color of the object 100 can be determined without the lighting system 20, the lighting system 20 may be omitted.

[0016] The imaging system 30 is a system for generating an image (digital image) of the surface of the object 100. In this embodiment, the imaging system 30 captures an image of the surface of the object 100 illuminated by the lighting system 20 to generate an image of the surface of the object 100. The imaging system 30 includes a plurality of cameras. Each camera includes one or more image sensors. The cameras may include one or more line sensors.

[0017] In this embodiment, the multiple cameras of the imaging system 30 include, as shown in Fig. 2, four (first) cameras 31 (311-314) and one (second) camera 32. Here, the first camera 31 and the second camera 32 are displaceable with respect to the object 100. In addition, the second camera 32 has a wider angle of view than the first camera 31 and is preferably capable of generating an image of the entire object 100. As an example, the second camera 32 is located at a position overlooking the object 100 and generates an image showing an overhead view of the object 100.

[0018] The painting system 40 is a system for painting the surface of the object 100. As shown in FIG. 2, the painting system 40 includes multiple painting units (two painting units 41, 42). In this embodiment, the painting units 41, 42 are both painting robots. The painting units 41, 42 are displaceable with respect to the object 100. The painting robots may have a conventionally known configuration, and therefore a detailed description thereof will be omitted. Note that the painting system 40 may include one or more painting units, and the number of painting units is not limited to two.

[0019] 1, the determination system 10 includes an input / output unit 11, a storage unit 12, and a processing unit 13. The determination system 10 may be realized by a computer system. The computer system may include one or more processors, one or more connectors, one or more communication devices, one or more memories, and the like.

[0020] The input / output unit 11 inputs and outputs information between the lighting system 20, the imaging system 30, and the painting system 40. In this embodiment, the input / output unit 11 is communicatively connected to the lighting system 20, the imaging system 30, and the painting system 40. The input / output unit 11 includes one or more input / output devices and uses one or more input / output interfaces.

[0021] The storage unit 12 is used to store information used by the processing unit 13. The storage unit 12 includes one or more storage devices. The storage device is, for example, a RAM (Random Access Memory) or an EEPROM (Electrically Erasable Programmable Read Only Memory). The storage unit 12 stores sample data used in the color judgment process. The sample data includes information on the target color of the surface of the object 100 (i.e., sample color data). As an example, the information on the target color of the surface of the object 100 is given as a reflectance value for each wavelength (see FIG. 11). For example, the information on the target color of the surface of the object 100 is given as a reflectance value in the wavelength range of 380 nm to 780 nm. In other words, the sample data has an aspect of a digital color sample. In addition, the sample data may include at least one of the shape of the object 100 (shape information of the object to be photographed) and the imaging conditions of the object 100. The imaging conditions of the object 100 may include the relative positional relationship between the object 100, the lighting system 20, and the imaging system 30 (i.e., information on the positional relationship between the object to be photographed, the lighting, and the camera). In other words, the imaging conditions of the object 100 may include information on the lighting by the lighting system 20 (lighting information). As an example, the sample data may be an image of the entire surface of the object 100 captured under specified imaging conditions. The memory unit 12 also stores reference information used in the paint judgment process. The reference information represents the desired paint state of the surface of the object 100. In other words, the reference information indicates the desired paint state of the object 100.

[0022] The processing unit 13 may be realized by, for example, one or more processors (microprocessors). That is, the one or more processors execute one or more programs (computer programs) stored in one or more memories to function as the processing unit 13. The one or more programs may be pre-recorded in one or more memories, or may be provided via a telecommunication line such as the Internet, or recorded on a non-transitory recording medium such as a memory card.

[0023] The processing unit 13 performs a setting process (see FIG. 3), a color judgment process (see FIG. 4), a paint judgment process (see FIG. 12), and a texture judgment process (see FIG. 15). As shown in FIG. 1, the processing unit 13 has an acquisition unit F11, a separation unit F12, and a judgment unit F13. The acquisition unit F11, the separation unit F12, and the judgment unit F13 do not indicate actual configurations, but indicate functions realized by the processing unit 13.

[0024] The acquisition unit F11 acquires an image of the surface of the object 100 (see FIGS. 9 and 10). In this embodiment, the acquisition unit F11 acquires the image of the surface of the object 100 from the imaging system 30. That is, the acquisition unit F11 receives the image from the imaging system 30 via the input / output unit 11. The image that the acquisition unit F11 acquires from the imaging system 30 is determined by the imaging conditions of the imaging system 30.

[0025] The separation unit F12 executes a separation process. The separation process is a process for obtaining a plurality of reflection states having different ratios of specular reflection components and diffuse reflection components on the surface of the object 100 from the image of the surface of the object 100 obtained by the acquisition unit F11. In the separation process, the separation unit F12 obtains a plurality of separated images from the image obtained by the acquisition unit F11 as images representing a plurality of reflection states having different ratios of specular reflection components and diffuse reflection components on the surface of the object 100. The plurality of separated images each represent an image of the surface of the object 100, but are images having different ratios of specular reflection components and diffuse reflection components. In this embodiment, the plurality of separated images are a first separated image P10 (see FIG. 5) and a second separated image (see FIG. 6). However, the number of the plurality of separated images is not limited to two, and it is also possible to use three or more separated images.

[0026] Here, the multiple reflection states include the state of light reflection on the surface of the object 100. When the surface of the object 100 is viewed from different directions, the ratio of the specular reflection component to the diffuse reflection component may change. Therefore, the multiple reflection states can be said to be the state of the surface of the object 100 viewed from different viewpoints. By capturing the surface of the object 100 with cameras at different locations, images that represent different reflection states on the surface of the object 100 can be obtained. When capturing the surface of the object 100 with cameras at different locations, the obtained images may include both the specular reflection component and the diffuse reflection component. However, it is possible to extract only the specular reflection component and only the diffuse reflection component from the image by arithmetic processing. As shown in FIG. 7, the specular reflection component has a peak in the region on the imaging plane I where the angle of incidence of light from the light source L to the surface S is equal to the angle of reflection of light on the surface S. Therefore, the specular reflection component is dominant in the region on the imaging plane I that corresponds to a predetermined range (θ±φ) centered on the reflection angle θ. The diffuse reflection component is dominant in the area on the imaging plane I that does not correspond to the predetermined range (θ±φ). Therefore, the intensity of the reflected light on the imaging plane I is given as a combination of the specular reflection component and the diffuse reflection component. The specular reflection component and the diffuse reflection component can be estimated by a statistical model. In addition, since the specular reflection component and the diffuse reflection component have different intensity distributions, they can be separated based on the gradation value (brightness value) of the image, as shown in FIG. 8.

[0027] In this embodiment, the separating unit F12 extracts a first separated image P10 (see FIG. 5) and a second separated image (see FIG. 6) from the image acquired by the acquiring unit F11. Therefore, the multiple reflection states may include a state in which only the specular reflection component exists and a state in which only the diffuse reflection component exists. The first separated image P10 represents a state in which only the specular reflection component exists on the surface of the object 100, and the second separated image P20 represents a state in which only the diffuse reflection component exists on the surface of the object 100. Here, the specular reflection component reflects the state of the surface of the object 100 better than the diffuse reflection component, and the diffuse reflection component reflects the color of the surface of the object 100 itself better than the specular reflection component.

[0028] The determination unit F13 performs a setting process (see FIG. 3), a color determination process (see FIG. 4), a painting process (see FIG. 12), and a texture determination process (see FIG. 15).

[0029] (Settings process) The setting process is a pre-processing of the color judgment process. In the setting process, the imaging system 30 is set. The judgment unit F13 determines the imaging conditions of the imaging system 30 in order to set the imaging system 30. The imaging conditions specify the operating states of the multiple cameras of the imaging system 30, particularly the multiple first cameras 31. The operating states may include the position with respect to the surface of the object 100, the imaging direction with respect to the surface of the object 100, the angle of view (field of view), and the magnification (zooming). In this embodiment, the four first cameras 311-314 of the imaging system 30 generate images (partial images) P31-P34 representing parts of the surface of the object 100, as shown in FIG. 9 and FIG. 10. In this embodiment, the multiple partial images P31-P34 are generated by the multiple cameras 31 with different imaging directions with respect to the object 100. Therefore, in order to generate an image (entire image) P30 representing the entire surface of the object 100 using partial images P31-P34 of the four first cameras 311-314 as shown in Fig. 9, the determination unit F13 determines the imaging conditions of the imaging system 30. The determination unit F13 sets the imaging system 30 according to the determined imaging conditions. This makes it possible to obtain an image of the entire object 100 using the imaging system 30. Note that in this embodiment, the entire surface of the object 100 may be the entire portion that is the subject of determination in the color determination process, and does not necessarily have to be the entire surface of the object 100 in a practical sense.

[0030] The setting process will be described below with reference to the flowchart in FIG.

[0031] First, the determination unit F13 acquires a plurality of partial images P31-P34 of the object 100 from the imaging system 30 by the acquisition unit F11 (S11). From each of the plurality of partial images P31-P34 acquired by the acquisition unit F11, the separation unit F12 acquires a first separated image and a second separated image (S12). The determination unit F13 synthesizes the first separated images of the plurality of partial images P31-P34 to generate a first composite image. The determination unit F13 also synthesizes the second separated images of the plurality of partial images P31-P34 to generate a second composite image (S13). The determination unit F13 determines the imaging conditions of the imaging system 30 (operation states of the four first cameras 311-314) so ​​that the continuity of the object 100 is maintained for each of the first composite image and the second composite image (S14). Here, the continuity of the object 100 (continuity of gradation of the object to be photographed) is maintained means that the shape of the object 100 is correctly reflected in a composite image obtained by combining partial images. FIG. 9 shows a state in which the continuity of the object 100 is maintained. FIG. 10 shows a state in which the continuity of the object 100 is not maintained. For ease of explanation, FIG. 9 and FIG. 10 show an example in which a composite image is generated from partial images P31 to P34, not the first and second separated images. The determination unit F13 can determine the imaging conditions by referring to the shape of the object 100 included in the sample data stored in the storage unit 12. The determination unit F13 determines the imaging conditions so that the shape of the object 100 in the first and second separated images matches the shape of the object 100 included in the sample data. The determination unit F13 sets the imaging system 30 according to the determined imaging conditions (S15). This updates the operating states of the four first cameras 311 to 314. As a result, imaging system 30 allows an image of the entire object 100 to be obtained.

[0032] In this way, the inspection system 1 creates a composite image (first and second composite images) by combining images (partial images) from multiple cameras 31, and calculates and outputs the imaging conditions from the composite images. Then, the inspection system 1 controls the angle of view and zooming of the cameras 31 so that the continuity of the gradation of the photographed object (object 100) is maintained from the composite image.

[0033] (Color Judgment Processing) The color judgment process is a process for judging the color of the surface of the object 100. More specifically, the color judgment process is a process for judging the color of the surface of the object 100 based on a plurality of reflection states in which the ratios of the specular reflection component and the diffuse reflection component are different on the surface of the object 100, which are obtained from the image of the surface of the object 100 acquired by the acquisition unit F11. In particular, in the color judgment process, the judgment unit F13 judges the color of the surface of the object 100 based on a plurality of separated images P10, P20. Furthermore, in the color judgment process, the judgment unit F13 judges the color of the surface of the object 100 based on the images P10, P20 representing the entire surface of the object 100 in each of a plurality of reflection states, which are obtained from a plurality of partial images P31 to P34. Note that the judgment of the color of the surface of the object 100 may be performed on a pixel-by-pixel basis in the images P10, P20.

[0034] The color determination process will be described below with reference to the flowchart in FIG.

[0035] First, the determination unit F13 acquires a plurality of partial images P31-P34 of the object 100 from the imaging system 30 by the acquisition unit F11 (S21). From each of the plurality of partial images P31-P34 acquired by the acquisition unit F11, the separation unit F12 acquires a first separated image and a second separated image (S22). The determination unit F13 synthesizes the first separated images of the plurality of partial images P31-P34 to generate a first composite image. The determination unit F13 also synthesizes the second separated images of the plurality of partial images P31-P34 to generate a second composite image (S23). The determination unit F13 compares each of the first composite image and the second composite image with information on the target color of the surface of the object 100 of the sample data stored in the storage unit 12 (S24). The information on the target color of the surface of the object 100 includes information on the target color for the first composite image and information on the target color for the second composite image. Therefore, the determination unit F13 performs a determination of the color of the object 100 for each of the first composite image and the second composite image. As an example, if the degree of agreement between the color obtained from the first composite image and the target color for the first composite image of the sample data is equal to or greater than a specified value, the determination unit F13 determines that the color obtained from the first composite image is acceptable. Similarly, if the degree of agreement between the color obtained from the second composite image and the target color for the second composite image is equal to or greater than a specified value, the determination unit F13 determines that the color obtained from the second composite image is acceptable. In this manner, the determination unit F13 determines the color of the surface of the object 100 based on images representing the entire surface of the object 100 in each of a plurality of reflection states obtained from a plurality of partial images. If the colors obtained from each of the first composite image and the second composite image are acceptable, the determination unit F13 determines that the result of the color determination process is acceptable (S25: Yes).

[0036] On the other hand, when at least one of the colors obtained from the first composite image and the second composite image is unacceptable, the judgment unit F13 determines the result of the color judgment process as unacceptable (S25: No). In this case, the judgment unit F13 repaints the object 100 (S26). In the repainting, the judgment unit F13 controls the painting system 40 according to the difference between the color obtained from the first composite image and the target color for the first composite image, and the difference between the color obtained from the second composite image and the target color for the second composite image. That is, the judgment unit F13 controls the painting system 40 that paints the surface of the object 100 based on the result of the color judgment process. This makes it possible to make the color of the surface of the object 100 closer to the target color. For example, FIG. 11 shows the relationship between the graphs G10 and G11 representing the colors obtained from the composite image (the first composite image or the second composite image) and the graph G20 representing the target color corresponding to the composite image. 11, G10 represents the color before repainting, and G11 represents the color after repainting. In this manner, the inspection system 1 can correct the color of the surface of the object 100.

[0037] (Painting process) The painting process is a process of painting (also called painting) the object 100. The painting process will be described below with reference to the flowchart in Fig. 12 and Fig. 13 and Fig. 14. Fig. 13 and Fig. 14 show a scene of the painting process.

[0038] First, the determination unit F13 determines a painting location (a coating location) on the surface of the object 100 (S31). In this embodiment, the painting system 40 includes two painting robots 41 and 42, so that two painting locations can be selected simultaneously from the surface of the object 100. Examples of the painting locations include a hood, a roof, a front door, a rear door, a front bumper, a rear bumper, a fender, a rear fender, and a trunk cover. The determination unit F13 determines the painting location using the imaging system 30. The determination unit F13 monitors the entire surface of the object 100 with the second camera 32, and grasps the unpainted locations (unpainted locations) on the surface of the object 100. The determination unit F13 selects the next painting location from the unpainted locations on the surface of the object 100.

[0039] Next, the determination unit F13 executes a pre-painting inspection (S32). In the pre-painting inspection, the determination unit F13 determines whether or not there is a foreign object at the painting location before painting the painting location. The determination unit F13 controls the imaging system 30, captures an image of the painting location using one of the multiple first cameras 31 (first camera 312 in FIG. 13), and acquires the image through the acquisition unit F11. The determination unit F13 detects foreign objects from the image of the painting location acquired by the acquisition unit F11 using image processing technology. When determining whether or not there is a foreign object, the state of the painting location (color tone, unevenness information) is also determined. The state of the painting location is reflected in the level of painting (adhesion level) by the painting system 40. As an example, the amount of paint discharged (spray amount, spray concentration) is determined according to the level of painting (adhesion level). Here, if no foreign object is detected (S33: No), the determination unit F13 starts painting (adhesion) of the painting location (S34).

[0040] When painting the painting location, the judgment unit F13 controls the painting system 40 as shown in FIG. 13, and paints the painting location (the bonnet and the right rear fender in FIG. 13) using the painting robots 41 and 42. Furthermore, the judgment unit F13 judges the state of the location being painted on the surface of the object 100. The judgment unit F13 controls the imaging system 30 to acquire an image of the painting location. In FIG. 13, the first cameras 311 and 314 are used to acquire an image of the painting location. The judgment unit F13 obtains a difference between the painting state of the surface (current surface) obtained from the image of the surface of the object 100 acquired by the acquisition unit F11 (i.e., the image of the painting location) and the target painting state of the surface of the object 100. The judgment unit F13 controls the painting system 40 so that the difference becomes small. As an example, the judgment unit F13 adjusts the spray concentration (ejection amount) from the painting robots 41 and 42. In the painting judgment process, the judgment unit F13 judges the insufficient coating (e.g., color tone and uniformity) of the painting location. In the painting judgment process, as in the color judgment process, the color of the surface of the object 100 may be judged based on a plurality of reflection states in which the ratios of the specular reflection component and the diffuse reflection component are different from each other on the surface of the object 100.

[0041] When painting is completed, the determination unit F13 executes a post-painting inspection (S35). In the post-painting inspection, the determination unit F13 judges the state after painting. Examples of the state after painting include the presence or absence of paint dripping, the presence or absence of drying, and the continuity with the painted area (continuity of color tone and surface condition). In this embodiment, it is determined whether or not there is paint dripping at the painting location. The determination unit F13 controls the imaging system 30, and captures an image of the painting location where painting has been completed using one of the multiple first cameras 31 (the first camera 313 in FIG. 13), and acquires the image through the acquisition unit F11. Paint dripping is detected from the image of the painting location acquired by the acquisition unit F11. For example, paint dripping can be detected based on the color tone and surface condition (e.g., color uniformity) of the painting location. The determination unit F13 obtains a difference between the painting state of the surface (current surface) obtained from the image of the surface of the object 100 acquired by the acquisition unit F11 (i.e., the image of the painting location) and the target painting state of the surface of the object 100. As the target coating state of the surface of the object 100 (a comparison image for determining whether paint is dripping), a portion where paint is evenly applied may be used from the entire image of the surface of the object 100 captured by the second camera 32. If the difference exceeds a threshold value (first threshold value), the determination unit F13 may determine that paint is dripping. Note that, in determining whether paint is dripping, similarly to the color determination process, the color of the surface of the object 100 may be determined based on a plurality of reflection states in which the ratios of specular reflection components and diffuse reflection components on the surface of the object 100 are different from one another.

[0042] Here, if no paint dripping is detected (S36: No), the judgment unit F13 judges whether or not there is an uncoated area based on the image of the entire surface of the object 100 captured by the second camera 32 (S371). If there is an uncoated area (S371: Yes), the judgment unit F13 determines the next area to be coated (S31). On the other hand, if there is no uncoated area (S371: No), the judgment unit F13 performs a final inspection (S372). In the final inspection, the judgment unit F13 may perform a color judgment process to inspect the color of the entire surface of the object 100.

[0043] On the other hand, if paint dripping is detected (S36: Yes), the determination unit F13 determines the degree of paint dripping (S381). More specifically, the determination unit F13 determines whether the degree of paint dripping is severe. Whether the degree of paint dripping is severe may be determined based on whether the paint dripping can be corrected by repainting. As an example, the determination unit F13 may determine that the paint dripping is severe when the difference between the image of the painting location obtained by the acquisition unit F11 and the comparison image exceeds a second threshold value that is greater than the first threshold value. Note that the first threshold value and the second threshold value may be the same. Note that, if paint dripping is detected, the determination unit F13 may associate the feature amount of the image of the paint dripping with the paint discharge amount, room temperature, humidity, ventilation flow rate, etc. by statistical analysis or machine learning, and generate information for preventing recurrence of paint dripping.

[0044] If the degree of paint dripping is not severe (S381: No), the judgment unit F13 executes re-application (repainting) (S382). In re-application, similar to painting (S34), the judgment unit F13 controls the painting system 40 to re-paint the painting location where paint dripping was detected. As an example, FIG. 14 shows the first camera 314 capturing an image of the trunk cover in order for the painting robot 42 to repaint the trunk cover. Then, when re-application is completed, the judgment unit F13 executes a post-painting inspection again (S35).

[0045] If a foreign object is detected during the painting process (S33: Yes) or if paint dripping is severe (S381: Yes), the decision unit F13 stops the painting process (application) (S39).

[0046] The painting process includes a painting judgment process (S34, S35). The painting judgment process is a process for judging the painted state of the surface of the object 100. More specifically, the painting judgment process is a process for judging the painted state of the surface of the object 100 based on the image of the surface of the object 100 acquired by the acquisition unit F11. In the painting judgment process, the judgment unit F13 obtains a difference between the painted state of the surface obtained from the image of the surface of the object 100 acquired by the acquisition unit F11 and the target painted state of the surface of the object 100. The judgment unit F13 controls the operating state of the multiple cameras 31, 32 of the imaging system 30 according to the result of the painting judgment process. That is, the judgment unit F13 controls the operating state of the multiple cameras 31, 32 of the imaging system 30 according to the progress of painting of the object 100. In particular, the multiple cameras 31, 32 include one or more first cameras 31 that generate an image of a part of the surface of the object 100 and a second camera 32 that generates an image of the entire surface of the object 100. The judgment unit F13 controls the operation state of one or more first cameras 31 according to the result of the painting judgment process. The judgment unit F13 also controls the operation state of one or more first cameras 31 according to the image captured by the second camera 32 and the result of the painting judgment process.

[0047] In this embodiment, the judgment unit F13 controls the multiple cameras 31 and 32 of the imaging system 30 to paint the entire object 100. Here, when the object 100 is a vehicle such as an automobile, it is large in size and has a wide variety of paint conditions (adhesion, dripping, drying, foreign matter, etc.) to be observed, so it is difficult to photograph the entire object 100 with one camera. However, the inspection system 1 of this embodiment links a first camera (local camera) 31 that photographs a local area where paint is being applied on the object 100 with a second camera (bird's-eye view camera) 32 that captures the entire paint condition of the object 100. Therefore, the inspection system 1 of this embodiment can sense the paint condition of the object 100 without overlooking anything.

[0048] (Texture determination processing) The texture determination process is a process for determining the texture of the surface of the object 100. The texture determination process is a process for determining the texture of the surface of the object 100 based on a change in luminance information between a plurality of series of images obtained by capturing images of the surface of the object 100 from different locations L1 to L3 (see FIG. 16). Here, the change in luminance information is defined by feature vectors (spatial feature vector and temporal feature vector).

[0049] The texture determination process will be described below with reference to the flowchart of Fig. 15 and Fig. 16. Fig. 16 shows the positional relationship between the second camera 32 and the object 100. Note that the first camera 31 can be used instead of the second camera 32.

[0050] First, the determination unit F13 acquires a plurality of sequential images obtained by imaging the surface of the object 100 from different positions L1 to L3 (see FIG. 16) by the acquisition unit F11 (S41). At least two of the multiple sequential images are obtained by imaging the surface of the object 100 by changing the position of the same camera 32. More specifically, as shown in FIG. 16, the multiple sequential images are obtained by taking images of the surface of the object 100 by the camera 32 at different positions L1, L2, L3, etc. As an example, the positions L1, L2, L3, etc. are arranged so as to surround the object 100 and are on a circle centered on the object 100. Note that the multiple sequential images can also be obtained by multiple cameras installed at different positions L1, L2, L3, etc. Two sequential images (first and second sequential images) of the multiple sequential images will be described below. Note that the first sequential image is an image captured before the second sequential image.

[0051] Next, the determination unit F13 calculates the difference in luminance value between pixels (spatial feature vector) (S42). The determination unit F13 extracts luminance information from the series of images. The luminance information is the difference in luminance value (pixel value) obtained from a plurality of pixels in the series of images. The difference in luminance value is the difference between the luminance value of a first region including one or more of the plurality of pixels in the series of images and the luminance value of a second region adjacent to the first region and including one or more of the plurality of pixels. As an example, the first region may be a region consisting of m×n pixels. Here, m and n are both integers of 1 or more. In this embodiment, the first region is a region consisting of 1×1 pixels (i.e., one pixel). The pixel at the center of the first region is called the first pixel (reference pixel). Here, the first region is the brightest region in the series of images (images composed of a plurality of pixels). Since the first region is only the first pixel, the first pixel is the pixel with the highest luminance value among the plurality of luminance values ​​in the series of images. On the other hand, the second region may be a region surrounding the first region. As an example, the second region may be a region consisting of M×N pixels with the first region at the center. Here, M and N are both integers equal to or greater than 3. In this embodiment, the second region is composed of pixels excluding the first pixel among the multiple pixels of the series of images. That is, the determination unit F13 obtains a difference between the luminance value of the first pixel and the luminance value of each pixel of the series of images, and replaces the luminance value of each pixel with the difference value. Then, the determination unit F13 obtains a feature vector (spatial feature vector) whose elements are the luminance value of the first pixel and the luminance values ​​(difference values) of the multiple pixels of the series of images after replacement. Note that the first pixel is not limited to the pixel with the largest luminance value, and may be any of the pixel with the smallest luminance value, the pixel whose luminance value is the average value of the image, and the pixel at the center of the image.

[0052] Next, the determination unit F13 calculates the difference in luminance value between frames (temporal feature vector) (S43). The determination unit F13 calculates the difference between the luminance value of a first attention region including one or more of the pixels of the first series image and the luminance value of a second attention region corresponding to the first attention region among the pixels of the second series image. That is, the first attention region and the second attention region are selected so as to correspond to the same part of the surface of the object 100. As an example, the first attention region is a region smaller than the first series image. The first attention region may be a region consisting of m×n pixels. Here, m and n are both integers equal to or greater than 1. The central pixel of the first attention region may be any of the pixel with the largest luminance value, the pixel with the smallest luminance value, and the pixel whose luminance value is the average value of the image. In this embodiment, the central pixel of the first attention region is the pixel whose luminance value is the average value of the image. The second attention region is a region (a region consisting of m×n pixels) having the same size as the first attention region. The pixel at the center of the second region of interest is the pixel with the smallest difference in luminance value from the pixel at the center of the first region of interest, and it is preferable that the luminance values ​​are the same. Then, the determination unit F13 obtains the difference in luminance values ​​between the first series of images (first region of interest) and the second series of images (second region of interest), and replaces the luminance value of the second series of images (second region of interest) with the difference value. That is, the determination unit F13 obtains the difference in luminance values ​​between the pixels included in the first region of interest of the first series of images and the pixels included in the second region of interest of the second series of images. This obtains the difference value of the luminance values ​​for m×n pixels. Then, the determination unit F13 obtains a feature vector (temporal feature vector) whose elements are the luminance value of the pixel at the center of the first region of interest and the pixel value (difference value) after replacement of the second series of images.

[0053] Finally, the determination unit F13 calculates the texture (texture amount) (S44). In this embodiment, the texture is given by combining the spatial feature vector and the temporal feature vector. In other words, in this embodiment, the texture of the surface of the object 100 is quantified in the form of combining the spatial feature vector and the temporal feature vector. Then, the determination unit F13 may numerically determine whether the texture satisfies the requirements. For example, the determination unit F13 may determine whether the texture satisfies the requirements based on whether the magnitude of the vector indicating the texture exceeds a threshold. If the texture satisfies the requirements, the determination unit F13 may determine that the texture inspection has passed. If the texture does not meet the requirements, the determination unit F13 may perform repainting or a defective product determination.

[0054] In this manner, in this embodiment, the amount of change in luminance value accompanying the relative displacement between the camera 32 and the object 100 is calculated on the spatial axis and the time axis and integrated. The amount of change in luminance value on the spatial axis (spatial change amount) is given by the difference between the luminance value of a reference pixel and the luminance value of its adjacent pixel. Examples of the reference pixel include the brightest pixel, the darkest pixel, the pixel of average brightness, and the pixel at the center of the image. The amount of change in luminance value on the time axis (temporal change amount) is given by the difference between the reference frame and the adjacent frame. Therefore, in this embodiment, it is possible to measure and digitize the texture (human perception of the surface condition) of the surface of the object 100 (especially a painted surface).

[0055] 1.3 Summary The inspection system 1 described above includes an acquisition unit F11 and a determination unit F13. The acquisition unit F11 acquires an image of the surface of the object 100. The determination unit F13 performs a color determination process. The color determination process is a process for determining the color of the surface of the object 100 based on a plurality of reflection states having different ratios of specular reflection components and diffuse reflection components on the surface of the object 100, which are obtained from the image of the surface of the object 100 acquired by the acquisition unit F11. Therefore, according to the inspection system 1, it is possible to improve the accuracy of determining the color of the surface of the object 100.

[0056] In other words, the inspection system 1 executes the following method (inspection method). The inspection method includes an acquisition step and a determination step. The acquisition step is a step of acquiring an image of the surface of the object 100. The determination step is a step of performing a color determination process. The color determination process is a process of determining the color of the surface of the object 100 based on a plurality of reflection states having different ratios of specular reflection components and diffuse reflection components on the surface of the object 100, which are obtained from the image of the surface of the object 100 acquired by the acquisition unit F11. Therefore, according to the inspection method, like the inspection system 1, it is possible to improve the accuracy of determining the color of the surface of the object 100.

[0057] The inspection method is realized by one or more processors executing a program (computer program). This program is a program for causing one or more processors to execute the inspection method. According to such a program, the accuracy of determining the color of the surface of the object 100 can be improved, similar to the inspection method. The program can be provided by a storage medium. This storage medium is a non-transitory storage medium readable by a computer, and stores the above-mentioned program. According to such a storage medium, the accuracy of determining the color of the surface of the object 100 can be improved, similar to the inspection method.

[0058] From another perspective, the inspection system 1 includes an imaging system 30, an acquisition unit F11, and a determination unit F13. The imaging system 30 captures an image of the surface of the object 100 to generate an image of the surface of the object 100. The acquisition unit F11 acquires the image of the surface of the object 100 from the imaging system 30. The determination unit F13 performs a coating determination process for determining the coating state of the surface of the object 100 based on the image of the surface of the object 100 acquired by the acquisition unit F11, and controls the imaging system 30 according to the result of the coating determination process. This inspection system 1 can improve the quality of the coating on the surface of the object 100.

[0059] From another perspective, the inspection system 1 includes an acquisition unit F11 and a determination unit F13. The acquisition unit F11 acquires a plurality of series of images obtained by imaging the surface of the object 100 from different positions L1 to L3. The determination unit F13 performs texture determination processing to determine the texture of the surface of the object 100 based on a change in luminance information between the plurality of series of images. According to this embodiment, it is possible to further improve the accuracy of determination of the texture of the surface of the object (100).

[0060] 2. Variations The embodiments of the present disclosure are not limited to the above-described embodiments. The above-described embodiments can be modified in various ways depending on the design, etc., as long as the object of the present disclosure can be achieved. Modifications of the above-described embodiments are listed below.

[0061] In the above embodiment, the camera of the imaging system 30 can detect light having a wavelength included in a predetermined band, for example. The predetermined band is, for example, 380 nm to 780 nm. However, the multiple cameras of the imaging system 30 may have filters with different transmission bands. For example, the four first cameras 311 to 314 may be configured to detect light having wavelengths in different bands. As shown in FIG. 17, the light having wavelengths in different bands includes, for example, light having wavelengths of 380 nm to 480 nm (blue light), light having wavelengths of 480 nm to 580 nm (green light), light having wavelengths of 580 nm to 680 nm (yellow light), and light having wavelengths of 680 nm to 780 nm (red light). In this way, by detecting light having wavelengths in different bands with the multiple cameras 31, it is possible to determine the color of the object 100 at a more detailed level. In particular, in the example of FIG. 17, each of the four bands is divided into nine bands of 10 nm each, and the band of 380 nm to 780 nm is divided into 36 bands for detection. Therefore, compared to the relatively commonly used three-band configuration, it is possible to determine the color of the object 100 at a finer level. Note that the same effect can be achieved by using a single camera with multiple filters with different transmission bands.

[0062] In one variant, the wavelength of light emitted by the illumination system 20 may be variable. This may be achieved by using light sources with different emitting colors or color filters. In other words, in the inspection system 1, at least one of the wavelength of light emitted by the illumination system 20 and the wavelength of light detected by the imaging system 3 may be variable.

[0063] In the above embodiment, the multiple partial images P31 to P34 are generated by multiple cameras 31 with different imaging directions relative to the object 100. However, the multiple partial images P31 to P34 may be obtained by imaging the surface of the object 100 with the same camera at different positions.

[0064] In the above embodiment, the multiple reflection states having different ratios of specular reflection components and diffuse reflection components on the surface of the object 100 are in the form of images, but these may be in other formats such as histograms instead of images. In other words, the reflection state of the surface of the object 100 does not necessarily need to be given as an image, and may be in a format that allows color determination based on the reflection state.

[0065] In one modified example, the setting process is not essential. When the setting process is not performed, sample data may be prepared for each of the images generated by the multiple cameras 31, and the color judgment process may be performed. This makes it possible to perform color judgment processes separately for different parts of the surface of the object 100 using the multiple cameras 31. In this case, images are not combined in the color judgment process.

[0066] In the above embodiment, in the repainting, the judgment unit F13 controls the painting system 40 according to the difference between the color obtained from the first composite image and the target color for the first composite image, and the difference between the color obtained from the second composite image and the target color for the second composite image. However, the judgment unit F13 may control the painting system 40 by using a trained model (toning model). The tone model is a trained model that has learned the relationship between the combination of the color before and the color after the correction and the control content of the painting system 40. In this case, the storage unit 12 stores the tone model. The tone model is generated by having an artificial intelligence program (algorithm) learn the relationship between the combination of the color before and the color after the correction and the control content of the painting system 40 using a training data set indicating the relationship between the combination of the color before and the color after the correction and the control content of the painting system 40. The artificial intelligence program is a machine learning model, and for example, a neural network, which is a type of hierarchical model, is used. The tone model is generated by having a neural network perform machine learning (for example, deep learning) using the training data set. That is, the color matching model may be generated by the processing unit 13 of the inspection system 1 or an external system. In the inspection system 1, the processing unit 13 may collect and store learning data for generating the color matching model. The learning data newly collected by the processing unit 13 in this manner can be used to re-learn the color matching model, thereby improving the performance of the color matching model (trained model). In particular, if the result of the color judgment process is again unsuccessful after repainting, the performance of the color matching model can be improved by re-learning.

[0067] In one modified example, the judgment unit F13 may use a model for the texture judgment process. The model is obtained by preparing a plurality of paint samples, creating a pair of the quality judgment of the paint and the texture amount, and modeling the relationship between the two. The modeling may be performed by regression analysis or machine learning. In this case, the judgment unit F13 can execute the quality judgment of the paint based on the texture amount. In one modified example, only the spatial feature amount vector may be used as the texture amount for texture measurement when the positional relationship between the camera 32 and the target object 100 is fixed.

[0068] In one variation, in the texture assessment process, the luminance information may be a difference in luminance values ​​obtained from a plurality of pixels in the series of images. This difference may be a difference between a luminance value of a first region including one or more of the plurality of pixels and a luminance value of a second region adjacent to the first region and including one or more of the plurality of pixels. Furthermore, the first region may be a first pixel of the plurality of pixels. The second region may be a second pixel of the plurality of pixels adjacent to the first pixel. Furthermore, the first region may be the brightest region in an image composed of a plurality of pixels.

[0069] In one modified example, the inspection system 1 (determination system 10) may be configured with a plurality of computers. For example, the functions of the inspection system 1 (determination system 10) (particularly, the acquisition unit F11, the separation unit F12, and the determination unit F13) may be distributed among a plurality of devices. Furthermore, at least a part of the functions of the inspection system 1 (determination system 10) may be realized, for example, by cloud (cloud computing).

[0070] The execution subject of the inspection system 1 (determination system 10) described above includes a computer system. The computer system has a processor and a memory as hardware. The processor executes a program recorded in the memory of the computer system, thereby realizing the function of the execution subject of the inspection system 1 (determination system 10) in the present disclosure. The program may be pre-recorded in the memory of the computer system, or may be provided through an electric communication line. The program may also be recorded and provided in a non-transitory recording medium such as a memory card, an optical disk, or a hard disk drive that can be read by the computer system. The processor of the computer system is composed of one or more electronic circuits including a semiconductor integrated circuit (IC) or a large scale integrated circuit (LSI). Field programmable gate arrays (FGPAs), application specific integrated circuits (ASICs), or reconfigurable logic devices that can reconfigure the connection relationship within the LSI or set up circuit sections within the LSI, which are programmed after the manufacture of the LSI, can also be used for the same purpose. The multiple electronic circuits may be integrated in one chip or distributed across multiple chips. The multiple chips may be integrated in one device or distributed across multiple devices.

[0071] 3. Aspects As is apparent from the above-described embodiment and modified examples, the present disclosure includes the following aspects. In the following, reference symbols are given in parentheses only to clarify the correspondence with the embodiment.

[0072] The first aspect is an inspection system (1) comprising an acquisition unit (F11) and a determination unit (F13). The acquisition unit (F11) acquires images (P30-P34) of the surface of an object (100). The determination unit (F13) performs a color determination process. The color determination process is a process for determining the color of the surface of the object (100) based on a plurality of reflection states having different ratios of specular reflection components and diffuse reflection components on the surface of the object (100), which are obtained from the images (P30-P34) of the surface of the object 100 acquired by the acquisition unit (F11). According to this aspect, it is possible to improve the accuracy of the determination of the color of the surface of the object (100).

[0073] The second aspect is based on the inspection system (1) of the first aspect. In the second aspect, the inspection system (1) further includes a separation unit (F12). The separation unit (F12) acquires a plurality of separated images (P10, P20) each representing an image (P30-P34) of the surface of the object (100) and having a different ratio of specular reflection component to diffuse reflection component from the image acquired by the acquisition unit (F11). In the color judgment process, the judgment unit (F13) judges the color of the surface of the object (100) based on the plurality of separated images (P10, P20). According to this aspect, the accuracy of judging the color of the surface of the object (100) can be further improved. In addition, the efficiency of the color judgment process can be improved.

[0074] The third aspect is based on the inspection system (1) of the first or second aspect. In the third aspect, the acquisition unit (F11) acquires a plurality of partial images (P31-P34) representing parts of the surface of the object (100) as images of the surface of the object (100). In the color judgment process, the judgment unit (F13) judges the color of the surface of the object (100) based on images (P10, P20) representing the entire surface of the object (100) in each of the plurality of reflection states obtained from the plurality of partial images. According to this aspect, it becomes possible to judge the color of the surface of a relatively large object (100).

[0075] The fourth aspect is based on the inspection system (1) of the third aspect. In the fourth aspect, the plurality of partial images (P31 to P34) are generated by a plurality of cameras (31) having different imaging directions with respect to the object (100). According to this aspect, it is possible to determine the color of the surface of a relatively large object (100) with a simple configuration.

[0076] The fifth aspect is based on the inspection system (1) of any one of the first to fourth aspects. In the fifth aspect, the inspection system (1) further includes an illumination system (20) and an imaging system (30). The illumination system (20) irradiates light onto the surface of the object (100). The imaging system (30) captures an image of the surface of the object (100) illuminated by the illumination system (20) to generate an image of the surface of the object (100). The acquisition unit (F11) acquires the image of the surface of the object (100) from the imaging system (30). At least one of the wavelength of the light emitted by the illumination system (20) and the wavelength of the light detected by the imaging system (30) is changeable. According to this aspect, the accuracy of determining the color of the surface of the object (100) can be improved.

[0077] The sixth aspect is based on the inspection system (1) according to any one of the first to fifth aspects. In the sixth aspect, the judgment unit (F13) judges the color of the surface of the object (100) by using sample data including information on a target color of the surface of the object (100) in the color judgment process. According to this aspect, it is possible to further improve the accuracy of judging the color of the surface of the object (100).

[0078] The seventh aspect is based on the inspection system (1) of the sixth aspect. In the seventh aspect, the sample data includes at least one of the shape of the object (100) and the imaging conditions of the object (100). According to this aspect, it is possible to further improve the accuracy of determining the color of the surface of the object (100).

[0079] The eighth aspect is based on the inspection system (1) according to any one of the first to seventh aspects. In the eighth aspect, the judgment unit (F13) controls a coating system (40) that coats the surface of the object (100) based on a result of the color judgment process. According to this aspect, the quality of the coating on the surface of the object (100) can be improved.

[0080] The ninth aspect is based on the inspection system (1) of the first aspect. In the ninth aspect, the inspection system (1) further includes an imaging system (30) that captures an image of the surface of the object (100) to generate an image of the surface of the object (100). The acquisition unit (F11) acquires the image of the surface of the object (100) from the imaging system (30). The judgment unit (F13) performs a coating judgment process to judge the coating state of the surface of the object (100) based on the image of the surface of the object (100) acquired by the acquisition unit (F11), and controls the imaging system (30) according to the result of the coating judgment process. According to this aspect, the quality of the coating on the surface of the object (100) can be improved.

[0081] The tenth aspect is based on the inspection system (1) of the ninth aspect. In the tenth aspect, the judgment unit (F13) determines a difference between a coating state of the surface of the object (100) obtained from an image of the surface of the object (100) acquired by the acquisition unit (F11) and a target coating state of the surface of the object (100). According to this aspect, it is possible to improve the quality of the coating of the surface of the object (100).

[0082] The eleventh aspect is based on the inspection system (1) of the ninth or tenth aspect. In the eleventh aspect, the imaging system (30) includes a plurality of cameras (31, 32). The judgment unit (F13) controls the operating state of the plurality of cameras (31, 32) of the imaging system (30) according to a result of the coating judgment process. According to this aspect, it is possible to improve the quality of the coating on the surface of the object (100).

[0083] The twelfth aspect is based on the inspection system (1) of the eleventh aspect. In the twelfth aspect, the multiple cameras (31, 32) include one or more first cameras (31) that generate an image of a portion of the surface of the object (100) and a second camera (32) that generates an image of the entire surface of the object (100). The judgment unit (F13) controls the operating state of the one or more first cameras (31) depending on the result of the coating judgment process. According to this aspect, the quality of the coating on the surface of the object (100) can be improved.

[0084] The thirteenth aspect is based on the inspection system (1) of the twelfth aspect. In the thirteenth aspect, the judgment unit (F13) controls the operation state of the one or more first cameras (31) according to the image captured by the second camera (32) and the result of the coating judgment process. According to this aspect, the quality of the coating on the surface of the object (100) can be improved.

[0085] The fourteenth aspect is based on the inspection system (1) of the first aspect. In the fourteenth aspect, the acquisition unit (F11) acquires a plurality of series of images obtained by imaging the surface of the object (100) from different locations (L1 to L3). The determination unit (F13) performs a texture determination process for determining the texture of the surface of the object (100) based on a change in luminance information between the plurality of series of images. According to this aspect, the accuracy of the determination of the texture of the surface of the object (100) can be further improved.

[0086] The fifteenth aspect is based on the inspection system (1) of the fourteenth aspect. In the fifteenth aspect, at least two of the plurality of series of images are obtained by imaging the surface of the object (100) with the same camera at different positions. According to this aspect, it is possible to further improve the accuracy of determining the texture of the surface of the object (100).

[0087] A sixteenth aspect is based on the inspection system (1) of the fourteenth or fifteenth aspect. In the sixteenth aspect, each of the plurality of series of images includes a plurality of pixels. The luminance information includes a difference in luminance values ​​obtained from the plurality of pixels. According to this aspect, it is possible to further improve the accuracy of determining the texture of the surface of the object (100).

[0088] The 17th aspect is based on the inspection system (1) of the 16th aspect. In the 17th aspect, the difference is a difference between a luminance value of a first region including one or more of the plurality of pixels and a luminance value of a second region adjacent to the first region and including one or more of the plurality of pixels. According to this aspect, it is possible to further improve the accuracy of determining the texture of the surface of the object (100).

[0089] The 18th aspect is based on the inspection system (1) of the 17th aspect. In the 18th aspect, the first region is a first pixel of the plurality of pixels. The second region is a second pixel of the plurality of pixels adjacent to the first pixel. According to this aspect, it is possible to further improve the accuracy of determining the texture of the surface of the object (100).

[0090] A 19th aspect is based on the inspection system (1) of the 17th or 18th aspect. In the 19th aspect, the first region is the brightest region in the image composed of the plurality of pixels. According to this aspect, it is possible to further improve the accuracy of determining the texture of the surface of the object (100).

[0091] The twentieth aspect is an inspection method including an acquisition step and a determination step. The acquisition step is a step of acquiring an image (P30-P34) of the surface of the object (100). The determination step is a step of performing a color determination process. The color determination process is a process of determining the color of the surface of the object (100) based on a plurality of reflection states having different ratios of specular reflection components and diffuse reflection components on the surface of the object (100), which are obtained from the image (P30-P34) of the surface of the object 100 acquired by the acquisition unit (F11). According to this aspect, it is possible to improve the accuracy of the determination of the color of the surface of the object (100). Note that the inspection system (1) of the second to nineteenth aspects can be applied to the twentieth aspect by reading it as an inspection method.

[0092] A twenty-first aspect is a program for causing one or more processors to execute the inspection method of the twentieth aspect. According to this aspect, it is possible to improve the accuracy of determining the color of the surface of the object (100).

[0093] A twenty-second aspect is a non-transitory computer-readable storage medium storing the program of the twenty-second aspect. According to this aspect, it is possible to improve the accuracy of determining the color of the surface of the object (100).

[0094] Furthermore, the present disclosure includes the following twenty-third to thirty-fourth aspects.

[0095] A twenty-third aspect is a color inspection device including a camera having a replaceable filter that transmits light in a specific band, and inspects the color of an object to be photographed using an image captured by the camera.

[0096] A 24th aspect is based on the color inspection device of the 23rd aspect. In the 24th aspect, the color inspection device includes a plurality of the cameras, synthesizes images from the plurality of cameras to create a composite image, and calculates and outputs imaging conditions from the composite image.

[0097] A 25th aspect is based on the color inspection device of the 24th aspect. In the 25th aspect, the filters provided in the plurality of cameras have different transmission bands from each other.

[0098] A 26th aspect is based on the color inspection apparatus of the 24th or 25th aspect. In the 26th aspect, the plurality of cameras photograph the object from different directions.

[0099] A 27th aspect is based on the coloring inspection device according to any one of the 24th to 26th aspects. In the 27th aspect, the coloring inspection device controls the angle of view and zooming of the camera so that the continuity of the gradation of the photographed object is maintained from the composite image.

[0100] A 28th aspect is based on the coloring inspection device of the 27th aspect. In the 28th aspect, the coloring inspection device controls the camera using shape information and illumination information of an object to be photographed.

[0101] A 29th aspect is based on the coloring inspection device according to any one of the 23rd to 29th aspects. In the 29th aspect, the coloring inspection device records sample color data, compares the color of the photographed object with the sample color data, and controls a painting unit that paints the photographed object.

[0102] A 30th aspect is a coloring inspection method, which includes photographing an object from different directions using a camera equipped with a plurality of filters with different transmission bands, combining the images from the multiple cameras to create a composite image, and inspecting the color of the object from the composite image.

[0103] A 31st aspect is based on the staining inspection method of the 30th aspect. In the 31st aspect, the staining inspection method outputs an imaging condition from the composite image.

[0104] A 32nd aspect is based on the color inspection method of the 30th or 31st aspect. In the 32nd aspect, the color inspection method controls the angle of view and zooming of the camera so that continuity of gradation of the photographed object is maintained from the composite image.

[0105] A 33rd aspect is based on the coloring inspection device of the 31st aspect. In the 33rd aspect, the coloring inspection device controls the camera using shape information and illumination information of an object to be photographed.

[0106] A 34th aspect is based on the coloring inspection device according to any one of the 30th to 33rd aspects. In the 34th aspect, the coloring inspection device records sample color data, compares the color of the photographed object with the sample color data, and controls a painting unit that paints the photographed object.

[0107] Furthermore, the present disclosure includes the following thirty-fifth to thirty-seventh aspects.

[0108] A 35th aspect is a system comprising a painting information acquisition unit (meaning a group of cameras) that acquires information on the painting condition at a certain point in time, a reference information storage unit that holds reference information indicating the desired painting condition at that point in time, the painting information acquisition unit, and a control unit connected to the reference information storage unit, wherein the control unit determines the difference between the information obtained from the painting information acquisition unit and the information obtained from the reference information storage unit, and transmits a control command to the painting information acquisition unit (meaning the group of cameras) based on the difference.

[0109] A 36th aspect is based on the system of the 35th aspect. In the 36th aspect, the control command changes an operation state (meaning pan, zoom) of a camera included in a painting information acquisition unit (meaning a group of cameras).

[0110] A 37th aspect is based on the system of the 36th aspect. In the 37th aspect, the painting information acquisition unit (meaning a camera group) has an overhead camera and a local camera, and the control command changes an operation state (meaning pan, zoom) of the local camera.

[0111] Furthermore, the present disclosure includes the following thirty-eighth to forty-first aspects.

[0112] A 38th aspect is a photographing method including the steps of displacing an imaging device relative to an object, and acquiring a change in brightness value information contained in information photographing the object before and after the displacement.

[0113] A 39th aspect is based on the imaging method of the 38th aspect. In the 39th aspect, information acquired by the imaging device before the displacement is defined as image information, the image information includes a plurality of pixels, and the imaging method acquires a change in a difference in luminance value between the plurality of pixels before and after the displacement.

[0114] A 40th aspect is based on the imaging method of the 39th aspect. In the 40th aspect, the difference is a difference between a luminance value of a first pixel of the image information and a luminance value of a second pixel adjacent to the first pixel.

[0115] A 41st aspect is based on the imaging method of the 40th aspect. In the 41st aspect, the first pixel is a pixel having the highest luminance value in the image information.

[0116] This application claims priority to U.S. Provisional Application No. 62 / 596,247, filed December 8, 2017, U.S. Provisional Application No. 62 / 699,935, filed July 18, 2018, and U.S. Provisional Application No. 62 / 699,942, filed July 18, 2018, the entire contents of which are incorporated by reference into the disclosure of this application. [Explanation of symbols]

[0117] 1. Inspection system F11 Acquisition Department F12 Separation part F13 Judgment section 20 Lighting System 30 Imaging System 31 Camera (1st Camera) 32 Camera (2nd Camera) 40 Painting System P10 First separated image (separated image) P20 Second separated image (separated image) P30 Images P30~P34 Images (partial images) L1~L3 Location 100 Objects

Claims

1. A determination unit is provided for determining a surface condition of the object, The determination unit is Acquiring a plurality of partial images in which specular reflection components are dominant by imaging the object under imaging conditions that maintain the continuity of the object; generating a composite image from the plurality of partial images such that continuity of the object is maintained; determining a surface condition of the object from the composite image; the plurality of partial images are obtained by imaging a surface of the object by displacing a plurality of cameras relative to the object; Inspection system.

2. The plurality of partial images in which the specular reflection component is dominant are images acquired in an area corresponding to a predetermined range (θ±φ) centered on a reflection angle θ on an imaging surface. The inspection system of claim 1 .

3. The plurality of partial images in which the specular reflection component is dominant are images acquired in a region on an imaging plane where an incident angle of light from a light source to a surface of the object is equal to an angle of reflection of the light on the surface. The inspection system of claim 2.

4. an illumination system for illuminating a surface of the object with light; the illumination system includes a lamp for irradiating the object with light; The lamp emits white light. The inspection system according to any one of claims 1 to 3.

5. The determination unit determines a color of a surface of the object. The inspection system of claim 4.

6. An inspection method for determining a surface condition of an object, comprising: A first step of acquiring a plurality of partial images in which specular reflection components are dominant, by imaging the object under imaging conditions that maintain continuity of the object; a second step of generating a composite image from the plurality of partial images such that continuity of the object is maintained; and a third step of determining a surface condition of the object from the composite image, the plurality of partial images are obtained by imaging a surface of the object by displacing a plurality of cameras relative to the object; Testing method.

Images