Inspection apparatus for an object and inspection method for an object

Abstract

To provide an object inspection device that can check that the visual field of a picked-up image of a product is in an appropriate state.SOLUTION: An inspection device inspects the appearance of an object by using imaging data obtained by picking up an image of the object. The inspection device includes: an image acquisition unit that acquires a reference image, and imaging data obtained by picking up an image of an object at a position along an optical axis of an imaging unit picking up the reference image; a probability map data acquisition unit that acquires probability map data indicating the presence probability of the object for every pixel; an area setting unit that, based on the probability map data, sets the imaging data and a specific area including the background in an image area in the reference image; an extraction image creation unit that extracts first extraction data from the imaging data by using the area set by the area setting unit, and extracts second extraction data from the reference image by using the area; and a display unit that displays the first extraction data and the second extraction data.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an inspection apparatus for an object that inspects the appearance of an object such as a steel product, and an inspection method for an object.

Background Art

[0002] Conventionally, the appearance of a product has been inspected using an image obtained by imaging the product. In such an inspection of the appearance of a product using an image, it is desirable that the field of view of the image be constant in order to maintain inspection accuracy.

[0003] In this inspection, an imaging area of a product to be inspected is extracted from an image, and it is determined whether there is an abnormality in the appearance of the product. As a method for extracting the imaging area from the image, for example, the background is removed from an extraction target image including the background and the product to be extracted.

[0004] As such an image extraction process, for example, an imaging device that aligns an image in which an object to be extracted is reflected and an image in which the object is not reflected, and removes the background by the difference between the two images is disclosed in Patent Document 1.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, even if the object is extracted using the imaging device described in Patent Document 1, it is impossible to confirm whether the field of view of the image is in an appropriate state. For this reason, conventionally, the user has visually confirmed the field of view by viewing an image captured in the past and an image captured most recently, and there is a demand for the development of a device or the like that confirms that the field of view of the captured image is appropriate.

[0007] The present invention has been made in view of the above problems, and an object inspection apparatus capable of confirming that the field of view of an image is in an appropriate state when inspecting the appearance of a product using an image obtained by imaging the product, and an object inspection method are provided.

Means for Solving the Problems

[0008] [1] An inspection apparatus for inspecting the appearance of an object using imaging data obtained by imaging the object, an image acquisition unit that acquires a reference image including a background arranged around the object, and imaging data including the object and the background and imaged from a position along the optical axis of an imaging unit where the reference image was imaged; a probability map data acquisition unit that acquires probability map data indicating the probability of existence of the object for each pixel in an image region corresponding to the imaging data; a region setting unit that sets a specific region including the background in the image regions in the imaging data and the reference image based on the probability map data; an extraction image generation unit that extracts the imaging data using the region set by the region setting unit to generate first extraction data, and extracts the reference image using the region to generate second extraction data; a display unit that displays the first extraction data and the second extraction data; an inspection unit that inspects the appearance of the object using the other extraction data including the object in the imaging data; An object inspection apparatus including the above. [2] The object inspection apparatus according to [1], further comprising a determination unit that determines the degree of coincidence between the field of view of the imaging data and the field of view of the reference image with reference to the first extraction data and the second extraction data generated by the extraction image generation unit. [3] The probability map data is generated using the determination result of the degree of coincidence between the field of view of the imaging data and the field of view of the reference image, and is an inspection apparatus for an object according to [1] or [2]. [4] The reference images are captured in different environments. The image acquisition unit selects and acquires the reference image captured in an environment similar to the environment in which the imaging data is captured, and is an inspection apparatus for an object according to any one of [1] to [3]. [5] The determination unit associates a feature point set in one of the first extraction data and the second extraction data with a feature point in the other extraction data corresponding to the feature point. The feature point is set based on the amount of change with respect to the periphery of each pixel of the one extraction data, and is an inspection apparatus for an object according to any one of [2] to [4]. [6] The determination unit determines using the learning result by deep learning, and is an inspection apparatus for an object according to any one of [2] to [5]. [7] The first extraction data and the second extraction data are displayed on the display unit in a state of overlapping each other, and is an inspection apparatus for an object according to any one of [1] to [6]. [8] An inspection method for inspecting the appearance of an object using imaging data in which the object is imaged, An image acquisition step of acquiring a reference image including a background arranged around the object and imaging data including the object and the background and captured from a position along the optical axis of the imaging unit in which the reference image is captured; A probability map data acquisition step of acquiring probability map data indicating the probability of existence of the object for each pixel in the image area corresponding to the imaging data; An area setting step of setting a specific area including the background in the image areas in the imaging data and the reference image based on the probability map data; An extraction image generation step of extracting first extraction data from the imaging data and extracting second extraction data from the reference image using the area set in the area setting step; A display step of displaying the first extraction data and the second extraction data; An inspection step of inspecting the appearance of the object using other extraction data including the object in the imaging data; An inspection method for an object, including: [9] The inspection method for an object according to [8], further comprising a determination step of determining the degree of coincidence between the field of view of the imaging data and the field of view of the reference image with reference to the first extraction data and the second extraction data generated by the extraction image generation step.

[10] The inspection method for an object according to [8] or [9], wherein the probability map data is generated using the determination result of the degree of coincidence between the field of view of the imaging data and the field of view of the reference image.

[11] The reference image is captured in different environments; The inspection method for an object according to any one of [8] to

[10] , wherein the image acquisition step selects and acquires the reference image captured in an environment similar to the environment in which the imaging data is captured.

[12] In the determination step, feature points set in one of the first extraction data and the second extraction data are associated with feature points in the other extraction data corresponding to the feature points; The inspection method for an object according to [9], wherein the feature points are set based on the amount of change with respect to the periphery of each pixel of the one extraction data.

[13] The inspection method for an object according to any one of [9] to

[11] , wherein in the determination step, the determination is made using the learning result by deep learning.

[14] The inspection method for an object according to any one of [8] to

[13] , wherein in the display step, the first extraction data and the second extraction data are displayed in a state of overlapping each other.

Advantages of the Invention

[0009] According to the present invention, based on probability map data indicating the probability of the presence of an object, a specific region to be extracted is set in the image regions of the imaging data and the reference image, and first extraction data is extracted from the imaging data using the region, and second extraction data is extracted from the reference image. By visually recognizing the first extraction data and the second extraction data displayed on the display unit, the user can easily determine whether the field of view of the imaging data is in an appropriate state.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Embodiments for Carrying Out the Invention

[0011] (First Embodiment) FIG. 1 is a block diagram showing the configuration of an inspection apparatus for an object (hereinafter also referred to as an inspection apparatus). As shown in FIG. 1, the inspection apparatus 100 includes an imaging unit 10 that images an object, an image database (hereinafter, the database is referred to as a DB) 20 that stores imaging data captured by the imaging unit 10, and a display unit 30 that displays the imaging data. The inspection apparatus 100 includes a control unit 40 that controls the entire inspection apparatus 100. Each of the imaging unit 10, the image DB 20, the display unit 30, and the control unit 40 is connected via a bus 50 so as to be capable of data communication.

[0012] The imaging unit 10 can use, for example, a camera having an imaging element (not shown) that converts light incident through a lens (not shown) into image data. The imaging unit 10 generates a still image as imaging data.

[0013] The imaging unit 10 is not particularly limited as long as it can generate a two-dimensional image. The imaging unit 10 can also be configured using a line scanner, a scanning optical sensor, or a sensor that detects a signal other than light (for example, a magnetic signal).

[0014] For example, a magnetic sensor is used to detect foreign objects by utilizing changes in magnetic field lines inside a metal material. When the imaging unit 10 is configured using a magnetic sensor, based on the intensity change of the magnetic signal generated by the magnetic sensor, the intensity is converted into components such as color or brightness, and an image is generated. Thus, the image can be used as imaging data.

[0015] The image DB 20 is not particularly limited, and known storage means such as an HDD (hard disk drive) or an SSD (solid state drive) can be used. The imaging data captured by the imaging unit 10 is stored in the image DB 20.

[0016] Specifically, the image DB 20 stores a reference image including the background arranged around the object to be inspected, and imaging data captured from a position along the optical axis of the imaging unit 10 that includes the object and the background and from which the reference image was captured. The image DB 20 also stores probability map data indicating the probability of the presence of the object for each pixel in the image area corresponding to the imaging data and the reference image.

[0017] The reference image may be image data in which only the background is captured, or may be image data used in past inspections from which the background has been extracted.

[0018] The probability map data is data indicating the probability of the presence of the object for each pixel. The probability map data has, for example, a percentage or a numerical value corresponding to the percentage input for each pixel. The probability map data may be, for example, one in which each pixel is represented by a color corresponding to the percentage.

[0019] The probability map data is created, for example, based on past inspection data. Specifically, the case where the object is present at a specific pixel is counted as "1", and the case where the object is not present is counted as "0". This counting is performed for N pieces of image data, and the percentage can be calculated by dividing the total value of this counting by the number N of the image data. By calculating this percentage for each pixel, the probability map data can be created.

[0020] Note that the image DB 20 is not limited to being configured integrally with the inspection apparatus 100, and may be stored, for example, in a server apparatus (not shown) capable of data communication with the inspection apparatus 100.

[0021] The display unit 30 may be any device that can display image data. For example, a liquid crystal display, an organic EL display, etc. can be used. In addition, a VR (Virtual Reality) headset, a projector, an LED vision in which LED (Light Emitting Diode) elements capable of emitting light of a plurality of colors are arranged, etc. can also be used.

[0022] The control unit 40 is a computer composed of a CPU, a ROM, and a RAM. The control unit 40 includes an image acquisition unit 41 that acquires a reference image and imaging data, and a probability map data acquisition unit 42 that acquires probability map data. The control unit 40 includes a region setting unit 43 that sets a specific region in the imaging data and the reference image, and an extraction image generation unit 44 that generates first extraction data and second extraction data using the region, and an inspection unit 45 that inspects the appearance of the object.

[0023] Each part 41 - 45 of the control unit 40 reads data and programs (computer software) stored in the ROM, and based on the data, executes arithmetic processing according to the program to be realized.

[0024] The region setting unit 43 sets a threshold value for the value of the probability map data. The region setting unit 43 sets the region composed of pixels exceeding the threshold value as the object region where the object exists. The region setting unit 43 sets the region composed of pixels below the threshold value as the background region where the object does not exist.

[0025] The extraction image generation unit 44 extracts the object region and the background region in the imaging data using the object region and the background region set by the region setting unit 43. Specifically, the extraction image generation unit 44 multiplies the specific pixels that are the background region of the imaging data by "1" and multiplies the specific pixels that are the object region of the imaging data by "0". By performing such processing, the extraction image generation unit 44 can make the object region black while keeping the background region as its original pixel value. The extraction image generation unit 44 can generate the first extraction data by separating and extracting the background region and the object region in this way. The generated first extraction data is stored in the image DB 20.

[0026] The extraction image generation unit 44 extracts the object region and the background region in the reference image using the object region and the background region set by the region setting unit 43. The extraction image generation unit 44 performs the same processing as the extraction processing of the first extraction data, and can generate the second extraction data by separating and extracting the background region and the object region. The generated second extraction data is stored in the image DB 20.

[0027] By performing such processing, the extraction image generation unit 44 can generate an extraction image as if the background region is cut out for both the imaging data and the reference image.

[0028] The inspection unit 45 inspects the appearance of the object using the other extraction data including the object in the imaging data. Here, it is preferable that the other extraction data is the data restored by the extraction image generation unit 44 before the inspection by the inspection unit 45 is performed. The other extraction data is not limited to such data. For example, reprocessing in which the processing performed by the extraction image generation unit 44 on the background region and the object region is reversed may be performed so that something as if the object region is cut out is used.

[0029] Specifically, the inspection unit 45 compares the inspection image created for inspection stored in the image DB 20 with the other extraction data, and inspects whether there is any abnormality in the appearance of the object by image processing.

[0030] FIG. 2 shows the inspection mode of the object. The objects OB1, OB2, and OB3 to be inspected for appearance may have the same appearance or may have different appearances from each other. In FIG. 2, the objects OB1, OB2, and OB3 have different appearances from each other.

[0031] The objects OB1, OB2, and OB3 are not particularly limited, and examples include steel products such as thin plates, thick plates, section steels, steel pipes, and electromagnetic steel sheets.

[0032] The objects OB1, OB2, and OB3 are conveyed in the direction of the arrow in the figure by a conveying device 60 such as a carriage. An imaging unit 10 is arranged on the downstream side of the conveying device 60.

[0033] The imaging unit 10 is arranged at a position where it can individually image each of the objects OB1, OB2, and OB3 placed on the conveying device 60 and has a field of view. The imaging unit 10 is arranged at a position where it can also image the background arranged around one of the objects OB1, OB2, and OB3. Note that the field of view can be adjusted by the arrangement position of the imaging unit 10, the angle of the optical axis of the lens of the imaging unit 10 with respect to the conveying device 60, the magnification of the lens, and the like.

[0034] The imaging unit 10 can use one in which the unit for imaging the reference image and the unit for imaging the imaging data are arranged at the same position. The imaging unit 10 is not limited to being arranged in such a positional relationship, and may be arranged at a position where a field of view substantially the same as the field of view of the reference image can be obtained.

[0035] For example, the imaging unit 10 that captures imaging data may be arranged at a position along the optical axis of the imaging unit 10 that captures the reference image. Specifically, in the axial direction of the optical axis of the imaging unit 10 that captures the reference image, an imaging unit 10 that captures imaging data can be provided at a position farther from the transport device 60 than the imaging unit 10. In this case, it is preferable to adjust the field of view by adjusting the zoom of the lens, or to use only the image area corresponding to the reference image in the imaging data.

[0036] Figure 3 shows a mode in which probability map data is generated. Above Figure 3, imaging data D1 in which the object OB1 is imaged, imaging data D2 in which the object OB2 is imaged, and imaging data D3 in which the object OB3 is imaged are shown.

[0037] The imaging data D1 to D3 are data imaged in the same field of view in the present embodiment. The objects OB1 to OB3 are, for example, thin plate coils. As shown in the imaging data D1 to D3, the object OB2 has a larger diameter than the object OB1. Also, the object OB3 has a smaller diameter than the objects OB1 and OB2.

[0038] In the center of Figure 3, probability map data MD generated using the imaging data D1 to D3 is shown. The region R1 located near the center of the probability map data MD is a region where the objects OB1 to OB3 are reflected in the imaging data D1 to D3, so the probability of the presence of the object is higher than that of the other regions R2 to R4.

[0039] The region R2 is formed so as to surround the region R1. Since the region R2 is a region where the objects OB1 and OB2 are reflected in the imaging data D1 to D2, the probability of the presence of the object is higher than that of the other regions R3 and R4.

[0040] The region R3 is formed so as to surround the regions R1 and R2. Since the region R3 is a region where the object OB2 is reflected in the imaging data D2, the probability of the presence of the object is higher than that of the other region R4.

[0041] Region R4 is formed so as to surround regions R1 to R3. Since region R4 corresponds to the background region in which the objects OB1 to OB3 are not reflected in the imaging data D1 to D3, the probability of the presence of the object is lower than that of the other regions R1 to R3. Thus, regions 1 to 4 have the presence probability according to the shapes of the objects OB1 to OB3 when the probability map data MD is generated.

[0042] FIG. 4 shows a processing flow showing a method for inspecting an object. The processing of the object inspection method is started, for example, when the power of the object inspection apparatus 100 is turned on. As shown in FIG. 3, the image acquisition unit 41 acquires the imaging data stored in the image DB20 and the reference image (step S101).

[0043] The probability map data acquisition unit 42 acquires the probability map data MD stored in the image DB20 (step S102).

[0044] The region setting unit 43 sets a threshold value for each pixel value of the probability map data MD, and sets an object region and a background region based on the threshold value (step S103).

[0045] The extracted image generation unit 44 extracts first extracted data from the imaging data and extracts second extracted data from the reference image using the object region and the background region set by the region setting unit 43 in the region setting step of step S103 (step S104). Note that the generated first extracted data and second extracted data are stored in the image DB20.

[0046] The control unit 40 reads out the first extracted data and the second extracted data from the image DB20 and causes the display unit 30 to display them (step S105).

[0047] The inspection unit 45 inspects the appearance of the object using the other first extracted data including the object among the imaging data generated in step S103 (step S106).

[0048] FIG. 5 shows a mode in which the first extraction data is generated in the extraction image generation step of step S104. In FIG. 5, an example of the first extraction data DA is shown. The object region OR arranged at the center of the first extraction data DA is colored black by the extraction image generation process of step S104. On the other hand, the color scheme of the background region BR arranged around the object region OR is maintained by the extraction image generation process of step S104.

[0049] FIG. 6 shows a display mode in the display step of step S105 in FIG. 4. In FIG. 6, the first extraction data DA is shown by a solid line on the display surface 31 of the display unit 30. Also, the second extraction data DB is shown by a broken line on the display surface 31 of the display unit 30. The first extraction data DA and the second extraction data DB are image regions of the same size, but in FIG. 6, for the sake of explanation, the second extraction data DB is shown smaller than the first extraction data DA.

[0050] The first extraction data DA and the second extraction data DB are displayed so as to overlap in a direction perpendicular to the display surface 31. In the example shown in FIG. 6, the second extraction data DB is shown on the first extraction data DA. Also, the second extraction data DB is shown in a color with transparency.

[0051] Therefore, if the first extraction data DA and the second extraction data DB are in the same field of view, the user can visually recognize the background regions BR of the first extraction data DA and the second extraction data DB without discomfort.

[0052] On the other hand, if the fields of view of the first extracted data DA and the second extracted data DB do not match each other, the background region BR of the first extracted data DA and the second extracted data DB will shift, and the background region BR will appear blurred to the user. Also, if the fields of view of the first extracted data DA and the second extracted data DB do not match each other, an object may be reflected in the background region BR. By visually recognizing such differences, the user can recognize that the fields of view of the first extracted data DA and the second extracted data DB are different from each other.

[0053] Note that the mode of displaying the first extracted data DA and the second extracted data on the display surface 31 of the display unit 30 is not limited to such a mode. For example, they may be displayed side by side in the left - right direction or the up - down direction of the display surface 31.

[0054] As described above, according to the present invention, based on the probability map data MD indicating the existence probabilities of the objects OB1 to OB3, a specific region to be extracted is set from among the imaging data and the image regions of the reference images. Also, the first extracted data DA and the second extracted data DB generated using the region are generated. The user can easily determine whether the field of view of the imaging data is in an appropriate state by visually recognizing the first extracted data DA and the second extracted data DB displayed on the display unit 30.

[0055] Note that in the region setting step of step S103, a threshold value was set for each pixel value of the probability map data MD, and based on the threshold value, an object region and a background region were set. However, the region setting step by the extraction image generation unit 44 is not limited to such a mode. The object region and the background region may be set by weighting according to the magnitude of the percentage value of each pixel of the probability map data MD. Specifically, the extraction image generation unit 44 may set the object region and the background region by multiplying the pixel values of the reference image and the imaging data by a numerical value corresponding to the percentage of the pixels of the corresponding probability map data MD.

[0056] (Second Embodiment) In the above-described embodiment, the user determines whether the field of view of the imaging data is in an appropriate state by visually recognizing the first extracted data DA and the second extracted data DB displayed on the display unit 30. Whether the field of view of the imaging data matches that of others than the user may be determined.

[0057] FIG. 7 is a block diagram showing the configuration of the inspection apparatus 200 according to the second embodiment. As shown in FIG. 7, the inspection apparatus 200 according to the second embodiment differs from the inspection apparatus 100 according to the first embodiment in that the determination unit 46 is provided in the control unit 40. Since the remaining points are the same as those of the inspection apparatus 100 according to the first embodiment, the same reference numerals are given to the same parts and the description thereof is omitted.

[0058] As shown in FIG. 7, the determination unit 46 of the control unit 40 refers to the first extracted data DA and the second extracted data DB, and based on these, determines the degree of coincidence between the field of view of the imaging data and the field of view of the reference image.

[0059] The determination unit 46, for example, associates the feature points of the first extracted data DA with the feature points of the second extracted data DB, and determines the degree of coincidence according to the similarity thereof. The feature points are set based on the amount of change with respect to the periphery of each pixel of one of the extracted data. Specifically, as the feature points, pixels whose value of a specific pixel has a larger amount of change than the values of the pixels around the specific pixel can be used.

[0060] For the extraction of the feature points, for example, an algorithm such as SIFT (Scale Invariant Feature Transform) can be used. The determination unit 46 extracts the feature points of, for example, the first extracted data DA that has been grayscale-converted, and extracts the feature points of the second extracted data DB corresponding to the feature points.

[0061] The determination unit 46 obtains the distance and vector between the feature points of the first extraction data DA and the feature points of the second extraction data DB. The determination unit 46 determines the degree of coincidence of the fields of view of both according to the distance of the obtained vector. For example, when the distance of the obtained vector is equal to or less than a predetermined threshold value, the determination unit 46 determines that the degree of coincidence of the fields of view of both is high. For example, when the distance of the obtained vector exceeds a predetermined threshold value, the determination unit 46 determines that the degree of coincidence of the fields of view of both is low.

[0062] The determination unit 46 extracts a plurality of feature points from the feature points of the first extraction data DA and the feature points of the second extraction data DB. Further, the determination unit 46 obtains a vector for each of the plurality of feature points. When the directions of the plurality of vectors are aligned, the determination unit 46 determines that the field of view is shifted in a specific direction (the direction of the vector). Further, when the directions of the plurality of vectors are radially different from each other, the determination unit 46 determines that the field of view is shifted due to factors such as different zoom magnifications and different installation positions in the optical axis direction of the imaging unit 10.

[0063] Furthermore, when the directions of the plurality of vectors are not aligned at all, the determination unit 46 determines that the field of view is shifted due to a plurality of factors such as the installation position of the imaging unit 10 and the zoom magnification. In this case, the determination unit 46 determines that the shift width of the field of view is very large.

[0064] Note that the determination of the degree of coincidence between the field of view of the imaging data by the determination unit 46 and the field of view of the reference image is not limited to such a mode. For example, deep learning may be used to learn the degree of coincidence between the field of view of the imaging data and the field of view of the reference image, and the learning result may be used for the determination. By doing so, the determination by the determination unit 46 can be performed by AI.

[0065] FIG. 8 shows a processing flow showing an inspection method for an object according to the second embodiment. The processing of the inspection method for the object is different from the processing flow showing the inspection method for the object in the first embodiment shown in FIG. 4 in that a determination step (step S205) is provided between the extraction image generation step (step S204) and the display step (step S206). Since the other points are the same as the processing flow showing the inspection method for the object in the first embodiment shown in FIG. 4, the description thereof is omitted.

[0066] In the determination step of step S205, the determination unit 46 extracts a plurality of feature points of the first extraction data DA and the feature points of the second extraction data DB. The determination unit 46 obtains the distance and the vector between the feature points of the first extraction data DA and the feature points of the second extraction data DB. The determination unit 46 determines the degree of coincidence between the field of view of the imaging data and the field of view of the reference image using at least one of the distance and the vector information.

[0067] As described above, according to the inspection apparatus 200 of the present embodiment, the determination unit 46 determines the degree of coincidence between the field of view of the imaging data and the field of view of the reference image, so that the determination can be performed quickly.

[0068] Note that the imaging data varies in the way the object and the background are reflected depending on the environment such as the time of imaging and the weather at the time of imaging. For this reason, it is preferable that the reference image is an image captured in an environment close to the environment in which the imaging data is captured.

[0069] Here, the environment is not particularly limited, and examples thereof include the time of imaging (time zone such as day or night), the weather at the time of imaging, the illuminance at the time of imaging, and the setting values of the imaging device (ISO sensitivity, aperture value, shutter speed, white balance, etc.).

[0070] The reference images are preferably captured in different environments and are associated with environment information that is information indicating the environment and stored in the image DB 20. Further, the imaging data is also preferably associated with the environment information and stored in the image DB 20.

[0071] In the image acquisition step of step S201, the image acquisition unit 41 may select and acquire a reference image captured in an environment similar to the environment in which the imaging data was captured. Specifically, the image acquisition unit 41 selects and acquires a reference image that is close to the time when the imaging data was captured.

[0072] The acquisition of the reference image by the image acquisition unit 41 is not limited to such a mode. For example, all the reference images stored in the image DB 20 may be read out, and a reference image captured in an environment similar to the environment in which the imaging data was captured may be selected and acquired.

[0073] By using a reference image captured in an environment similar to the environment in which the imaging data was captured in this way, the pixel values of the background can be made closer to the imaging data, and the occurrence of misjudgment can be suppressed.

[0074] Also, in the extracted image generation step of step S204, the extracted image generation unit 44 may, for example, exclude objects, people, etc. that are not preferable as background targets from the imaging data and the reference image, and generate a first extracted image and a second extracted image. By generating the first extracted image and the second extracted image in this way, the accuracy of the determination by the determination unit 46 can be improved.

[0075] Furthermore, in the extracted image generation step of step S204, when a mark or the like suitable for identifying the feature points of the background is reflected in the imaging data or the reference image, the extracted image generation unit 44 may generate a first extracted image and a second extracted image including the range in which the object is reflected.

[0076] By generating the first extracted image and the second extracted image in this way, the determination unit 46 can perform processing easily and quickly by extracting feature points limited to the range in which the object is reflected, for example, and the accuracy of the determination can be improved.

[0077] Furthermore, in the above-described embodiments, an example in which the probability map data MD is generated based on the imaging data D1 in which the object OB1 is imaged, the imaging data D2 in which the object OB2 is imaged, and the imaging data D3 in which the object OB3 is imaged has been described.

[0078] The probability map data MD may reflect the result of the extracted image generation step in step S204, that is, the extraction result when the first extracted data DA or the second extracted data DB is generated.

[0079] For example, when the first extracted data DA is generated and the background region BR does not contain the objects OB1 to OB3, the result is added at a rate corresponding to the statistic. On the other hand, when the first extracted data DA is generated and the background region BR contains the objects OB1 to OB3, the result is subtracted at a rate corresponding to the statistic. The probability map data MD can be obtained for each pixel by using the data calculated in this way as an average value. By updating the probability map data MD at any time in this way, the extraction accuracy of the background region BR and the object region OR can be further improved.

[0080] In addition, for example, when the object reflected in the imaging data has a shape with a low inspection frequency, it is inappropriate as the data used for updating the probability map data MD. In such a case, it is preferable not to use the imaging data for updating the probability map data MD. By not using such a case with a low inspection frequency for updating the probability map data MD, it is possible to improve the extraction accuracy of the background region BR and the object region OR.

[0081] Also, as for the imaging data determined to have a low degree of coincidence in the determination result of the degree of coincidence of the visual field by the determination unit 46, it is preferable not to use it for updating the probability map data MD. The data with a low degree of coincidence in the visual field has a low consistency with the probability map data MD. Therefore, if the data with a low degree of coincidence in the visual field is used for updating the probability map data MD, the accuracy of the probability map data MD may decrease.

Description of Symbols

[0082] 100, 200 Inspection device for objects 10 Imaging unit 20 Image database 30 Display unit 40 Control unit 41 Image acquisition unit 42 Probability map data acquisition unit 43 Region setting unit 44 Extracted image generation unit 45 Inspection unit 46 Judgment unit OB1 to OB3 Objects MD Probability map data DA First extraction data DB Second extraction data BR Background region

Claims

1. An inspection apparatus for inspecting the appearance of an object using imaging data obtained by imaging the object, an image acquisition unit that acquires a reference image including a background arranged around the object, and imaging data that includes the object and the background and is imaged from a position along the optical axis of an imaging unit where the reference image was imaged; a probability map data acquisition unit that acquires probability map data indicating the probability of existence of the object for each pixel in an image region corresponding to the imaging data; a region setting unit that sets a background region including the background and an object region including the object among the image regions in the imaging data and the reference image based on the probability map data; an extraction image generation unit that generates first extraction data by extracting the imaging data by classifying the background region and the object region using the background region and the object region set by the region setting unit, and generates second extraction data by extracting the reference image by classifying the background region and the object region using the background region and the object region; a display unit that displays the first extraction data and the second extraction data; an inspection unit that inspects the appearance of the object using the first extraction data including the object region in the imaging data; An inspection apparatus for an object, comprising:

2. The inspection apparatus for an object according to claim 1, further comprising a determination unit that determines the degree of coincidence between the field of view of the imaging data and the field of view of the reference image with reference to the first extraction data and the second extraction data generated by the extraction image generation unit.

3. The inspection apparatus for an object according to claim 1 or 2, wherein the probability map data is updated excluding the determined imaging data for which the degree of coincidence between the field of view of the imaging data and the field of view of the reference image is determined to be low.

4. The reference images are imaged in different environments, The inspection apparatus for an object according to claim 1 or 2, wherein the image acquisition unit selects and acquires the reference image captured in an environment similar to the environment in which the captured data was captured.

5. The reference images are captured in different environments, The inspection apparatus for an object according to claim 3, wherein the image acquisition unit selects and acquires the reference image captured in an environment similar to the environment in which the captured data was captured.

6. The determination unit associates a feature point set in one of the first extraction data and the second extraction data with a feature point in the other extraction data corresponding to the feature point, The inspection apparatus for an object according to claim 2, wherein the feature point is set based on a change amount with respect to the periphery of each pixel of the one extraction data.

7. The inspection apparatus for an object according to claim 2, wherein the determination unit determines using a learning result by deep learning.

8. The inspection apparatus for an object according to claim 1 or 2, wherein the first extraction data and the second extraction data are displayed on the display unit in a state of overlapping each other.

9. An inspection method for inspecting the appearance of an object using captured data of the object, An image acquisition step of acquiring a reference image including a background arranged around the object and captured data captured from a position along the optical axis of the imaging unit including the object and the background and in which the reference image was captured, A probability map data acquisition step of acquiring probability map data indicating the probability of existence of the object for each pixel in an image region corresponding to the captured data, A region setting step of setting a background region including the background and an object region including the object in the image regions in the captured data and the reference image based on the probability map data, Using the background area and the object area set in the area setting step, the imaging data is divided into the background area and the object area to extract the first extraction data, and using the background area and the object area, the reference image is extracted by dividing the background area and the object area to generate the second extraction data; an extraction image generation step; A display step of displaying the first extraction data and the second extraction data; An inspection step of inspecting the appearance of the object using the first extraction data including the object area in the imaging data; An object inspection method including:

10. The object inspection method according to claim 9, further comprising a determination step of determining the degree of coincidence between the field of view of the imaging data and the field of view of the reference image with reference to the first extraction data and the second extraction data generated by the extraction image generation step.

11. The object inspection method according to claim 9 or 10, wherein the probability map data is updated excluding the determined imaging data for which the degree of coincidence between the field of view of the imaging data and the field of view of the reference image is determined to be low.

12. The reference images are captured in different environments, The object inspection method according to claim 9 or 10, wherein the image acquisition step selects and acquires the reference image captured in an environment similar to the environment in which the imaging data is captured.

13. The reference images are captured in different environments, The object inspection method according to claim 11, wherein the image acquisition step selects and acquires the reference image captured in an environment similar to the environment in which the imaging data is captured.

14. In the determination step, the feature points set in one of the first extraction data and the second extraction data are associated with the feature points in the other extraction data corresponding to the feature points, The inspection method of an object according to claim 10, wherein the feature points are set based on the amount of change with respect to the periphery of each pixel of the extraction data of one side.

15. The inspection method of an object according to claim 10, wherein in the determination step, determination is made using the learning result by deep learning.

16. The inspection method of an object according to claim 9 or 10, wherein in the display step, the first extraction data and the second extraction data are displayed in a state of overlapping each other.

Images