Information processing device, information processing method and program

Abstract

To provide an information processing device, etc., that can appropriately cope with an inspection based on the entire image of an imperfection.SOLUTION: An information processing device includes: a learning unit that performs learning based on a teacher image including the entire image of an imperfection; and a divided image generating unit that divides an image obtained by imaging an object to be inspected so as to generate a plurality of divided images. The teacher image is generated based on the divided image, and the divided image has a larger overlap width than the maximum value of the representative size of the imperfection. The information processing device may further include a size storing unit that stores the maximum size of the representative size of the imperfection, and the divided image generating unit may define the overlap width based on the maximum value of the representative size stored in the size storing unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Patent Document 1 discloses an inspection method in which a captured image is divided to generate divided image data, and an inspection is performed based on the divided image data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7129669 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technique disclosed in Patent Document 1 does not guarantee that the entire image of the defect is included in the divided images, and is not capable of handling inspection based on the entire image of the defect.

[0005] Therefore, in one aspect, an object of the present disclosure is to provide an information processing device and the like that can appropriately handle inspection based on an overall image of a defect. [Means for solving the problem]

[0006] According to one aspect of the present disclosure, A learning unit that performs learning based on a teacher image including an overall image of a defect; a divided image generating unit that divides an image obtained by photographing an object to be inspected to generate a plurality of divided images; Equipped with The teacher image is generated based on the divided image, An information processing apparatus is provided in which the divided images have an overlapping width larger than a maximum value of the representative size of the defect. Effect of the Invention

[0007] In one aspect, the present disclosure can adequately accommodate inspection based on a global view of a defect. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 illustrates an example of a configuration of an information processing apparatus according to an embodiment of the present invention. [Diagram 2] 13 is a diagram illustrating an example of a teacher image including an image of Anisakis. [Diagram 3] FIG. 13 is a diagram illustrating a method for diversifying teacher images. [Figure 3A] FIG. 13 is a diagram illustrating a method for diversifying teacher images. [Figure 3B] FIG. 13 is a diagram illustrating a method for diversifying teacher images. [Figure 4] 11A and 11B are diagrams illustrating a method for generating divided images by a divided image generating unit. [Figure 4A] FIG. 1 illustrates typical sizes of Anisakis. [Diagram 5] FIG. 13 is a diagram showing the positional relationship between divided images and Anisakis. [Figure 5A] FIG. 13 is a diagram showing the positional relationship between divided images and Anisakis. [Figure 5B] FIG. 13 is a diagram showing the positional relationship between divided images and Anisakis. [Figure 6] FIG. 2 is a diagram illustrating an example of an area to be inspected within an object to be inspected; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The present disclosure is an example of application to inspection of foreign bodies in translucent foods, specifically to inspect raw fish meat for Anisakis (a parasite) as the object of inspection. However, the object of the present invention is not limited thereto. It may also be applied to other situations, such as inspection of salad leaves, or inspection of various types of meat to find unwanted foreign bodies such as bones or plastics. Furthermore, the object of inspection is not limited to food.

[0010] It is difficult to detect small defects such as Anisakis based on the original image (full-size image) of the photographed object to be inspected, and it is effective to divide the original image into a plurality of divided images and generate a teacher image based on the divided images. The present disclosure relates to a method for generating divided images for detecting small defects with high accuracy. In particular, the present disclosure discloses a method for generating divided images that include the entire image of a small defect such as Anisakis and that does not cut off the defect at the edge of the image.

[0011] Hereinafter, embodiments will be described with reference to the accompanying drawings.

[0012] FIG. 1 is a diagram illustrating an example of the configuration of an information processing apparatus according to the present embodiment.

[0013] 1, information processing device 10 includes a learning unit 11 that performs learning based on a teacher image including the entire defect, a divided image generating unit 12 that generates a plurality of divided images by dividing an image obtained by photographing an inspection object, an inference unit 13 that performs inference on the inspection image using an algorithm obtained by learning in the learning unit 11, and a size storage unit 14 that stores the maximum value of the representative size of defects. Information processing device 10 can be configured using a single or multiple computers on which a predetermined program is implemented.

[0014] Next, a description will be given of the processing related to learning in the learning unit 11. Hereinafter, a case where Anisakis in food is detected as a defect in the inspection object 1 will be exemplified.

[0015] FIG. 2 is a diagram illustrating a teacher image including an image of Anisakis.

[0016] In this embodiment, learning is performed by the learning unit 11 based on a teacher image generated from an image 20 of a region including an overall image of Anisakis 2 among images of the inspection object 1, which is a food. By excluding images that do not include an overall image of Anisakis 2 from the teacher image, it is sufficient to store only images including an overall image of Anisakis 2 as teacher images, so that the amount of data related to the teacher image can be reduced. In addition, by narrowing down the teacher images to images effective for learning, it is possible to improve the efficiency of learning in the learning unit 11 and the detection accuracy of Anisakis 2. The image 20 may be, for example, a part of an original image obtained by photographing the entire inspection object 1, or an image obtained by photographing a part of the inspection object 1 in an enlarged manner. The size of the image 20 can be determined according to the size of Anisakis 2. The teacher image and the inspection image may be multicolor (color) images or monochrome images, but it is considered that a multicolor image can provide more useful information and improve the inspection accuracy.

[0017] In this embodiment, a diversified teacher image is generated based on one image 20. The generated teacher image is used for learning in the learning unit 11.

[0018] The image 20 is selected based on the photographing conditions (e.g., lighting method, lighting intensity, exposure (camera sensitivity)) that allow easy extraction of the contour of the Anisakis 2. For example, dark images in which the contour of the Anisakis 2 is not clear or images that are too bright are eliminated.

[0019] Further, a size suitable for detecting Anisakis 2 is selected as the size of the image 20. For example, an image that is too large relative to the size of Anisakis 2 may not provide a resolution suitable for detecting Anisakis 2.

[0020] 3 to 3B are diagrams illustrating a method for diversifying teacher images.

[0021] 3, image 20 shows Anisakis 2 located approximately at the center of the region, while image 20A and image 20B show Anisakis 2 located at a position shifted from the center of the region. Image 20A and image 20B can be generated (synthesized) based on image 20.

[0022] 3A shows an image 20 in which the Anisakis 2 is in a predetermined direction, and an image 20C and an image 20D in which the Anisakis 2 is rotated. The image 20C and the image 20D can be generated (synthesized) based on the image 20.

[0023] In the example of FIG. 3B, images 20E and 20F are shown in which the relative sizes of Anisakis 2 are different from those of image 20, which includes Anisakis 2 of a predetermined size. That is, the size of Anisakis 2 in image 20E is larger than that of image 20, and the size of Anisakis 2 in image 20F is smaller than that of image 20. Images 20E and 20F can be generated (synthesized) based on image 20. Note that, for convenience of drawing, images 20E and 20F are drawn as being different in size from image 20 in FIG. 3B, but the sizes of images 20E and 20F are the same as that of image 20, and the sizes of Anisakis 2 are different.

[0024] In this embodiment, images 20A to 20F are generated by changing the position, angle, and size of Anisakis 2 for one image 20, and these images are used as teacher images, thereby diversifying the teacher images. The teacher images generated based on the image 20 are not limited to images 20A to 20F, and a large number of teacher images can be generated by gradually changing and combining the parameter values ​​of the position, angle, and size of Anisakis 2 contained in the image 20. Each of these teacher images includes a whole image of Anisakis 2. In this embodiment, the inference unit 13 detects Anisakis 2 based on an examination image including a whole image of Anisakis 2. Therefore, the common property of including a whole image of Anisakis 2 can be given to both the teacher image and the examination image, and the detection accuracy of Anisakis 2 can be significantly improved.

[0025] A large number of images including Anisakis 2, such as those corresponding to image 20, are prepared, and similar processing (processing to change the position, angle, and size of Anisakis 2) is performed on these images, thereby making it possible to prepare a large number of diverse teacher images. These teacher images include teacher images in which Anisakis 2 of various sizes, shapes, and orientations are present in various regions (for example, the center and the end) of the image. Therefore, the accuracy of inference in the inference unit 13, that is, the detection accuracy of Anisakis 2, can be improved. In other words, Anisakis 2 can be detected with high accuracy during inference in the inference unit 13 described later, without performing operations on the test image, such as operations to reduce / enlarge, shift the position, or rotate the test image. This makes it possible to improve the detection speed of Anisakis 2 during inference.

[0026] Regarding the diversity of teacher images, multiple images 20 with different shooting conditions (e.g., lighting method, lighting intensity, exposure (camera sensitivity)) may be prepared as images corresponding to the same region. In this case, images based on shooting conditions within a range that allows the contour of Anisakis 2 to be easily extracted may be selected as teacher images. For example, dark images in which the contour of Anisakis 2 is not clear or images that are too bright may be eliminated, and the remaining images may be used as teacher images. By preparing teacher images with appropriately different shooting conditions, the detection accuracy of Anisakis 2 may be improved. For example, it may be possible to suppress missed detection of Anisakis 2.

[0027] In this embodiment, among the divided images described later, one including an overall image of Anisakis is used as the image 20. The divided images are generated by dividing an original image 100 of the photographed inspection object 1. This embodiment is characterized by a method of dividing the original image 100 so as to obtain a divided image including an overall image of the defect Anisakis. This makes it possible to improve the learning accuracy by dividing the original image 100, while preventing the generation of learning data in which defects are cut into pieces due to division. In other words, defects in the teacher image to be learned are prevented from being cut into pieces. Specifically, the divided images are generated so as to have an overlapping width larger than the maximum value of the representative size of the defect.

[0028] Next, the divided images generated by the divided image generating unit 12 will be described.

[0029] FIG. 4 is a diagram showing a method of generating divided images by the divided image generating unit.

[0030] In Fig. 4, the divided image generating unit 12 divides an original image 100 of the photographed inspection object 1 to generate a plurality of divided images. In the divided images IM(x,y), x and y are integers indicating the order of the divided images IM(x,y) in the x and y directions in Fig. 4, respectively. The size and shape of the divided images IM(x,y) are the same regardless of the values ​​of x and y.

[0031] The divided image IM(0,0) shown in FIG. 4 is located at the center of the original image 100. In addition, in FIG. 4, the divided image IM(1,0) is an image adjacent to the divided image IM(0,0) in the x direction (rightward in FIG. 4). The divided image IM(-1,0) is an image adjacent to the divided image IM(0,0) in the -x direction (leftward in FIG. 4). The divided image IM(0,1) is an image adjacent to the divided image IM(0,0) in the y direction (upward in FIG. 4). The divided image IM(0,-1) is an image adjacent to the divided image IM(0,0) in the -y direction (downward in FIG. 4). Similarly, the divided image IM(x,y) relating to the required combination of integers (x,y) can be generated, and the inspection target range of the inspection target object 1 can be covered by the divided image IM(x,y).

[0032] The size of the divided image IM(x,y) can be determined according to the size of the Anisakis 2. For example, it may be determined according to the average value Av of the representative size of the Anisakis 2 described later. Naturally, the size of the divided image IM(x,y) is set to a size that allows the entire image of the Anisakis 2 to be included in the image. For example, the lengths L1 and L2 of one side of the divided image IM(x,y) shown in FIG. 4 may be set to a value equal to or greater than the value (Av+ΔL) obtained by adding the average value Av of the representative size of the Anisakis 2 to the overlap width ΔL described later. A plurality of preset values ​​may be prepared as the size of the divided image IM(x,y), and an appropriate size, for example, a size in which the lengths L1 and L2 are equal to or greater than the value (Av+ΔL), may be selected from these. Note that, although a horizontally long divided image IM(x,y) is illustrated in FIG. 4, the shape of the divided image is arbitrary.

[0033] 4, adjacent divided images IM(x,y) overlap each other via an overlap width ΔL in each of the x and y directions. In this embodiment, the overlap width ΔL is set to a value larger than the maximum value representing the size of Anisakis 2 (representative size).

[0034] FIG. 4A is a diagram illustrating a typical size of Anisakis, and FIGS. 5 and 5B are diagrams showing the positional relationship between the divided images and Anisakis.

[0035] As shown in FIG. 4A, the size of the Anisakis 2 is represented, for example, by the maximum width L0 of the contour of the Anisakis 2 in the image. That is, in the example shown in FIG. 4A, the maximum width L0 is set as the representative size. In this embodiment, the overlap width ΔL is set to a value larger than the maximum value of the maximum width L0. For example, the value of the overlap width ΔL is set to a value 1.1 times or more the maximum value of the maximum width L0. In addition, when the defect is surrounded by a figure, the representative size of the figure may be obtained and this may be set as the representative size of the defect. In this case, the representative size of the figure may be, for example, the diameter if the figure is a circle. In addition, if the figure is a rectangle, the diagonal length may be used, and in the case of a polygon (a quadrangle other than a rectangle, a pentagon, a hexagon, etc.), the longest diagonal length may be used.

[0036] By setting the value of the overlap width ΔL to be larger than the maximum representative size in this manner, it is possible to reliably secure a divided image that includes the entire image of the Anisakis 2.

[0037] 5, only a portion of the image of Anisakis 2 is included in the divided image IM(k,m) at the right end of the divided image IM(k,m). However, the divided image IM(k+1,m) adjacent to the divided image IM(k,m) includes the entire image of Anisakis 2.

[0038] In the example of Fig. 5A, only a portion of the image of Anisakis 2 at the bottom end of the divided image IM(k,m) is included in the divided image IM(k,m). However, the divided image IM(k,m-1) adjacent to the divided image IM(k,m) includes the entire image of Anisakis 2.

[0039] In the example of Fig. 5B, only a portion of the image of Anisakis 2 at the lower right end of divided image IM(k,m) is included in divided image IM(k,m). However, divided image IM(k+1,m-1) adjacent to divided image IM(k,m) includes the entire image of Anisakis 2.

[0040] The inference unit 13 extracts Anisakis 2 and detects the position of Anisakis 2 based on an algorithm constructed by learning from an image 20 including an entire image of Anisakis 2. This enables highly accurate detection of Anisakis 2. As described above, the detection accuracy of Anisakis 2 in this embodiment is related to the fact that the teacher image is given the property of including an entire image of Anisakis 2.

[0041] Next, a method for setting the value of the overlap width ΔL will be described. The divided image generating unit 12 acquires the representative size of Anisakis 2 (for example, the maximum width L0 shown in FIG. 4A) based on the teacher image and stores it in the size storage unit 14. Here, when a new teacher image is generated, the divided image generating unit 12 sequentially acquires the representative size of Anisakis 2 contained in this teacher image and adds it to and stores it in the size storage unit 14. The divided image generating unit 12 may also sequentially acquire the representative size of Anisakis 2 detected by the inference unit 13 and store it in the size storage unit 14. As described above, the maximum value of the representative size of Anisakis 2 stored in the size storage unit 14 is reflected in the value of the overlap width ΔL. In this case, the representative sizes of all defects may be stored, and the maximum value may be extracted and updated from among them.

[0042] Instead of storing all representative sizes, it is possible to store and update only the maximum representative size stored in the size storage unit 14. In this case, only when a newly detected or calculated representative size is larger than the maximum value stored in the size storage unit 14, the maximum value is updated.

[0043] When the maximum representative size is updated, the value of the overlap width ΔL is updated according to the updated maximum representative size. Note that the actual overlap width between adjacent divided images IM(x,y) is allowed to be larger than the overlap width ΔL. This makes it possible to prevent the inspection range (the range in which the divided images IM(x,y) are arranged) from unnecessarily expanding outside the inspection target 1.

[0044] If the update width of the maximum representative size is larger than a predetermined width, the notification unit 15 may notify the user of the fact. The notification may be in the form of an alarm. For example, if the new maximum value is a predetermined value (for example, a value 1.5 times or more of the stored maximum value), the new maximum value may be based on an error. Specifically, in such a case, the maximum value may not be derived from Anisakis 2, but may be due to another object that was mistakenly placed due to an error during annotation. Also, it may be due to other types of defects rather than Anisakis 2. Alternatively, for example, the maximum value may be due to an error in the process of generating the teacher image when acquiring the image or generating the divided image. Therefore, in such a case, the user can know the possibility of an abnormality by receiving a notification from the notification unit 15. Also, if necessary, the user can cancel the update of the maximum representative size and the update of the value of the overlap width ΔL.

[0045] In this embodiment, the size of the teacher image and the size of the inspection image to be inspected by the inference unit 13 can be made the same. By matching the sizes of both images, the detection accuracy of Anisakis 2 can be maximized. However, it is not essential to match the sizes of both images. For example, when the size of the divided image (the value of the overlap width ΔL) is updated according to the progress of inference in the inference unit 13, it goes without saying that the learning that has been done using the teacher image does not need to be invalidated. However, additional learning using a teacher image with a size updated to match the size of the divided image is not excluded.

[0046] In this embodiment, when a single segmented image contains multiple Anisakis 2, a segmented image including the entire image of each Anisakis 2 can be generated. That is, a separate segmented image can be generated for each Anisakis 2, and a teacher image can be generated based on each segmented image.

[0047] FIG. 6 is a diagram illustrating an example of an area to be inspected within an object to be inspected.

[0048] 6, when Anisakis 2 is not present at the outer edge of the inspection object 1, only the area 1A narrowed down to exclude the outer edge may be set as the area to be inspected. In this case, since only the area 1A needs to be covered by the divided image, it is possible to suppress the generation of a divided image that does not include Anisakis 2, and to shorten the inspection time in the inference unit 13.

[0049] As described above, according to this embodiment, the divided image generating unit 12 generates divided images so that the overlapping width is greater than the maximum value of the representative size of the defect (Anisakis). Therefore, a divided image including the entire image of the defect can be reliably generated. The learning unit 11 performs learning on the divided image including the entire image of the defect, so that the defect (Anisakis) can be detected with high accuracy.

[0050] Although each embodiment has been described above in detail, the present invention is not limited to the specific embodiment, and various modifications and changes are possible within the scope of the claims. In addition, it is also possible to combine all or a plurality of the components of the above-described embodiments. [Explanation of symbols]

[0051] 10. Information processing device 11 Learning Department 12. Divided image generation unit 13 Reasoning part 14 Size memory section 15 Notification Department

Claims

1. A learning unit that performs learning based on a teacher image including an overall image of a defect; a divided image generating unit that divides an image obtained by photographing an object to be inspected to generate a plurality of divided images; Equipped with The teacher image is generated based on the divided image, An information processing apparatus, wherein the divided images have an overlapping width larger than the maximum value of the representative size of the defect.

2. a size storage unit that stores a representative size of the defect; The information processing apparatus according to claim 1 , wherein the divided image generating section determines the overlapping width based on a maximum value of the representative sizes stored in the size storage section.

3. The information processing apparatus according to claim 2 , wherein the divided image generating section detects a representative size of the defect based on a contour of the defect.

4. The divided image generating unit obtains a maximum value of the representative size, The information processing apparatus according to claim 2 , wherein the size storage section updates the maximum representative size when the acquired maximum representative size is larger than the maximum representative size that has already been stored.

5. The information processing apparatus according to claim 1 , wherein the learning section performs learning based only on a teacher image including an entire image of the defect.

6. an inference unit that performs inference on an inspection image using an algorithm obtained by learning in the learning unit; The information processing apparatus according to claim 1 , wherein the teacher image and the test image have the same size.

7. The information processing apparatus according to claim 1 , wherein when one of the divided images includes a plurality of the defects, the divided image generating section generates the divided image for each of the defects.

8. The information processing apparatus according to claim 1 , wherein a size of the divided image is defined in accordance with a size of the defect.

9. The information processing apparatus according to claim 1 , wherein the divided image generating unit generates the divided image so as to cover a region narrowed down to a portion of the object to be inspected.

10. A learning step of performing learning based on a teacher image including an overall image of the defect; a divided image generating step of dividing an image obtained by photographing an object to be inspected to generate a plurality of divided images; Equipped with The teacher image is generated based on the divided image, An information processing method, wherein the divided images have an overlap width greater than a maximum value of the representative size of the defect.

11. A learning step of performing learning based on a teacher image including an overall image of the defect; a divided image generating step of dividing an image obtained by photographing an object to be inspected to generate a plurality of divided images; A program for causing a computer to execute the following: The teacher image is generated based on the divided image, The divided images have an overlap width greater than a maximum representative size of the defects.

Images