Detection method and program

Abstract

To provide a detection method capable of easily and accurately detecting an elliptical shaped object included in an object.SOLUTION: A detection method includes the steps of: acquiring an image in which at least a part of an object including an elliptical shaped object is reflected (S11); dividing the acquired image into blocks in blocks of the same size as the size of the block used for teacher data at the time of learning a machine learning model (S12); and detecting an abnormal place of the object on the basis of the number of pixels representing the elliptical shaped object in each block obtained by inputting an image divided into the blocks in the already-learned model (S13).SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to a detection method and a program. [Background technology]

[0002] A technology for detecting objects in an image using image recognition based on machine learning has been known. For example, such a technology is used to detect products from images captured on a production line and to control the quality of the products. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-022484 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned conventional techniques have room for further improvement in terms of simple and accurate object detection using machine learning. In particular, it is known that the detection accuracy of objects with rounded edges, such as elliptical objects, is low.

[0005] Therefore, the present disclosure provides a detection method that can easily and accurately detect an elliptical object included in an object. [Means for solving the problem]

[0006] A detection method according to one aspect of the present disclosure acquires an image showing at least a portion of an object including an oval-shaped object, divides the acquired image into blocks of the same size as the blocks used in the training data when training a machine learning model, and detects abnormal areas of the object based on the number of pixels representing the oval-shaped object in each of the blocks obtained by inputting the divided image into a trained model, which is the trained machine learning model; in detecting the abnormal areas, if the number of pixels output from the trained model is greater than a first threshold and less than a second threshold, it is determined that the part of the object corresponding to the block is not the abnormal area; and if the number of pixels output from the trained model is less than the first threshold or greater than the second threshold, it is determined that the part of the object corresponding to the block is the abnormal area.

[0007] A program according to one aspect of the present disclosure is a program for causing a computer to execute the detection method. [Effects of the Invention]

[0008] According to the present disclosure, a detection method is provided that can easily and accurately detect an abnormal portion of an object. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram for explaining an overview of a detection system according to an embodiment. [Figure 2] FIG. 2 is a block diagram illustrating an example of a functional configuration of the detection system according to the embodiment. [Figure 3] FIG. 3 is a flowchart illustrating an example of the operation of the detection device according to the embodiment. [Figure 4] FIG. 4 is a flowchart showing the detailed flow of step S13 in FIG. [Figure 5] FIG. 5 is a diagram showing an example of an image acquired in step S11 of FIG. [Figure 6]FIG. 6 is a diagram showing an example of processing the image acquired in step S11 of FIG. [Figure 7] FIG. 7 is a diagram for explaining the process of detecting an abnormal portion. [Figure 8] FIG. 8 is a diagram showing an example of a pixel histogram of the detection result. [Figure 9] FIG. 9 is a diagram showing an example of the determination result. [Figure 10] FIG. 10 is a diagram illustrating an example of the filtering process. [Figure 11] FIG. 11 is a diagram showing the results of Comparative Example 1 and Example 1. [Figure 12] FIG. 12 is a diagram showing the results of Comparative Example 2 and Example 2. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, the embodiments will be described in detail with reference to the drawings. Note that the embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component placement and connection forms, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components not recited in independent claims will be described as optional components.

[0011] It should be noted that the drawings are schematic diagrams and are not necessarily strict illustrations. In addition, in the drawings, substantially the same components are denoted by the same reference numerals, and overlapping descriptions may be omitted or simplified.

[0012] (Embodiment) [composition] First, the configuration of a detection system according to an embodiment will be described. Fig. 1 is a diagram for explaining an overview of the detection system according to an embodiment. Fig. 2 is a block diagram showing an example of the functional configuration of the detection system according to an embodiment.

[0013] As shown in FIG. 1 , the detection system 100 detects an abnormal portion of an object 1 that includes an oval-shaped object. An abnormal portion is, for example, a portion where an oval-shaped object has a shape abnormality such as a chip. The detection system 100 is used, for example, for quality inspection of the object 1. The object 1 is an object that includes an oval-shaped object, such as an object having an oval-shaped object on its surface, an object composed of oval-shaped objects, or an object that includes an oval-shaped object as part of its structure. The object may be formed from various materials such as fiber, metal, resin, wood, carbon, gel, or clay as long as it includes an oval-shaped object. The oval-shaped object may be, for example, an oval, a polygon with rounded corners, or a circle when viewed from above. In the following, a cloth will be used as an example of the object 1 that includes an oval-shaped object, but this is merely an example and is not limiting.

[0014] 1 and 2, the detection system 100 includes, for example, a detection device 10 and an imaging device 30. FIG. 1 also illustrates an illumination device 20 and a conveying device 40. The detection system 100 may include the illumination device 20 and the conveying device 40. The detection system 100 may detect the presence or absence of an abnormality in an oval-shaped object included in the target object 1 conveyed by the conveying device 40. The conveying device 40 is, for example, a conveyor.

[0015] Each component of the detection system 100 will be described below.

[0016] [Lighting equipment] The illumination device 20 is, for example, a light-emitting device that illuminates the object 1 within the angle of view of the imaging device 30. Two or three illumination devices 20 may be provided. In this case, the two or more illumination devices 20 irradiate the area to be photographed by the imaging device 30 with light from different directions. The type of light source is not particularly limited, and may be, for example, a light source that irradiates white light based on an LED (Light Emitting Diode). Furthermore, when two or more illumination devices 20 are provided, each illumination device 20 may be provided with a light source that emits light of the same wavelength or the same wavelength range, or may be provided with a light source that emits light of different wavelengths. The type of illumination device 20 may be selected appropriately depending on the type of object 1, the color or surface shape of an elliptical object included in the object 1, etc.

[0017] [Imaging device] The imaging device 30 captures an image including at least a portion of the object 1. The image may be a still image or a moving image. The image may be a monochrome image or a color image. The imaging device 30 captures an image including at least a portion of the object 1, including an elliptical object, by detecting light illuminating the object 1 from the lighting device 20. The imaging device 30 has an imaging element for detecting light, such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

[0018] [Detection device] The detection device 10 is a device that detects an abnormality in an elliptical object included in the target object 1 based on an image captured by an imaging device 30. The detection device 10 is, for example, a stationary information terminal such as a personal computer, but may also be a portable information terminal such as a smartphone or tablet terminal. In the examples of FIGS. 1 and 2, the detection device 10 and the imaging device 30 are separate entities, but the imaging device 30 may also be a camera provided in the detection device 10. The detection device 10 includes, for example, a communication unit 11, a control unit 12, a memory unit 13, a learning unit 14, a display unit, and an operation reception unit.

[0019] [Communications Department] The communication unit 11 is a communication module (communication circuit) that enables the detection device 10 to communicate with the lighting device 20, the imaging device 30, and the transport device 40 via a local communication network. The communication unit 11 is, for example, a wireless communication circuit that performs wireless communication, but may also be a wired communication circuit that performs wired communication. The communication standard for communication performed by the communication unit 11 is not particularly limited.

[0020] [Control Unit] The control unit 12 performs information processing to control the operation of the detection device 10. The control unit 12 is realized, for example, by a microcomputer, but may also be realized by a processor or a dedicated circuit. Specifically, the control unit 12 includes an acquisition unit 12a, an image processing unit 12b, a determination unit 12c, and an output unit 12d. The acquisition unit 12a, the image processing unit 12b, the determination unit 12c, and the output unit 12d are all realized by a processor executing a program for performing the above information processing.

[0021] The acquisition unit 12a acquires an image in which at least a part of the target object 1 including an elliptical object is captured.

[0022] The image processing unit 12b performs image processing on the image (image data) acquired by the acquisition unit 12a to generate learning image data or detection image data. Specifically, the image processing unit 12b generates learning image data or detection image data by dividing the image data acquired by the acquisition unit 12a into blocks of a predetermined size.

[0023] For example, to generate the training image data, an image (referred to as a first image) showing at least a portion of the object 1 is divided into a plurality of blocks (referred to as first blocks) of a predetermined size, and the first image is repeatedly divided into predetermined sizes by shifting the first blocks, for example, vertically, horizontally, or diagonally, within a range that does not exceed the first blocks, thereby generating the plurality of training image data. Note that the teacher data for machine learning is composed of the plurality of training image data and annotations indicating the areas of the plurality of elliptical objects that are associated with the plurality of training image data as correct answer data, respectively.

[0024] Furthermore, for example, in generating the image data for detection, the image processing unit 12b generates an image by dividing the image (image data) acquired by the acquisition unit 12a into blocks of the same size as the block size used for the training data when the machine learning model was trained. At this time, prior to dividing the image into blocks, the image processing unit 12b may exclude from the detection target an area other than at least a part of the object 1 shown in the image (in other words, an area where the object 1 is not shown), and divide the detection target area in the image into blocks of the same size as the size used for the training data when the machine learning model was trained.

[0025] Furthermore, when the judgment unit 12c judges that an abnormal part has been detected in the object 1, the image processing unit 12b may generate a marking image in which the abnormal part in the image is marked as a result of the judgment (hereinafter referred to as the judgment result).

[0026] The determination unit 12c uses a trained machine learning model (hereinafter referred to as the trained model) stored in the storage unit 13 to detect an abnormal portion of the object 1 based on the number of pixels representing an elliptical object in each block obtained by inputting the image (so-called detection image data) divided into blocks by the image processing unit 12b into the trained model. More specifically, in detecting an abnormal portion, if the number of pixels output from the trained model is greater than a first threshold and less than a second threshold, the determination unit 12c determines that the portion of the object 1 corresponding to the block is not an abnormal portion, and if the number of pixels output from the trained model is equal to or less than the first threshold or equal to or greater than the second threshold, the determination unit 12c determines that the portion of the object 1 corresponding to the block is an abnormal portion. The threshold may be set as appropriate depending on the required detection accuracy and the type of object to be detected.

[0027] The output unit 12d outputs the result of the determination made by the determination unit 12c (hereinafter referred to as the determination result). The determination result is output, for example, as at least one of an image, a character, a symbol, and a sound. For example, when an abnormal portion is detected in the target object 1, the output unit 12d outputs a marking image generated by the image processing unit 12b as the detection result.

[0028] [Storage] The storage unit 13 is a storage device that stores a control program executed by the control unit 12, etc. The storage unit 13 may temporarily store training data and detection image data. The storage unit 13 updates the stored trained model to a machine learning model (so-called trained model) generated by the learning unit 14. The storage unit 13 is realized by, for example, a semiconductor memory.

[0029] [Study Department] The learning unit 14 performs machine learning using training data. The learning unit 14 uses machine learning to input an image divided into blocks of a predetermined size, and generates a machine learning model that outputs the number of pixels representing elliptical objects in each of the blocks included in the image. The machine learning model is, for example, a convolutional neural network (CNN). The machine learning model may be any CNN, and is not particularly limited, but may be, for example, a DeepCrack (A Deep Hierarchical Feature Learning Architecture for Crack Segmentation) Network. The trained machine learning model (so-called trained model) includes trained parameters adjusted by machine learning. The generated trained model is stored in the storage unit 13. The learning unit 14 is realized, for example, by a processor executing a program stored in the storage unit 13. The training data used for training the machine learning model includes first data consisting of a plurality of first blocks obtained by dividing a first image showing at least a portion of the object 1 into predetermined sizes, annotations indicating the areas of each of the plurality of elliptical objects included in the first blocks, and second data consisting of a plurality of second blocks obtained by further dividing the first image into predetermined sizes by shifting the first blocks within a range not exceeding the first blocks, and annotations indicating the areas of each of the plurality of elliptical objects included in the second blocks. The first blocks may be shifted vertically, horizontally, or diagonally when dividing the first image, and may be shifted by any distance as long as the shift does not exceed the range of the first blocks. In this way, a plurality of training data can be created from a first image showing at least a portion of the object 1.

[0030] [Display] The display unit 15 is a display device that displays an image under the control of the control unit 12. The display unit 15 is realized by, for example, a liquid crystal panel or an organic EL (Electro Luminescence) panel.

[0031] [Operation reception section] The operation reception unit 16 receives operations from the user. Specifically, the operation reception unit 16 is realized by a mouse, a microphone, a touch panel, or the like.

[0032] The operation receiving unit 16 may include a microphone (not shown) or a speaker (not shown). The microphone acquires sound and outputs a sound signal in response to the acquired sound. The microphone is specifically a condenser microphone, a dynamic microphone, a MEMS microphone, or the like. The speaker outputs sound (mechanical sound), for example, in response to the spoken sound acquired by the microphone. This allows the user to interactively input instructions to execute scene control.

[0033] The operation receiving unit 16 may include a camera (not shown). The camera captures an image of the user operating the detection device 10. Specifically, the camera captures the movements of the user's mouth, eyes, fingers, etc. In this case, the operation receiving unit 16 accepts the user's operation based on the image of the user captured by the camera. The camera is realized by, for example, a CMOS image sensor.

[0034] As described above, the detection device 10 may include a camera. In this case, the camera may capture an image that shows at least a part of the target object 1, which includes an elliptical object.

[0035] [Operation] Next, a description will be given of the operation of the detection system 100 according to this embodiment. Fig. 3 is a flowchart showing an example of the operation of the detection device 10 according to this embodiment.

[0036] First, the imaging device 30 of the detection system 100 captures an image that captures at least a part of the target object 1. At this time, the lighting device 20 may illuminate the angle of view of the imaging device 30. The imaging device 30 transmits the captured image to the detection device 10. Note that, although an example in which the imaging device 30 is an external device to the detection device 10 has been described here, the imaging device 30 may also be provided in the detection device 10.

[0037] Next, as shown in FIG. 3, the detection device 10 acquires an image captured by the imaging device 30, in which at least a portion of the object is captured (S11). FIG. 5 is a diagram showing an example of an image acquired in step S11 of FIG. 3. FIG. 6 is a diagram showing an example of processing the image acquired in step S11 of FIG. 3. For example, as shown in FIG. 5, when the acquired image is a moving image, the detection device 10 may extract multiple frame images (so-called still images) from the moving image. The detection device 10 performs the following processing for each still image.

[0038] For example, as shown in (a) of Figure 6, when the detection device 10 acquires an image (still image), it divides the acquired image into blocks of the same size as the block size used in the training data when training the machine learning model, as shown in (b) of Figure 6 (S12).

[0039] Next, the detection device 10 inputs the image divided into blocks into a trained model and detects abnormalities in the object based on the number of pixels representing elliptical objects in each block (S13). For example, as shown in (c) of Figure 6, the detection device 10 outputs an image in which the pixels representing elliptical objects in each block in the image are colored black as an output result of the trained model.

[0040] FIG. 4 is a flowchart showing the detailed flow of step S13 in FIG.

[0041] The detection device 10 inputs the image divided into blocks into a trained model and outputs the number of pixels representing elliptical objects in each block (S21). More specifically, the detection device 10 derives the number of pixels representing elliptical shapes in each block in the image from the output result of the trained model ((c) of FIG. 6). FIG. 7 is a diagram for explaining the detection process of abnormal areas. FIG. 7 shows blocks determined to be normal and blocks determined to be abnormal, arbitrarily extracted from the output result shown in (c) of FIG. 6.

[0042] Next, the detection device 10 starts loop processing for each block (S22). The detection device 10 determines whether the number of pixels output from the trained model in step S21 is greater than a first threshold and less than a second threshold (S23). FIG. 8 is a diagram showing an example of a pixel histogram of the detection result. The first threshold and the second threshold are set, for example, based on the histogram shown in FIG. 8. The detection device 10 may count the number of pixels representing elliptical objects in each of the output blocks, generate a histogram of the counted number of pixels in the multiple blocks, and display the generated histogram on the display unit 15. The user may set the above thresholds from the displayed histogram according to the desired accuracy.

[0043] If the detection device 10 determines that the number of pixels is greater than the first threshold and less than the second threshold (Yes in S23), it determines that the portion of the object 1 corresponding to the block is not an abnormal portion (S24). For example, if the first threshold for the number of pixels representing the elliptical object in the block (the number of pixels in FIG. 7) is set to 35,000 and the second threshold is set to 37,000, the divided images (so-called blocks) of (a) to (e) in FIG. 7 have the number of pixels greater than the first threshold and less than the second threshold, and therefore the portion of the object 1 corresponding to the divided image is determined to be a normal portion.

[0044] On the other hand, if the detection device 10 determines that the number of pixels is equal to or less than the first threshold value or equal to or greater than the second threshold value (No in S23), it determines that the part of the object 1 corresponding to that block is an abnormal part (S25). For example, in the segmented image of (f) in Fig. 7, the number of pixels is equal to or less than the first threshold value, so the part of the object 1 corresponding to that segmented image is determined to be an abnormal part.

[0045] When the detection device 10 has performed the above process for all blocks included in the image, it ends the loop process for each block (S26).

[0046] Next, the detection device 10 determines whether or not an abnormal portion has been detected in the object 1 (more specifically, a portion corresponding to the object 1 captured as a subject in the image) (S27). FIG. 9 is a diagram showing an example of the determination result. For example, as shown in FIG. 9(a), the detection device 10 determines whether or not an abnormal portion has been detected in the object 1, and if it determines that an abnormal portion has been detected, it marks the segmented image determined to be the abnormal portion.

[0047] If the detection device 10 determines that an abnormality has been detected in the object 1 (Yes in S27), it generates a marking image (see (b) of Figure 9) in which the abnormality in the image is marked (S28), and outputs the generated marking image as the determination result (S29).

[0048] On the other hand, if the detection device 10 determines that no abnormality has been detected in the object 1 (No in S27), the processing ends. Note that in step S27, if the detection device 10 determines that no abnormality has been detected in the object 1 (No), it may output "No abnormality" by voice or text, or may output a symbol such as a circle, or may turn on a green lamp to notify the user that no abnormality has been detected.

[0049] In step S12, prior to dividing the blocks, the detection device 10 may exclude from the detection target an area other than at least a portion of the object 1 shown in the image (in other words, an area in the image where the object 1 is not shown), and divide the detection target area in the image (hereinafter also referred to as the target area) into blocks of the same size as the size used as training data when training the machine learning model. This process is called a filtering process. The filtering process will be specifically described with reference to FIG. 10.

[0050] Fig. 10 is a diagram showing an example of filtering processing. As shown in Fig. 10(a), the detection device 10 extracts a target region indicating the detection target (in other words, the target object 1) appearing in the image.

[0051] Next, as shown in FIG. 10(b), the detection device 10 paints in areas in the image where the object 1 does not appear (in other words, non-target areas) to exclude the non-target areas from detection targets.

[0052] Next, as shown in (c) of FIG. 10, the detection device 10 divides the target region into blocks of the same size as the blocks used as training data when training the machine learning model.

[0053] Next, the detection device 10 inputs the block-divided image (e.g., (c) of Figure 10) into the trained model, and obtains an output result (e.g., (d) of Figure 10) in which pixels representing elliptical objects in each block of the image are displayed in black.

[0054] [Effects, etc.] Next, the effects of the detection system 100, the detection device 10, the detection method, and the program according to this embodiment will be described.

[0055] As described above, the detection method of this embodiment acquires an image that shows at least a portion of an object that includes an oval-shaped object, divides the acquired image into blocks of the same size as the blocks used in the training data when training the machine learning model, and detects abnormal areas in the object based on the number of pixels representing oval-shaped objects in each block obtained by inputting the divided image into a trained model, which is a trained machine learning model.In detecting abnormal areas, if the number of pixels output from the trained model is greater than a first threshold and less than a second threshold, it is determined that the part of the object corresponding to the block is not an abnormal area, and if the number of pixels output from the trained model is less than the first threshold or greater than the second threshold, it is determined that the part of the object corresponding to the block is an abnormal area.

[0056] A device that executes such a detection method can use a trained model to easily and accurately detect pixels that represent multiple elliptical objects in a divided image (so-called blocks) obtained by dividing an image that shows at least a portion of an object into blocks. Therefore, a device that executes the detection method can easily and accurately detect abnormal areas in an object that includes elliptical objects.

[0057] Furthermore, the detection method according to this embodiment may further generate a marking image in which the abnormal area in the image is marked when an abnormal area is detected in the object, and output the generated marking image as the judgment result.

[0058] A device that executes such a detection method can output abnormalities in an object in a form that is easy for a user to visually recognize.

[0059] In addition, the detection method according to this embodiment may exclude from the detection target all areas other than at least a portion of the object appearing in the image prior to dividing the blocks, and divide the detection target area in the image into blocks of the same size as the size used as training data when training the machine learning model.

[0060] A device that executes this detection method can efficiently detect oval-shaped objects because it detects only the area where the object is reflected, and therefore the device that executes the detection method can more efficiently detect abnormalities in the object.

[0061] In addition, in the detection method according to this embodiment, the machine learning model may be a convolutional neural network.

[0062] A device that implements such a detection method can efficiently perform convolution operations using an image as an input.

[0063] In addition, in the detection method of this embodiment, the training data used to train the machine learning model may include first data consisting of a plurality of first blocks obtained by dividing a first image that shows at least a portion of the object into sizes and annotations indicating the areas of each of a plurality of elliptical objects included in the first blocks, and second data consisting of a plurality of second blocks obtained by shifting the first blocks in the first image to the above sizes without exceeding the first blocks and annotations indicating the areas of each of a plurality of elliptical objects included in the second blocks.

[0064] A device that executes such a detection method can generate a large amount of training data from a small number of images, thereby improving the learning effect of the machine learning model and the accuracy of detecting elliptical objects by the machine learning model. As a result, the device that executes the detection method can more easily and accurately detect elliptical objects included in the target object.

[0065] (Other embodiments) Although the detection method and program according to the present disclosure have been described based on the above-mentioned embodiments, the present disclosure is not limited to these embodiments. As long as the modifications do not deviate from the spirit of the present disclosure, various modifications conceivable by those skilled in the art may also be included in the scope of the present disclosure.

[0066] Furthermore, each component included in the detection system, detection device, detection method, and program according to the above-described embodiments is typically realized as an LSI, which is an integrated circuit. These components may be individually integrated into single chips, or some or all of them may be integrated into a single chip.

[0067] Furthermore, the integration is not limited to LSI, but may be realized by dedicated circuits or general-purpose processors. FPGAs (Field Programmable Gate Arrays), which can be programmed after LSI fabrication, or reconfigurable processors, which allow the connections and settings of circuit cells within LSIs to be reconfigured, may also be used.

[0068] In each of the above embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may also be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a storage medium such as a hard disk or semiconductor memory.

[0069] Furthermore, all of the numbers used above are examples for specifically explaining the present disclosure, and the embodiments of the present disclosure are not limited to the numbers shown as examples.

[0070] The division of functional blocks in the block diagram is an example, and multiple functional blocks may be realized as a single functional block, one functional block may be divided into multiple blocks, or some functions may be moved to another functional block.Furthermore, the functions of multiple functional blocks having similar functions may be processed in parallel or in time-sharing by a single piece of hardware or software.

[0071] The order in which the steps in the flowchart are executed is merely an example for specifically explaining the present disclosure, and an order other than the above may be used. Also, some of the steps may be executed simultaneously (in parallel) with other steps.

[0072] In addition, this disclosure also includes forms obtained by making various modifications to the above-mentioned embodiments that a person skilled in the art would think of, and forms realized by arbitrarily combining the components and functions of each embodiment within the scope of this disclosure. [Example]

[0073] The detection device and detection method according to the present disclosure will be specifically described below in examples, but the present disclosure is not limited to the following examples.

[0074] Below, we used the DeepCrack Network as a machine learning model to detect abnormalities in the weave of fabric products. Images of fabric products without abnormalities were used.

[0075] A fabric product is composed of multiple warp threads and multiple weft threads. The warp threads are woven above and below the weft threads, resulting in multiple weave patterns. Each of these weave patterns appears as a grain of rice (i.e., an oval shape). These weave patterns appear on the surface of the fabric product. However, if the weave is disrupted, the weave patterns will not appear as grains of rice in a planar view. For example, (1) if a warp thread passes behind a weft thread instead of the front, the warp thread in that area will not appear on the surface, resulting in a state where there are no weave patterns. Also, (2) if a warp thread passes behind a weft thread instead of the front, the weave patterns will appear on the surface in areas where there are no weave patterns, resulting in larger weave patterns. Below, the detection accuracy of elliptical objects in Comparative Example 1 and Example 1 was verified using a fabric product containing a weave pattern as the elliptical object. In Comparative Example 2 and Example 2, the detection accuracy of elliptical objects for abnormal areas in the fabric product was verified.

[0076] (Comparative Example 1) In Comparative Example 1, a machine learning model trained using training data generated by a conventional method was used. The training data in Comparative Example 1 includes data (hereinafter referred to as first data) consisting of a plurality of blocks (hereinafter referred to as first blocks) obtained by dividing an image (hereinafter referred to as first image) showing a fabric product into blocks of a predetermined size, and annotations indicating each of a plurality of weave areas included in the first blocks. FIG. 11 shows the results detected by the detection method of Comparative Example 1. FIG. 11 is a diagram showing the results of Comparative Example 1 and Example 1. In FIG. 11, pixels representing elliptical objects (here, weave) are shown in black. As shown in FIG. 11, in Comparative Example 1, the pixels showing black were sparse, and the weave detection accuracy was not very good.

[0077] Example 1 In Example 1, a machine learning model trained using training data generated by the method of the present disclosure was used. In Example 1, the same image as in Comparative Example 1 was used to generate the training data, but the training data generation method was different. Therefore, more training image data could be generated from a single image than in Comparative Example 1. The training data in Example 1 included: first data consisting of a plurality of blocks (called first blocks) obtained by dividing an image (called a first image) showing a fabric product into blocks of the same size as in Comparative Example 1; annotations indicating each of the plurality of weave areas included in the first blocks; and second data consisting of a plurality of blocks (called second blocks) obtained by further dividing the first blocks into the same size as in Comparative Example 1 by shifting the first blocks vertically, horizontally, or diagonally within the first image without exceeding the first blocks; and annotations indicating each of the plurality of mesh areas included in the second blocks. The results detected by the detection method in Example 1 are shown in FIG. 11.

[0078] As shown in FIG. 11, black pixels were observed densely, and it was confirmed that the detection accuracy of the weave was improved compared to Comparative Example 1.

[0079] (Comparative Example 2) Comparative Example 2 was performed in the same manner as Comparative Example 1, except that an image containing an abnormal portion of a fabric product was input into a machine learning model and anomaly detection was performed. The results detected using the detection method of Comparative Example 2 are shown in FIG. 12. FIG. 12 is a diagram showing the results of Comparative Example 2 and Example 2. As shown in FIG. 12, the left side of the input image contains an abnormal portion where the yarn is not woven in and protrudes from the surface of the fabric product. In FIG. 12, as in FIG. 11, pixels representing oval-shaped objects (here, weave patterns) are shown in black.

[0080] As shown in Figure 12, in Comparative Example 2, pixels are displayed in black as if multiple weave patterns exist in the abnormal area where the thread protrudes. In other words, in Comparative Example 2, the presence of weave patterns is erroneously detected in an area where the thread protrudes and no weave patterns exist. As a result, the change in the total number of pixels is small, and it is determined that there are no abnormalities in the area shown in the input image.

[0081] Example 2 In Example 2, the same procedures as in Example 1 were carried out except that an image containing an abnormal portion of a fabric product was input into a machine learning model to perform anomaly detection, as in Comparative Example 2. The results detected by the detection method of Example 2 are shown in FIG.

[0082] 12, in Example 2, at an abnormal location where a thread protrudes, pixels are displayed in black along the shape of the protruding thread. As a result, the change in the total number of pixels becomes large, and it is determined that an abnormality exists in the range shown in the input image.

[0083] (summary) From the results of Comparative Examples 1 and 2 and Examples 1 and 2, it was found that when training data was generated using the method of the present disclosure, each of the learning image data (so-called divided images) generated from a single image generates a second block, a third block, a fourth block, etc. by shifting the first block vertically, horizontally, or diagonally within a range not exceeding the first block, and each of the second, third, and fourth blocks has an area overlapping with the first block. By using divided images generated by shifting and re-dividing a single image (first image) in this way for machine learning, it was confirmed that the learning effect can be easily improved, and as a result, detection accuracy is improved. [Industrial Applicability]

[0084] The present disclosure can easily and accurately detect an oval-shaped object, and therefore can easily and accurately detect an abnormality in an object that includes an oval-shaped object. Therefore, the present disclosure can be used for quality inspection of products in various fields, such as food, industrial products, or daily necessities. [Explanation of symbols]

[0085] 1. Object 10. Detection Device 11 Communications Department 12 Control Unit 12a Acquisition part 12b Image processing section 12c Judgment part 12d Output section 13 Storage section 14 Learning Department 15 Display section 16 Operation reception section 20 Lighting equipment 30 Imaging device 40 Conveyor 100 Detection System

Claims

1. A detection method executed by a detection system including a computer, comprising: the detection system comprising: Acquire an image showing at least a part of an object including an elliptical object; Dividing the acquired image into blocks of the same size as the block size used for the training data when learning the machine learning model; Detecting an abnormal portion of the object based on the number of pixels representing the elliptical object in each of the blocks obtained by inputting the image divided into blocks into a trained model, which is the trained machine learning model; The detection system, in detecting the abnormality portion, If the number of pixels output from the trained model is greater than a first threshold and less than a second threshold, it is determined that the portion of the object corresponding to the block is not the abnormal portion; If the number of pixels output from the trained model is equal to or less than the first threshold value, or equal to or more than the second threshold value, it is determined that the part of the object corresponding to the block is the abnormal part. Detection method.

2. The detection system further generates a marking image in which the abnormality is marked in the image when the abnormality is detected in the object, The generated marking image is output as a determination result. The detection method according to claim 1 .

3. The detection system, Prior to dividing the image into blocks, an area other than at least a part of the object shown in the image is excluded from detection targets; Dividing the detection target area in the image into blocks of the same size as the size used as the training data when training the machine learning model. The detection method according to claim 1 or 2.

4. The machine learning model is a convolutional neural network. The detection method according to any one of claims 1 to 3.

5. The training data used to train the machine learning model is First data including a plurality of first blocks obtained by dividing a first image showing at least a part of an object into the size, and annotations indicating the areas of each of a plurality of elliptical objects included in the first blocks; second data including a plurality of second blocks obtained by redividing the first block in the first image to the size by shifting the first block within a range not exceeding the first block, and annotations indicating the areas of each of a plurality of elliptical objects included in the second blocks; Contains The detection method according to any one of claims 1 to 4.

6. A method for causing a computer to execute the detection method according to any one of claims 1 to 5. program.

Images