Program and determination device
By aligning and correcting fabric images to match a specific angle and removing distortions, the method enhances the accuracy of defect detection in fabrics using machine learning models.
Patent Information
- Application Number
- JP2023079103
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-12
- Publication Date
- 2024-11-22
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing methods for determining defects in fabrics using machine learning models are inaccurate due to deviations in fabric orientation and deformations such as skewing and distortion, which reduce the accuracy of defect detection.
A program for a computer that includes image correction functions to align fabric images with a specific angle and correct distortions, followed by inputting the corrected images into a machine learning model trained on aligned and undistorted fabric images, ensuring accurate defect detection.
The proposed method effectively determines the presence of defects in fabrics by correcting image orientation and distortion, thereby improving the accuracy of defect detection in woven or knitted fabrics.
Smart Images

Figure 2024163476000001_ABST
Abstract
Description
[Technical field]
[0001] The present specification relates to a technique for determining the presence or absence of defects in a woven or knitted fabric. [Background technology]
[0002] Techniques using machine learning to identify defects on a material surface have been proposed. For example, in the technique described in Patent Literature 1, an image of a target material surface is input to a neural network to generate a defect-reduced version, the defect-reduced version is compared with the input image to identify differences, and defects on the target material surface are identified at the positions of the identified differences. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2020-187735 A Summary of the Invention [Problem to be solved by the invention]
[0004] In order to determine the presence or absence of defects in a woven or knitted fabric, a scanned image of the fabric and a machine learning model may be used. However, when a scanned image of the fabric is used, the accuracy of the judgment result obtained using the machine learning model may decrease due to various causes. For example, the orientation of the fabric in the scanned image may deviate from the orientation suitable for the machine learning model. Such a deviation in the orientation of the fabric may decrease the accuracy of the judgment result. In addition, the fabric is more easily deformed than hard members such as metal, hard resin, and glass. For example, when the fabric is transported, the fabric may be deformed, such as skewed or distorted. The deformation of the fabric in the scanned image may decrease the accuracy of the judgment result.
[0005] This specification discloses a technique for appropriately determining whether or not a fabric has a defect. [Means for solving the problem]
[0006] The technology disclosed in this specification can be realized in the following application examples.
[0007] [Application Example 1] A program for a computer that judges whether or not there is a defect in a woven or knitted fabric, the program causing a computer to realize the following functions: a function for acquiring data of a read image, which is an optically read image of the target fabric; a correction function for performing a correction process on the read image, the correction process including a first correction that brings an angle of a first direction, which is a direction in which a first pattern formed by yarn is regularly arranged in the image representing the target fabric, relative to a first reference direction closer to a first specific angle; and a judgment function for inputting the image processed by the correction process into a machine learning model to judge whether or not there is a defect in the target fabric, the machine learning model being trained using an image of fabric in which the angle of the first direction is the first specific angle.
[0008] According to this configuration, even if the angle of the first direction indicated by the texture represented in the read image deviates from the first specific angle, it is possible to appropriately determine whether or not there is a defect in the texture.
[0009] [Application Example 2] A program for a computer to determine the presence or absence of defects in a woven or knitted fabric, the program causing the computer to realize a function of acquiring data of a read image, which is an optically read image of a target fabric, and a judgment function of inputting the read image into a machine learning model to determine the presence or absence of defects in the target fabric, the machine learning model being trained using a plurality of training images representing fabric, the plurality of training images being configured such that a distribution range of angles of a plurality of first directions of the plurality of training images relative to a first reference direction is a first angle range, the first direction being a direction in which a first pattern formed by yarns in the image of the fabric is regularly arranged, and the first angle range being determined using a distribution of angles of the first direction based on one or both of the rotation and skew of the fabric estimated using a read image of a fabric supported by a conveying device that conveys the target fabric for reading the target fabric.
[0010] According to this configuration, even if the angle of the first direction indicated by the texture represented by the read image changes, it is possible to appropriately determine whether or not there is a defect in the texture.
[0011] [Application Example 3] A program for a computer that determines whether or not there is a defect in a woven or knitted fabric, the program causing a computer to realize a function of acquiring data of a read image, which is an optically read image of the target fabric, a correction function of performing a correction process on the read image, the correction process including distortion correction that corrects distortion of the target fabric in an image representing the target fabric, and a judgment function of inputting the image processed by the correction process into a machine learning model to determine whether or not there is a defect in the target fabric.
[0012] According to this configuration, even if the texture shown in the read image has distortion, it is possible to appropriately determine whether or not there is a defect in the texture.
[0013] [Application Example 4] A program for a computer that judges whether or not there is a defect in a woven or knitted fabric, the program causing the computer to realize a function of acquiring data of a read image, which is an optically read image of the target fabric, and a judgment function of inputting the read image into a machine learning model to judge whether or not there is a defect in the target fabric, the machine learning model being trained using a second type of image obtained by performing image processing, including the addition of distortion, on a first type of image representing the fabric.
[0014] According to this configuration, even if the texture shown in the read image has distortion, it is possible to appropriately determine whether or not there is a defect in the texture.
[0015] The technology disclosed in this specification can be realized in various forms, such as a method and device for determining the presence or absence of a defect, a method and device for generating a machine learning model, a computer program for realizing the functions of those methods or devices, a recording medium (e.g., a non-transitory recording medium) on which that computer program is recorded, and the like. [Brief description of the drawings]
[0016] [Figure 1] FIG. 1 is an explanatory diagram illustrating a data processing device according to an embodiment; [Diagram 2] FIG. 1 is a perspective view of a digital camera 110, a fabric 700, and a conveying device 900. [Diagram 3] 1 is a flow chart illustrating an example of a training process. [Figure 4] 13A-13G are diagrams showing information processed in the training process. [Diagram 5] 13 is a flowchart showing an example of an estimation process of a first angle AA1. [Figure 6] FIG. 3 is a block diagram illustrating an example of an anomaly detection model 310. [Figure 7] 13 is a flowchart illustrating an example of an inspection process. [Figure 8] 13A to 13D are diagrams showing examples of block images and anomaly maps. [Figure 9] 13 is a flowchart showing an example of a training process according to the second embodiment. [Figure 10] 11 is a histogram showing an example of the distribution of a first angle AA1. [Figure 11] 13 is a flowchart showing an example of an inspection process according to a second embodiment. [Figure 12] 13 is a flowchart showing an example of a training process according to a third embodiment. [Figure 13] 13 is a flowchart showing an example of an estimation process of a first angle AC1. [Figure 14]1A-1E are diagrams showing information processed in relation to a first angle AC1, and 1F-1J are diagrams showing information processed in relation to a second angle AC2. [Figure 15] 13 is a flowchart showing an example of an estimation process of a second angle AC2. [Figure 16] 13 is a flowchart illustrating an example of an inspection process according to a third embodiment. [Figure 17] 13 is a flowchart showing an example of a training process according to a fourth embodiment. [Figure 18] 13A is a histogram showing an example of the distribution of the first angle AC1, and FIG. 13B is a histogram showing an example of the distribution of the second angle AC2. [Figure 19] 13 is a flowchart showing an example of a training process according to the fifth embodiment. [Figure 20] 13A-13E are diagrams showing information processed in the training process. [Figure 21] FIG. 3 is a block diagram illustrating an example of a distortion correction model 320. [Figure 22] 13 is a flowchart illustrating an example of an inspection process according to the fifth embodiment. [Diagram 23] 23 is a flowchart showing an example of a training process according to the sixth embodiment. [Figure 24] 23 is a flowchart showing an example of an inspection process according to the sixth embodiment. [Diagram 25] FIG. 13 is a diagram illustrating an example of distortion correction using keypoint matching. [Figure 26] 23 is a flowchart showing an example of a training process according to the seventh embodiment. [Figure 27] 23 is a flowchart showing an example of a training process according to the eighth embodiment. [Figure 28] 23 is a flowchart showing an example of an inspection process according to the eighth embodiment. [Figure 29] 13 is a flowchart showing an example of a training process according to the ninth embodiment. [Diagram 30] FIG. 1 is a diagram showing an example of a knitting pattern. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0017] A. First Example: A1. Equipment configuration: 1 is an explanatory diagram showing a data processing device according to an embodiment. The data processing device 200 is, for example, a personal computer. The data processing device 200 performs various data processing for inspecting the appearance of an object (for example, a woven or knitted fabric). Hereinafter, it is assumed that the appearance of a woven fabric 700 is inspected.
[0018] The data processing device 200 includes a processor 210, a storage device 215, a display unit 240, an operation unit 250, a graphics processing unit 260 (referred to as GPU 260), and a communication interface 270. These elements are connected to each other via a bus. The storage device 215 includes a volatile storage device 220 and a non-volatile storage device 230.
[0019] The processor 210 is a device configured to perform data processing, and is, for example, a central processing unit (CPU) or a system on a chip (SoC). The volatile storage device 220 is, for example, a DRAM, and the non-volatile storage device 230 is, for example, a flash memory. The non-volatile storage device 230 stores data of a training program 232, an inspection program 233, an anomaly detection model 310, and a distortion correction model 320. The anomaly detection model 310 and the distortion correction model 320 are program modules that form a machine learning model. Details of the programs 232-233 and the models 310 and 320 will be described later. The distortion correction model 320 will be used in other embodiments described later (omitted in this embodiment).
[0020] The display unit 240 is a device configured to display images, such as a liquid crystal display or an organic EL display. The operation unit 250 is a device configured to receive operations by a user, such as a button, a lever, or a touch panel overlaid on the display unit 240. The display unit 240 and the operation unit 250 may form a so-called touch screen. The user can input various requests and instructions to the data processing device 200 by operating the operation unit 250.
[0021] The GPU 260 is a computing device configured to execute various numerical calculations such as image processing, machine learning, etc. The GPU 260 executes various calculations according to instructions from the processor 210. Note that a driver program (not shown) for controlling the GPU 260 may be provided by the manufacturer of the GPU 260.
[0022] The communication interface 270 is an interface for communicating with other devices (e.g., one or more of a USB interface, a wired LAN interface, an IEEE802.11 wireless interface, and an industrial camera interface (e.g., CameraLink, CoaXPress, etc.)). The digital camera 110 and the conveying device 900 are connected to the communication interface 270. The conveying device 900 is a device that conveys the textile 700. The digital camera 110 is used to photograph the textile 700.
[0023] 2 is a perspective view of the digital camera 110, the textile 700, and the conveying device 900. In this embodiment, the textile 700 is conveyed by the conveying device 900. The conveying device 900 has two rollers 910 and 920. The textile 700 is guided by the two rollers 910 and 920. Between the two rollers 910 and 920, the textile 700 forms a flat portion. The light source 120 irradiates light onto the flat portion of the textile 700. The shooting direction D110 of the digital camera 110 faces the flat portion of the textile 700.
[0024] The conveying device 900 has a rotary encoder 912 connected to the first roller 910 and a drive motor 922 connected to the second roller 920. The drive motor 922 conveys the fabric 700 from the first roller 910 to the second roller 920 by rotating the second roller 920. The data processing device 200 can obtain the conveying position of the fabric 700 using data from the rotary encoder 912. The data processing device 200 can control the conveyance of the fabric 700 by controlling the drive motor 922.
[0025] An example of a pattern of the woven fabric 700 is shown at the bottom of FIG. 2. The woven fabric 700 is woven using warp threads Wp and weft threads Wf. The woven fabric 700 may be woven in various ways. Patterns W1, W2, and W3 in the figure indicate "plain weave", "twill weave", and "satin weave", respectively. In either case, the woven fabric 700 is configured such that patterns associated with the weave are regularly arranged. When a plurality of specific patterns are arranged at approximately equal intervals in a specific direction, it can be said that the specific patterns are regularly arranged in a specific direction. The captured image of the woven fabric 700 generated by the digital camera 110 shows the fine patterns of the woven fabric 700.
[0026] A2. Training process: 3 is a flow chart showing an example of a training process for the anomaly detection model 310 (FIG. 1). The anomaly detection model 310 is used to detect anomalies in the textile 700 in an inspection process described below. The anomaly detection model 310 may be any model that detects anomalies. Furthermore, the training process for the anomaly detection model 310 may be any process suitable for the anomaly detection model 310.
[0027] In this embodiment, the anomaly detection model 310 is trained using an image of a training fabric without anomalies (details of the anomaly detection model 310 will be described later). Here, it is assumed that a photographed image of a training fabric is used. In the photographed image, the image of the fabric may be tilted (i.e., the image of the fabric may be rotated) due to various causes. Causes of the rotation of the image of the fabric include, for example, an error in adjusting the orientation of the digital camera 110 with respect to the conveying direction of the fabric by the conveying device 900, an error in the mounting position of the fabric 700 with respect to the conveying device 900, and deformation of the soft fabric. Unintended rotation of the image of the fabric in the photographed image may reduce the accuracy of the inspection using the anomaly detection model 310. Therefore, in this embodiment, the processor 210 corrects the angle of the image of the fabric in the photographed image in each of the training process and the inspection process.
[0028] The operator attaches a training fabric 700t (also referred to as a training fabric 700t) to the conveying device 900. The training fabric 700t is a fabric 700 without any abnormalities. Then, the operator operates the operation unit 250 of the data processing device 200 (FIG. 1) to input an instruction to start the training process. In response to the instruction, the processor 210 starts the training process according to the training program 232.
[0029] In S110, the processor 210 provides a shooting instruction to the digital camera 110. The digital camera 110 shoots the training fabric 700t. The processor 210 acquires shot image data from the digital camera 110.
[0030] 4(A)-4(G) are diagrams showing information processed in the training process. FIG. 4(A) shows an example of a captured image of a woven fabric 700. In the figure, a captured image Ba1 and an enlarged view of a part Ba1P are shown. The captured image Ba1 is a rectangular image having two sides parallel to a first direction Dx and two sides parallel to a second direction Dy perpendicular to the first direction Dx. The data of the captured image Ba1 is color bitmap data representing the color values of each of the components R (red), G (green), and B (blue) for each pixel. Note that the color components represented by the captured image data may be other color components (e.g., luminance values) instead of R, G, and B.
[0031] As shown in the figure, the photographed image Ba1 includes a first portion Pa1 and a second portion Pa2, which are different in brightness from each other. Here, the first portion Pa1 is a bright island-like portion, and the second portion Pa2 is a dark remaining portion. The photographed image Ba1 shows a plurality of first portions Pa1 arranged regularly. The pattern shown by the first portion Pa1 and the second portion Pa2 shows a pattern formed by the weft thread Wf and the warp thread Wp (FIG. 2). For example, when the weaving method of the woven fabric 700 shows the third pattern W3, the weft thread Wf forms an island-like portion surrounded by the warp thread Wp. In the photographed image Ba1, the first portion Pa1 may show the weft thread Wf, and the second portion Pa2 may show the warp thread Wp.
[0032] FIG. 4(A) shows a pattern PTa consisting of two first portions Pa1 and a second portion Pa2 surrounding the first portions Pa1. This pattern PTa is an example of a pattern formed by threads Wf and Wp. This pattern PTa is regularly arranged toward the lower right. Although not shown, on the photographed image Ba1, a plurality of patterns PTa are arranged linearly toward the lower right to form a line PTL. The first direction Da is the direction in which the line PTL extends. Although not shown, on the photographed image Ba1, a plurality of lines PTL are arranged in the second direction Dy.
[0033] In S130 (FIG. 3), the processor 210 divides the captured image into a plurality of block images. The size of one block image (i.e., width (number of pixels in the first direction Dx) and height (number of pixels in the second direction Dy)) is a size acceptable to the anomaly detection model 310 (FIG. 1).
[0034] In S140, the processor 210 estimates a first angle AA1 of each block image. FIG. 5 is a flowchart showing an example of a process for estimating the first angle AA1. In S410, the processor 210 binarizes the block image. Any method for binarization may be used (for example, Otsu's binarization). FIG. 4(B) shows a portion of the binarized block image. As shown in the figure, the binarized image Ba2 represents a first portion Pa1 and a second portion Pa2.
[0035] In S420 (FIG. 5), the processor 210 selects a feature point of interest from a plurality of feature points included in the binarized image. In this embodiment, an island-like portion (here, the first portion Pa1) is used as the feature point. The processor 210 selects the unprocessed first portion Pa1 as the feature point of interest.
[0036] In S430, the processor 210 selects the closest feature point from the area to the lower right of the target feature point. For example, if the feature point Pa1a (FIG. 4(B)) is the target feature point, the feature point Pa1b to the lower right of the target feature point Pa1a is selected. These two feature points Pa1a and Pa1b form a pattern PTa.
[0037] In S440 (FIG. 5), an angle AA1v of a vector Va extending from the starting point to the end point is calculated. FIG. 4(C) shows an example of the vector Va. The vector Va extends from the starting point Ps to the end point Pe. In this embodiment, the starting point Ps is the center of gravity of the feature point of interest selected in S420, and the end point Pe is the center of gravity of the feature point selected in S430. The angle AA1v is the angle of the vector Va with respect to the reference direction. In this embodiment, the reference direction is the first direction Dx (the first direction Dx is also referred to as the reference direction Dx). The angle AA1v is an angle at which the direction obtained by rotating the reference direction Dx clockwise by the angle AA1v is the same as the direction of the vector Va.
[0038] In S450 (FIG. 5), processor 210 determines whether or not processing of all feature points is complete. If unprocessed feature points remain (S450: No), processor 210 proceeds to S420 to process new feature points. As shown in FIG. 4(D), processor 210 calculates angles AA1v of the multiple vectors Va of the multiple first portions Pa1.
[0039] When the processing of all the feature points is completed (FIG. 5: S450: Yes), in S460, the processor 210 estimates the first angle AA1 of the block image using the multiple angles AA1v of the multiple vectors Va. The first angle AA1 may be various representative angles represented by the multiple angles AA1v. In this embodiment, the processor 210 generates a histogram of the angle AA1v. FIG. 4(E) shows an example of the histogram of the angle AA1v. The horizontal axis indicates the angle AA1v, and the vertical axis indicates the frequency F. Note that zero degrees and 360 degrees of the angle AA1v indicate the same angle. The same applies to other angles described later. The processor 210 adopts the angle associated with the maximum frequency F as the first angle AA1.
[0040] The method of estimating the first angle AA1 may be various other methods. For example, the processor 210 may calculate an approximation curve Cav that approximates the histogram, and adopt the angle of the peak of the approximation curve Cav as the first angle AA1. The approximation curve Cav may be expressed by various functions (for example, a Gaussian function that represents a normal distribution). The processor 210 may also adopt various summary statistics (average, median, mode, etc.) of the multiple angles AA1v as the first angle AA1.
[0041] As shown in Figures 4(B) and 4(C), vector Va indicates the direction in which two first portions Pa1 (e.g., feature points Pa1a and Pa1b) that form pattern PTa are arranged. The multiple patterns PTa are arranged regularly in approximately the same direction as this vector Va. The first angle AA1 indicates the direction in which the multiple patterns PTa are arranged regularly (i.e., the first direction Da, which is the direction in which the line PTL formed by the multiple patterns PTa extends).
[0042] After estimating the first angle AA1 (FIG. 5: S460), the processor 210 ends the process of FIG. 5. In addition, in S140 (FIG. 3), the processor 210 determines the first angle AA1 for each block image. When the first angle AA1 of all block images has been determined, the process proceeds to S150.
[0043] In S150, the processor 210 corrects the angle of each block image so that the first angle AA1 becomes the first specific angle AA1Z. In this embodiment, the first specific angle AA1Z is determined in advance. The processor 210 also corrects the first angle AA1 by rotating the block image.
[0044] FIG. 4C shows an example of the first specific angle AA1Z. Like the angle AA1v, the first specific angle AA1Z indicates an angle with respect to the reference direction Dx. The direction Daz is a direction determined by the first specific angle AA1Z (referred to as the reference direction Daz). It is preferable to adopt a direction associated with a typical first angle AA1 as the reference direction Daz. This makes it possible to reduce the rotation angle of the block image for correction.
[0045] FIG. 4(F) shows an example of correction of the first angle AA1. The processor 210 rotates the block image Ba3 so that the first angle AA1 is equal to the first specific angle AA1Z. By this rotation, the processor 210 generates data of the corrected block image Ba4. The angle of rotation is the difference between the first specific angle AA1Z and the angle AA1v. The center of rotation may be the center of the block image Ba3. The height and width of the corrected block image Ba4 are the same as the height and width of the block image Ba3. The portion P3a of the block image Ba3 that protrudes from the corrected block image Ba4 may be cut off. The color of the pixels of the portion P4a of the corrected block image Ba4 to which the block image Ba3 is not assigned may be set to a predetermined color (e.g., black). Alternatively, the processor 210 may acquire a partial image corresponding to the portion P4a from the original captured image acquired in S110 (FIG. 3) (i.e., the captured image before division), and assign the acquired partial image to the portion P4a. Here, the processor 210 acquires a partial image from the captured image rotated at the rotation angle of the block image Ba3. Hereinafter, the corrected block image is referred to as a training block image.
[0046] As described later, in this embodiment, the image is corrected so that the first angle AA1 is the same as the first specific angle AA1Z in the inspection process. In such an angle-corrected image, a small tilt of the image of the fabric 700 may remain. For example, since the fabric 700 is soft, the image of the fabric 700 may be tilted in a part of the image. Therefore, in this embodiment, the processor 210 generates data of a plurality of training images showing various first angles AA1 by data augmentation.
[0047] In S152 (FIG. 3), the processor 210 determines a specific range AA1S using the distribution of the multiple first angles AA1 obtained in S140. FIG. 4(G) is a histogram showing an example of the distribution of the first angle AA1. The horizontal axis indicates the first angle AA1, and the vertical axis indicates the frequency F. The processor 210 determines a reference range AA1A so as to include most of the multiple first angles AA1. The processor 210 determines the width (referred to as a first width) of the specific range AA1S by narrowing the width of the reference range AA1A. For example, the processor 210 calculates the first width of the specific range AA1S by multiplying the width of the reference range AA1A by a predetermined first coefficient (e.g., 0.1 or more and 0.9 or less) that is greater than zero and less than 1. The processor 210 adopts the range of the first width centered on the first specific angle AA1Z as the specific range AA1S. The first coefficient is experimentally determined in advance so that the first angle AA1 indicated by the tilt of the image of the textile 700 in the angle-corrected image falls within a specific range AA1S.
[0048] The method of determining the reference range AA1A may be various methods. For example, the processor 210 may arrange the multiple first angles AA1 in ascending order, and may adopt the distribution range of the remaining first angles AA1 excluding a part of the upper angles (e.g., 5%) and a part of the lower angles (e.g., 5%) as the reference range AA1A. The processor 210 may also calculate an approximation curve Caa that approximates the histogram, and adopt the range determined by the approximation curve Caa as the reference range AA1A. For example, the approximation curve Caa may be expressed by a Gaussian function that represents a normal distribution. The reference range AA1A may be a range having a width four times the standard deviation centered on the average of the normal distribution. In this way, it is preferable that the processor 210 determines the reference range AA1A so as to include the remaining multiple first angles AA1 excluding the outliers. In either case, the range including the first specific angle AA1Z is adopted as the specific range AA1S.
[0049] In S154 (FIG. 3), the processor 210 generates new data of a plurality of training block images by data augmentation. In this embodiment, the processor 210 generates a plurality of training block images by rotating the training block image (i.e., the corrected block image) generated in S150 at various angles. Here, the processor 210 generates a plurality of training block images so that a plurality of first angles AA1 of the plurality of training block images finally obtained are approximately uniformly distributed within a specific range AA1S. In S150, a plurality of corrected block images are generated. The processor 210 generates a plurality of training block images using the plurality of corrected block images. If a missing portion (e.g., the portion P4a in FIG. 4(F)) occurs due to the rotation of the image, the processor 210 may fill in the missing portion using the original captured image (S110) as described in S150 (FIG. 3).
[0050] Here, the processor 210 will ultimately generate data of N training block images (N is an integer equal to or greater than 2). The number N of training block images may be any number suitable for training the anomaly detection model 310 (e.g., about several hundreds or several thousands). In order to generate the N training block images, the processor 210 may acquire data of a plurality of captured images in S110. Then, in S130, the processor 210 may acquire a plurality of block images from each of the plurality of captured images. Here, it is preferable that the portions of the training fabric 700t represented by the captured images are different between the plurality of captured images. For example, the processor 210 may cause the conveying device 900 to convey the fabric 700t every time the fabric 700t is photographed.
[0051] In S290 (FIG. 3), the processor 210 trains the anomaly detection model 310 using N training block images. FIG. 6 is a block diagram showing an example of the anomaly detection model 310. In this embodiment, the anomaly detection model 310 is configured using an anomaly detection technique called PaDiM (a Patch Distribution Modeling Framework for Anomaly Detection and Localization). PaDiM is disclosed, for example, in the paper "T. Defard, A. Setkov, A. Loesch, and R. Audigier, "Padim: a patch distribution modeling framework for anomaly detection and localization", arXiv:2011.08785(2020), https: / / arxiv.org / abs / 2011.08785, posted on 17 Nov 2020".
[0052] In the PaDiM technology, a feature vector is generated from image data using a feature extraction model. A plurality of feature vectors are generated in advance from a plurality of image data representing normal objects, and a mean vector and a covariance matrix indicating the distribution of the plurality of feature vectors are calculated (the set of the mean vector and the covariance matrix is also called Gaussian parameters). The Gaussian parameters are calculated by a training process. In an inspection process, a feature vector is generated from image data representing an object to be inspected. Then, the distance obtained from the feature vector and the Gaussian parameters (here, the Mahalanobis distance) is calculated as an anomaly score.
[0053] In this embodiment, a part of an image classification model 310t for classifying images into a plurality of classes is used as the feature extraction model 310a. Input image data Id is input to the image classification model 310t. The input image data Id is bitmap data representing a rectangular image having two sides parallel to a first direction Dx and two sides parallel to a second direction Dy perpendicular to the first direction Dx. The image classification model 310t uses the input image data Id to generate output data Od indicating the type of the input image data Id (e.g., the confidence level for each class).
[0054] Various models can be adopted as the image classification model 310t. In this embodiment, a known model called ResNet18 is used as the image classification model 310t. ResNet18 is disclosed in, for example, the paper "K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in ICML, 2016."
[0055] In an embodiment, the image classification model 310t includes a plurality of processing blocks BC1-BC4, BF. Each of the processing blocks BC1-BC4 is a Convolutional Neural Network (CNN). Each of the processing blocks BC1-BC4 includes one or more convolution layers. The processing block BF includes one or more fully connected layers. The processing block BF generates output data Od by reducing the dimension of a feature map from a previous processing block.
[0056] The image classification model 310t has a plurality of calculation parameters. For example, each convolution layer has a filter weight and a bias used for the convolution calculation. Each fully connected layer has a weight and a bias used for the calculation. The plurality of calculation parameters are adjusted by a training process of the image classification model 310t. Note that data of various trained image classification models is published on the Internet. In this embodiment, the published trained image classification model data is used as the image classification model 310t. The trained image classification model has been trained using a large number of image data (for example, a dataset called ImageNet classified into 1000 classes). Such a trained model can appropriately extract the features of the image of the textile 700.
[0057] A plurality of feature maps obtained from a plurality of convolution layers of the image classification model 310t are used to generate the feature vector. In this embodiment, the processor 210 uses a first feature map fm1 from a convolution layer included in the first processing block BC1, a second feature map fm2 from a convolution layer included in the second processing block BC2, and a third feature map fm3 from a convolution layer included in the third processing block BC3. The feature extraction model 310a included in the anomaly detection model 310 is the first part of the image classification model 310t, and a part including the processing blocks BC1-BC3 is used. The feature extraction model 310a may include the entire image classification model 310t.
[0058] The feature maps fm1, fm2, and fm3 each represent the feature amount of a plurality of pixels aligned along a first direction Dx and a second direction Dy. The width W indicates the number of pixels in the first direction Dx of the first feature map fm1, and the height H indicates the number of pixels in the second direction Dy of the first feature map fm1. The feature maps fm1, fm2, and fm3 have different resolutions. The order of high resolution is fm1, followed by fm2, and then fm3.
[0059] In the figure, the numbers n1, n2, and n3 represent the number of channels of the feature maps fm1, fm2, and fm3, respectively. Here, it is assumed that n1 < n2 < n3.
[0060] The processor 210 converts the feature maps fm2 and fm3 into feature maps fm2r and fm3r having the same resolution as the largest feature map fm1 by a known enlargement process (i.e., resolution conversion process. For example, nearest neighbor). The resolution conversion is performed for each channel. The entirety of these feature maps fm1, fm2r, and fm3r represents W*H*(n1 + n2 + n3) feature amounts.
[0061] The processor 210 forms a feature matrix FM using R channels out of the n1 + n2 + n3 channels of the feature maps fm1, fm2r, and fm3r (R < n1 + n2 + n3). The R channels may be selected in advance. Alternatively, the processor 210 may randomly select (e.g., using random numbers) the R channels. The feature matrix FM is composed of H*W*R elements. The feature matrix FM represents the feature vector of each of the H*W pixels. The feature vector xij in the figure indicates the feature vector at the position (i, j) (i indicates the pixel position in the first direction Dx, and j indicates the pixel position in the second direction Dy). The feature vector xij is an R-dimensional vector having R elements.
[0062] At the bottom of FIG. 6, N training block images TI1 - TIN are shown. The processor 210 calculates N feature matrices FM using the N training block images TI1 - TIN. As a result, N feature vectors are calculated for each of the H*W pixel positions.
[0063] At the bottom of FIG. 6, N feature vectors xij,1-xij,N calculated from N training block images TI1-TIN are shown. These N feature vectors are associated with positions (i, j). The processor 210 calculates a mean vector μij and a covariance matrix Σij using the N feature vectors xij,1-xij,N. The mean vector μij indicates an average feature vector xij obtained from an image of the normal textile 700t. The covariance matrix Σij is a matrix of covariance between elements of a plurality of feature vectors xij obtained from an image of the normal textile 700t. The covariance matrix Σij is calculated according to the formula described in the above-mentioned paper by PaDiM. The set of the mean vector μij and the covariance matrix Σij is also called a Gaussian parameter.
[0064] The processor 210 calculates Gaussian parameters for each of the H*W pixels. As a result, the processor 210 determines the value of each element of the Gaussian matrix GM composed of the H*W Gaussian parameters. The processor 210 stores data of the anomaly detection model 310 including the feature extraction model 310a and the Gaussian matrix GM in the storage device 215 (here, the non-volatile storage device 230). Then, the processor 210 ends the training process in Fig. 3. As described above, in this embodiment, the value of each element of the Gaussian matrix GM is determined by the training process.
[0065] A3.Inspection process: 7 is a flowchart showing an example of the inspection process. An operator attaches the fabric 700 to be inspected (also referred to as the target fabric 700) to the conveying device 900 (FIG. 2). Then, the operator operates the operation unit 250 of the data processing device 200 (FIG. 1) to input an instruction to start the inspection process. In response to the instruction, the processor 210 starts the inspection process according to the inspection program 233.
[0066] In S710, the processor 210 transports the target fabric 700 by controlling the drive motor 922 (FIG. 2) so as to capture an image of an uninspected portion of the target fabric 700. Note that in the first S710 after the start of the inspection process, transportation of the target fabric 700 may be omitted.
[0067] In S720, the processor 210 provides a shooting instruction to the digital camera 110. The digital camera 110 shoots the target fabric 700. The processor 210 acquires shot image data from the digital camera 110.
[0068] In S730, the processor 210 divides the captured image into a plurality of block images in the same manner as in S130 (FIG. 3).
[0069] In S740, the processor 210 estimates the first angle AA1 of each of the multiple block images. The method of estimating the first angle AA1 is the same as the method of estimating the first angle AA1 in S140 (FIG. 3).
[0070] In S750, the processor 210 corrects the angle of each block image so that the first angle AA1 becomes the first specific angle AA1Z. The correction method is the same as the correction method in S150 (FIG. 3).
[0071] In S890, the processor 210 inputs the corrected block image into the anomaly detection model 310 and judges whether the target textile 700 has an anomaly (i.e., a defect). The judgment method may be any method suitable for the anomaly detection model 310. In this embodiment, the processor 210 calculates an anomaly score for each of the H*W positions by the method described in the above-mentioned PaDiM paper. Specifically, the processor 210 calculates a feature matrix FM by inputting the corrected block image into the anomaly detection model 310 (FIG. 6). Here, the R channels used during training are used. The feature matrix FM represents the feature vectors for each of the H*W positions. The processor 210 calculates the anomaly score by using the feature vector xij and the Gaussian parameters (μij, Σij). As the anomaly score, the Mahalanobis distance between the feature vector xij and the Gaussian parameters (μij, Σij) is adopted (as the calculation formula for the anomaly score, the calculation formula described in the above-mentioned PaDiM paper is adopted). An anomaly score is calculated for each of the H*W locations, with a larger anomaly score indicating a higher probability of an anomaly.
[0072] The anomaly scores for each of the H*W positions form a so-called anomaly map (also called a heat map). Figures 8(A)-8(D) show examples of block images and anomaly maps. Note that the resolution of the anomaly map is lower than that of the block images. In the figures, each image is scaled to the same size for ease of comparison.
[0073] 8A shows a block image BIa and an anomaly map BHa of a normal target fabric 700a. As shown, the anomaly map BHa does not show any abnormal areas with large anomaly scores.
[0074] 8B shows a block image BIb of a target fabric 700b having a linear defect 701b and an anomaly map BHb. Such a defect 701b may be formed due to various causes. For example, a defect in a yarn (e.g., a weft yarn Wf or a warp yarn Wp (FIG. 2)) may form the defect 701b. Also, the defect 701b may be formed due to contact between the target fabric 700b and another member (e.g., a device for transporting the fabric 700). The anomaly map BHb shows a linear anomaly portion ANb indicating the defect 701b.
[0075] 8(C) shows a block image BIc of a target fabric 700c having a point-like defect 701c and an anomaly map BHc. Such a defect 701c may be formed due to various causes. For example, fraying of a yarn (e.g., a weft yarn Wf or a warp yarn Wp (FIG. 2)) may form the defect 701c. Also, the defect 701c may be formed due to contact between the target fabric 700b and another member (e.g., a device for transporting the fabric 700). The anomaly map BHc shows a point-like abnormal portion ANc indicating the defect 701c.
[0076] 8D shows a block image BId of a target fabric 700d having a stain 701d and an anomaly map BHd. The anomaly map BHd shows an anomaly portion ANd indicative of the stain 701d.
[0077] The processor 210 calculates an anomaly score for each of the multiple positions of the multiple block images. The processor 210 uses the multiple anomaly scores to determine whether an anomaly condition for determining that the target textile 700 has an anomaly is satisfied. The anomaly condition may be various conditions. For example, the anomaly condition may be that the maximum anomaly score is equal to or greater than a first threshold. The first threshold may be determined experimentally in advance so as to appropriately detect an anomaly in the target textile 700. The anomaly condition may be that the total number of pixels having an anomaly score equal to or greater than a second threshold is equal to or greater than a third threshold. The second and third thresholds may be determined experimentally in advance so as to appropriately detect an anomaly in the target textile 700.
[0078] In S894 (FIG. 7), the processor 210 determines whether the abnormal condition is satisfied. If the abnormal condition is not satisfied (S894: No), in S896, the processor 210 determines whether the conveyance is completed. If an uninspected portion of the target fabric 700 remains, the determination result of S896 is No. In this case, the processor 210 proceeds to S710 and inspects the uninspected portion of the target fabric 700. If the conveyance is completed (S896: Yes), the processor 210 ends the inspection process.
[0079] If the abnormality condition is satisfied in S894 (S894: Yes), the processor 210 executes a first process in S898. The first process may be any process related to an abnormality. In this embodiment, the processor 210 stops the conveyance of the target fabric 700 and notifies the operator of the abnormality. Various methods may be used to notify the operator of the abnormality. For example, the processor 210 may display an error message or an abnormality map on the display unit 240. Alternatively, the processor 210 may output an alarm sound from a speaker (not shown). After S898, the processor 210 ends the inspection process.
[0080] As described above, in this embodiment, the processor 210 of the data processing device 200 executes the following process to determine whether or not the target fabric 700 has a defect. In S720 and S730 (FIG. 7), the processor 210 acquires block image data representing a captured image of the target fabric 700. The block image data is an example of read image data, which is an image of the target fabric 700 optically read. In S750, the processor 210 performs a correction process for the block image. As shown in FIG. 4(F), the correction process includes a correction to bring the first angle AA1 of the first direction Da (FIG. 4(F)) relative to the reference direction Dx closer to the first specific angle AA1Z. As described in FIG. 4(A) and FIG. 4(F), the first angle AA1 indicates the first direction Da, which is the direction in which a plurality of patterns PTa in the image representing the target fabric 700 are regularly arranged. The pattern PTa is formed by the yarns Wf and Wp. In S890, the processor 210 inputs the processed block image by the correction process to the anomaly detection model 310 to determine the presence or absence of a defect in the target textile 700. As described in S150 and S290 of Fig. 3 and Fig. 5, the anomaly detection model 310 (specifically, the Gaussian matrix GM) is trained using an image of the training textile 700t in which the first angle AA1 is the first specific angle AA1Z (e.g., the corrected block image Ba4 (Fig. 4(F))).
[0081] According to this configuration, even if the first angle AA1 indicated by the target textile 700 represented by the block image deviates from the first specific angle AA1Z, the block image is corrected so that the first angle AA1 becomes the first specific angle AA1Z (the block image is an example of a captured image). Furthermore, the multiple training block images include a training block image in which the first angle AA1 is the first specific angle AA1Z. Therefore, the processor 210 can appropriately determine the presence or absence of a defect in the target textile 700 by using the anomaly detection model 310.
[0082] In this embodiment, as described in FIG. 3, the anomaly detection model 310 is trained using a plurality of training block images representing the training textile 700t. As described in S152 and S154, the plurality of training block images are configured such that the distribution range of the plurality of first angles AA1 of the plurality of training block images is a specific range AA1S including a first specific angle AA1Z. Then, as described in S152 (FIG. 3) and FIG. 4(G), the specific range AA1S is determined using the distribution of the first angle AA1 estimated using the photographed image of the training textile 700t. Here, the training textile 700t is supported by a conveying device 900 that conveys the target textile 700 for photographing the target textile 700. Also, the first angle AA1 indicates an angle based on the rotation of the textile 700 (for example, the training textile 700t).
[0083] According to this configuration, even if a tilt in the image of the target textile 700 remains in the corrected block image, the anomaly detection model 310 has been trained using a training block image that shows such a tilt, so the processor 210 can appropriately determine whether or not there is a defect in the target textile 700 by using the anomaly detection model 310.
[0084] The specific range AA1S is determined using the distribution of the first angle AA1 estimated using the captured image of the training fabric 700t. Therefore, it is easy to avoid using a specific range AA1S that is wider than the width of the actual distribution of the first angle AA1. As a result, the burden of training the anomaly detection model 310 can be reduced.
[0085] It should be noted that, if the effect of the tilt of the image of the target fabric 700 in the corrected block image (i.e., the effect on the inspection result) is small, S152 and S154 may be omitted.
[0086] B. Second Example: B1. Training process: 9 is a flowchart showing an example of a training process of Example 2. In this example, the processor 210 trains the anomaly detection model 310 using a plurality of training block images showing different first angles AA1.
[0087] S110, S130, and S140 are respectively the same as S110, S130, and S140 in Fig. 3. The processor 210 estimates the first angle AA1 of each of the multiple block images representing the normal fabric 700t.
[0088] In S142b, the processor 210 determines an angle range AA1R using the distribution of the multiple first angles AA1. The angle range AA1R is used for data augmentation, which will be described later. FIG. 10 is a histogram showing an example of the distribution of the first angles AA1. The horizontal axis indicates the first angle AA1, and the vertical axis indicates the frequency F. The processor 210 determines the angle range AA1R (specifically, the lower limit AA1L and the upper limit AA1H) so as to include most of the multiple first angles AA1. The method of determining the angle range AA1R may be the same as the method of determining the reference range AA1A (FIG. 4(G)) described in S152 (FIG. 3).
[0089] In S144b, the processor 210 determines a first specific angle AA1Z. In this embodiment, the processor 210 adopts the center value of the angle range AA1R (that is, the average value of the lower limit AA1L and the upper limit AA1H) as the first specific angle AA1Z.
[0090] S150 is the same as S150 in Fig. 3. The processor 210 corrects the angle of each block image so that the first angle AA1 becomes the first specific angle AA1Z.
[0091] In S154b, the processor 210 generates data of a plurality of training block images by data augmentation. In this embodiment, the processor 210 generates a plurality of training block images by rotating the corrected block images generated in S150 at various angles.
[0092] Here, the processor 210 generates data of the multiple training block images so that the multiple first angles AA1 of the multiple training block images finally obtained are approximately uniformly distributed within the angle range AA1R. For example, the processor 210 randomly determines the angle of rotation of each of the multiple training block images so that the multiple angles (corresponding to the multiple first angles AA1) obtained from the first specific angle AA1Z by the rotation are approximately uniformly distributed within the angle range AA1R.
[0093] As described above, in S150, data of a plurality of corrected block images is generated. The processor 210 generates a plurality of training block images using the plurality of corrected block images. If a missing portion (e.g., portion P4a in FIG. 4(F)) occurs due to the rotation of the image, the processor 210 may fill in the missing portion using the original captured image (S110) as described in S150 (FIG. 3).
[0094] Also, the processor 210 may acquire data of a plurality of captured images in S110 in order to generate N training block images, similarly to the embodiment of FIG.
[0095] S290 is the same as S290 in Fig. 3. The processor 210 trains the anomaly detection model 310 (here, the Gaussian matrix GM) using the N training block images. Then, the processor 210 ends the processing in Fig. 9.
[0096] B2. Inspection process: Fig. 11 is a flowchart showing an example of the inspection process of Example 2. The main difference from the inspection process of Fig. 7 is that S740 and S750 for correcting the first angle AA1 are omitted.
[0097] S710, S720, and S730 are the same as S710, S720, and S730, respectively, in Fig. 7. The processor 210 divides the captured image into a plurality of block images.
[0098] In S890b, the processor 210 inputs each of the multiple block images into the anomaly detection model 310 to determine whether the target textile 700 has an anomaly. The only difference from S890 in Fig. 7 is that an uncorrected block image is used instead of the corrected block image.
[0099] S894, S896, and S898 after S890b are the same as S894, S896, and S898, respectively, in Fig. 7. If the determination result in S896 is No, the processor 210 proceeds to S710.
[0100] As described above, in this embodiment, the processor 210 executes the following process to determine the presence or absence of a defect in the target textile 700. In S720 and S730 (FIG. 11), the processor 210 acquires data of a block image representing a photographed image of the target textile 700. In S890b, the processor 210 inputs the block image to the anomaly detection model 310 to determine the presence or absence of a defect in the target fabric. As described in FIG. 9, the anomaly detection model 310 is trained using a plurality of training block images representing the training textile 700t. The data of the plurality of training block images is generated by data augmentation in S154b. In this embodiment, the first angle AA1 of the training block image is changed by data augmentation. As described in FIG. 4(A) and FIG. 4(F), the first angle AA1 indicates the direction in which the plurality of patterns PTa are regularly arranged in the image of the textile 700 (i.e., the first direction Da in which the line PTL formed by the plurality of patterns PTa extends). The pattern PTa is formed by the yarns Wf and Wp. As described in S154b, the training block images are configured such that the distribution range of the first angles AA1 of the first directions Da of the training block images relative to the reference direction Daz is an angle range AA1R. As described in S154b and FIG. 10, the angle range AA1R is determined using the distribution of the first angles AA1 estimated using the captured images of the training fabric 700t. Here, the training fabric 700t is supported by a conveying device 900 that conveys the target fabric 700 for capturing the target fabric 700. Also, the first angle AA1 indicates the angle of the first direction Da based on the rotation of the fabric 700 (e.g., the training fabric 700t).
[0101] According to this configuration, a plurality of training block images are configured such that the distribution range of a plurality of first angles AA1 of the plurality of training block images is an angle range AA1R, and the angle range AA1R is determined using the distribution of the first angles AA1 estimated using the photographed images. Therefore, even if the first angle AA1 indicated by the target textile 700 represented by the photographed images changes, the processor 210 can appropriately determine the presence or absence of a defect in the target textile 700 by using the anomaly detection model 310. In addition, the use of an angle range AA1R wider than the width of the actual distribution of the first angles AA1 can be easily avoided. As a result, the burden of training the anomaly detection model 310 can be reduced.
[0102] C. Third Example: C1. Training process: FIG. 12 is a flowchart showing an example of the training process of the third embodiment. The difference from the embodiment of FIG. 3 is that in addition to the tilt caused by the rotation of the image of the textile 700, the skew of the textile 700 is corrected. The textile 700 may be skewed due to various causes. The cause of the skew of the textile 700 may be, for example, the application of an uneven force to the textile 700 in the manufacturing process of the textile 700, an error in the mounting position of the textile 700 relative to the conveying device 900, deformation of the soft textile 700, etc. Note that data augmentation of the angle (S152, S154 (FIG. 3)) is omitted.
[0103] S110 and S130 are the same as S110 and S130, respectively, in Fig. 3. The processor 210 divides the captured image into a plurality of block images.
[0104] In S140c, the processor 210 estimates a first angle AC1 of each block image. In this embodiment, the first angle AC1 is different from the first angle AA1 in FIG. 4(F) and indicates a direction approximately parallel to the first direction Dx (details will be described later).
[0105] Fig. 13 is a flowchart showing an example of the estimation process of the first angle AC1. S410 is the same as S410 in Fig. 5. The processor 210 binarizes the block image.
[0106] 14(A)-14(E) are diagrams illustrating information processed in relation to the first angle AC1. FIG. 14(A) illustrates an example of a portion of a binarized block image. The binarized image Bc2 represents a plurality of first portions Pc1, which are bright island-like portions, and a second portion Pc2, which is a dark remaining portion. The binarized image Bc2 represents a pattern represented by the plurality of first portions Pc1 and the second portion Pc2. This pattern is a pattern formed by the yarns Wf and Wp (FIG. 2).
[0107] FIG. 14(A) shows a first pattern PTc1 consisting of two first portions Pc1 (for example, first portions Pc11 and Pc12) and a second portion Pc2 surrounding the first portions Pc1. This first pattern PTc1 is an example of a pattern formed by threads Wf and Wp. The multiple first patterns PTc1 are regularly arranged in a direction approximately parallel to the first direction Dx. The multiple first patterns PTc1 are arranged linearly in a direction approximately parallel to the first direction Dx to form lines PTLc1. The first direction Dc1 is the direction in which the lines PTLc1 extend. Although not shown, the multiple lines PTLc1 are arranged in the second direction Dy on the binarized image Bc2. The multiple lines PTLc1 are also arranged in the second direction Dy on the captured image.
[0108] S420 (FIG. 13) is the same as S420 in FIG. 5. In this embodiment, the island-like first portion Pc1 is used as a feature point. The processor 210 selects the unprocessed first portion Pc1 as a feature point of interest.
[0109] In S430c1, the processor 210 selects a feature point from a search area located in the first direction Dx of the feature point of interest. FIG. 14(B) shows the feature point of interest Pc11 and the selection area SA11. In this embodiment, the selection area SA11 is a circular area. The center Pc11s of the selection area SA11 is disposed at a position away from the center of gravity position Pc11c of the feature point of interest Pc11 in the first direction Dx by a distance Ds1. The value Rc1 indicates the radius of the selection area SA11. As shown in FIG. 14(A), the selection area SA11 associated with the feature point of interest Pc11 includes a first portion Pc12. The processor 210 selects the first portion Pc12 from this selection area SA11.
[0110] The distance Ds1 and the radius Rc1 are experimentally determined in advance in accordance with the woven fabric 700 so that the selection area SA11 may include one entire first portion Pc1, but may not include two entire first portions Pc1. One or more first portions Pc1 may be in contact with the selection area SA11. The processor 210 selects the first portion Pc1 that is included in the selection area SA11 with the largest size (e.g., the number of pixels) from the one or more first portions Pc1 that are in contact with the selection area SA11.
[0111] In S440c1, the processor 210 calculates the angle AC1v of the vector Vc1 from the starting point to the end point. FIG. 14(C) shows an example of the vector Vc1. The method of determining the vector Vc1 is the same as the method of determining the vector Va in FIG. 4(C). The starting point Pc1s of the vector Vc1 is the position of the center of gravity of the feature point Pc1 (for example, the first part Pc11) selected in S420, and the end point Pc1e of the vector Vc1 is the position of the center of gravity of the feature point Pc1 (for example, the first part Pc12) selected in S430c1. The method of calculating the angle AC1v is the same as the method of calculating the angle AA1v in FIG. 4(C). In this embodiment, the reference direction of the angle AC1v is the first direction Dx.
[0112] S450 (Fig. 13) is the same as S450 in Fig. 5. If unprocessed feature points remain (S450: No), the processor 210 proceeds to S420 to process new feature points.
[0113] When the processing of all feature points is completed (S450: Yes), in S460c1, the processor 210 estimates a first angle AC1 of the block image using the multiple angles AC1v of the multiple vectors Vc1. The estimation method of the first angle AC1 is the same as the estimation method of the first angle AA1 in S460 of FIG. 5. FIG. 14(D) shows an example of a histogram of the angles AC1v. The processor 210 adopts the angle associated with the maximum frequency F as the first angle AC1. The estimation method of the first angle AC1 may be various other methods (for example, a method using an approximation curve Ccv1, a method using various summary statistics (average, median, mode, etc.) of the multiple angles AC1v, etc.).
[0114] 14(A), vector Vc1 indicates the direction in which two first portions Pc1 (e.g., feature points Pc11 and Pc12) that form first pattern PTc1 are arranged. The multiple first patterns PTc1 are arranged regularly in approximately the same direction as vector Vc1. First angle AC1 indicates the direction in which the multiple first patterns PTc1 are arranged regularly (i.e., first direction Dc1, which is the direction in which line PTLc1 formed by the multiple first patterns PTc1 extends).
[0115] After estimating the first angle AC1 (FIG. 13: S460c1), the processor 210 ends the processing in FIG 13. Note that in S140c (FIG. 12), the processor 210 estimates the first angle AC1 of each of the multiple block images.
[0116] In S150c, the processor 210 corrects the angle of each block image so that the first angle AC1 becomes the first specific angle AC1Z. In this embodiment, the direction of the first specific angle AC1Z is predetermined to a direction associated with a typical first angle AA1 (in this embodiment, the +Dx direction). Hereinafter, the +Dx direction is also referred to as the first reference direction. The angle correction method is the same as the correction method in S150 (FIG. 5) and FIG. 4(F). FIG. 14(E) shows an example of correction of the first angle AC1. The processor 210 generates data of the corrected block image Bc4 by rotating the block image Bc3 so that the first angle AC1 becomes the same as the first specific angle AC1Z.
[0117] Through the above steps S140c and S150c (FIG. 12), the processor 210 corrects the angle based on the rotation of the block image. Through steps S160c and S170c (FIG. 12), the processor 210 corrects the angle based on the shear (also called skew) of the corrected block image.
[0118] In S160c, the processor 210 estimates the second angle AC2 of each corrected block image. The second angle AC2 indicates a direction approximately parallel to the second direction Dy. The block image to be processed is the block image corrected in S150c.
[0119] Fig. 15 is a flowchart showing an example of the estimation process of the second angle AC2. In S410c2, the processor 210 acquires a binary image of the corrected block image of S150c (Fig. 12). The processor 210 may acquire data of the binary image by binarizing the corrected block image. Alternatively, the processor 210 may acquire data of the binary image of the corrected block image by rotating the binary image generated in S410 (Fig. 13) by the same angle as in S150c (Fig. 12).
[0120] 14(F)-14(J) are diagrams showing information processed in relation to the second angle AC2. FIG. 14(F) shows an example of a portion of a binarized block image. The binarized image Bc2r shows a plurality of first portions Pc1 which are bright island-like portions and second portions Pc2 which are dark remaining portions.
[0121] FIG. 14(F) shows a second pattern PTc2 consisting of two first portions Pc1 (for example, first portions Pc21 and Pc22) and a second portion Pc2 surrounding the first portions Pc1. This second pattern PTc2 is an example of a pattern formed by threads Wf and Wp. Although not shown in the figure, the multiple second patterns PTc2 are regularly arranged in a direction approximately parallel to the -Dy direction. The multiple second patterns PTc2 are linearly arranged in a direction approximately parallel to the -Dy direction to form lines PTLc2. The second direction Dc2 is the direction in which the lines PTLc2 extend. Although not shown in the figure, the multiple lines PTLc2 are arranged in the first direction Dx on the binarized image Bc2r. The multiple lines PTLc2 are also arranged in the first direction Dx on the captured image.
[0122] S420 (FIG. 15) is the same as S420 in FIG. 5. In this embodiment, the island-shaped first portion Pc1 is used as a feature point. The processor 210 selects the unprocessed first portion Pc1 as a feature point of interest.
[0123] In S430c2, the processor 210 selects a feature point from a search area located in the -Dy direction of the target feature point. FIG. 14(G) shows the target feature point Pc21 and the selection area SA21. In this embodiment, the selection area SA21 is a circular area. The center Pc21s of the selection area SA21 is located at a position away from the center of gravity position Pc21c of the target feature point Pc21 in the -Dy direction by a distance Ds2. The value Rc2 indicates the radius of the selection area SA21. As shown in FIG. 14(F), the selection area SA21 associated with the target feature point Pc21 includes a first portion Pc22. The processor 210 selects the first portion Pc22 from this selection area SA21.
[0124] The method of determining the distance Ds2 and the radius Rc2 is similar to the method of determining the distance Ds1 and the radius Rc1 in Fig. 14(B), respectively. Furthermore, the processor 210 selects the first portion Pc1 that is included in the selection region SA21 and has the largest size (e.g., the largest number of pixels) from among one or more first portions Pc1 that contact the selection region SA21.
[0125] In S440c2, the processor 210 calculates the angle AC2v of the vector Vc2 from the starting point to the end point. FIG. 14(H) shows an example of the vector Vc2. The method of determining the vector Vc2 is the same as the method of determining the vector Va in FIG. 4(C). The starting point Pc2s of the vector Vc2 is the position of the center of gravity of the target feature point Pc1 (for example, the first part Pc21) selected in S420 (FIG. 15), and the end point Pc2e of the vector Vc2 is the position of the center of gravity of the feature point Pc2 (for example, the first part Pc22) selected in S430c2. The method of calculating the angle AC2v is the same as the method of calculating the angle AA1v in FIG. 4(C). In this embodiment, the reference direction of the angle AC2v is the -Dy direction.
[0126] S450 (Fig. 15) is the same as S450 in Fig. 5. If unprocessed feature points remain (S450: No), the processor 210 proceeds to S420 to process new feature points.
[0127] When the processing of all the feature points is completed (S450: Yes), in S460c2, the processor 210 estimates the second angle AC2 of the block image using the multiple angles AC2v of the multiple vectors Vc2. The estimation method of the second angle AC2 is the same as the estimation method of the first angle AA1 in S460 of FIG. 5. FIG. 14(I) shows an example of a histogram of the angles AC2v. The processor 210 adopts the angle associated with the maximum frequency F as the second angle AC2. The estimation method of the second angle AC2 may be various other methods (for example, a method using an approximation curve Ccv2, a method using various summary statistics (average, median, mode, etc.) of the multiple angles AC2v, etc.).
[0128] As shown in Fig. 14(F), vector Vc2 indicates the direction in which two first portions Pc1 (e.g., feature points Pc21, Pc22) that form second pattern PTc2 are arranged. The multiple second patterns PTc2 are arranged regularly in approximately the same direction as vector Vc2. Second angle AC2 indicates the direction in which the multiple second patterns PTc2 are arranged regularly (i.e., second direction Dc2, which is the direction in which line PTLc2 formed by the multiple second patterns PTc2 extends).
[0129] After estimating the second angle AC2 (FIG. 15: S460c2), the processor 210 ends the processing in FIG 15. Note that in S160c (FIG. 12), the processor 210 estimates the second angle AC2 of each of the multiple block images.
[0130] In S170c, the processor 210 corrects the angle of each corrected block image so that the second angle AC2 becomes a second specific angle AC2Z. In this embodiment, the direction of the second specific angle AC2Z is predetermined to a direction associated with a typical second angle AA2 (in this embodiment, the -Dy direction). Hereinafter, the -Dy direction is also referred to as the second reference direction. In addition, the processor 210 corrects the second angle AC2 by shearing (also referred to as skew) the block image.
[0131] FIG. 14(J) shows an example of correction of the second angle AC2. The processor 210 generates data of a new corrected block image Bc5 by shear transforming the corrected block image Bc4 generated in S150c (FIG. 12) so that the second angle AC2 is the same as the second specific angle AC2Z (shear transform is a type of affine transformation). Various processes can be adopted as the shear transform process. For example, the processor 210 may move a pixel line (e.g., pixel line PLc) parallel to the first direction Dx of the corrected block image Bc4 parallel to the first direction Dx. The amount of movement may be set to a value proportional to the distance from a predetermined reference pixel line (e.g., pixel line PLcs at the end of the +Dy direction). The proportionality coefficient indicates the amount of change in the second angle AC2 due to the shear transform, and is associated with the difference between the second angle AC2 and the second specific angle AC2Z. If the shear transformation of the image results in missing portions (eg, portion P5c), the processor 210 may fill in the missing portions using the original captured image (S110 (FIG. 12)), as described in S150 (FIG. 3).
[0132] As described above, the angle correction based on rotation (S140c, S150c) and the angle correction based on shear (S160c, S170c) are performed. Hereinafter, the corrected block images obtained by these processes are referred to as training block images. The processor 210 generates data of N training block images. As in the embodiment of FIG. 3, the processor 210 may obtain data of a plurality of captured images in S110 in order to generate N training block images. The processor 210 may also increase the total number of training block images by data augmentation. The data augmentation process may include various processes such as enlargement, reduction, and rotation.
[0133] S290 is the same as S290 in Fig. 3. The processor 210 trains the anomaly detection model 310 (here, the Gaussian matrix GM) using the N training block images. Then, the processor 210 ends the processing in Fig. 12.
[0134] C2. Inspection process: Fig. 16 is a flowchart showing an example of the inspection process of the third embodiment. The main difference from the inspection process of Fig. 7 is that in addition to the correction of the first angle AC1 by rotation, the correction of the second angle AC2 by shear transformation is performed.
[0135] S710, S720, and S730 are the same as S710, S720, and S730, respectively, in Fig. 7. The processor 210 divides the captured image into a plurality of block images.
[0136] In S740c, the processor 210 estimates the first angle AC1 of each of the multiple block images. The method of estimating the first angle AC1 is the same as the method of estimating the first angle AC1 in S140c (FIG. 12).
[0137] In S750c, the processor 210 corrects the angle of each block image so that the first angle AC1 becomes the first specific angle AC1Z. The correction method is the same as the correction method in S150c (FIG. 12) (here, image rotation).
[0138] In S760c, the processor 210 estimates the second angle AC2 of each of the multiple corrected block images obtained in S750c. The method of estimating the second angle AC2 is the same as the method of estimating the second angle AC2 in S160c (FIG. 12).
[0139] In S770c, the processor 210 corrects the angle of each corrected block image so that the second angle AC2 becomes the second specific angle AC2Z. The correction method is the same as the correction method in S170c (FIG. 12) (here, shear transformation of the image).
[0140] S890 is the same as S890 in Fig. 7. The processor 210 inputs each of the multiple corrected block images generated by S740c-S770c to the anomaly detection model 310 to determine whether the target fabric 700 has an anomaly.
[0141] S894, S896, and S898 after S890 are the same as S894, S896, and S898, respectively, in Fig. 7. If the determination result in S896 is No, the processor 210 proceeds to S710.
[0142] As described above, in this embodiment, the processor 210 executes a correction process to determine whether or not the target textile 700 has a defect (FIG. 16). The correction process includes a correction of S750c (called a first correction) and a correction of S770c (called a second correction). The first correction (S750c) is a correction of the first angle AC1 by rotating the image as in the embodiment of FIG. 7. The second correction (S770c) is a correction of the second angle AC2 by shear transformation of the image. Specifically, as shown in FIG. 14(J), the second correction brings the second angle AC2 of the second direction Dc2 with respect to the second reference direction (here, the -Dy direction) closer to the second specific angle AC2Z by shearing of the affine transformation. As described in FIG. 14(F), the second angle AC2 indicates the second direction Dc2 in which the multiple second patterns PTc2 in the image representing the target textile 700 are regularly arranged. The second pattern PTc2 is formed by the yarns Wf and Wp. In S890, the processor 210 inputs the block image processed by the correction process to the anomaly detection model 310 to determine whether or not the target textile 700 has a defect. As described in S170c and S290 of FIG. 12, the anomaly detection model 310 (specifically, the Gaussian matrix GM) is trained using a plurality of training images (here, training block images) representing the training textile 700t. The plurality of training images include a plurality of training images corrected by S170c (FIG. 12). The correction by S170c brings the second angle AC2 of the second direction Dc2 of the training image with respect to the second reference direction (here, the -Dy direction) closer to the second specific angle AC2Z by shearing of the affine transformation, as shown in FIG. 14(J).
[0143] According to this configuration, even if the second angle AC2 indicated by the target textile 700 represented by the block image deviates from the second specific angle AC2Z, the block image is corrected so that the second angle AC2 becomes the second specific angle AC2Z (the block image is an example of a captured image). The multiple training images also include multiple training images in which the second angle AC2 of the training images is corrected to approach the second specific angle AC2Z. Therefore, the processor 210 can appropriately determine the presence or absence of a defect in the target textile 700 by using the anomaly detection model 310.
[0144] D. Fourth Example: D1. Training process: Fig. 17 is a flowchart showing an example of a training process of the fourth embodiment. The difference from the embodiment of Fig. 9 is that in addition to the data augmentation that changes the first angle AC1, data augmentation that changes the second angle AC2 is performed.
[0145] S110 and S130 are the same as S110 and S130, respectively, in Fig. 9. The processor 210 divides the captured image into a plurality of block images.
[0146] S140c is the same as S140c in Fig. 12. The processor 210 estimates the first angle AC1 of each block image.
[0147] In S142d, the processor 210 determines a first angle range AC1R using the distribution of the multiple first angles AC1. FIG. 18(A) is a histogram showing an example of the distribution of the first angles AC1. The processor 210 determines the first angle range AC1R (specifically, a lower limit AC1L and an upper limit AC1H) so as to include most of the multiple first angles AC1. The method of determining the first angle range AC1R is the same as the method of determining the angle range AA1R in S142b of FIG. 9. The first angle range AC1R may be determined using an approximation curve Cd1 that approximates the histogram of the first angles AC1.
[0148] In S144d (FIG. 17), the processor 210 determines the first specific angle AC1Z. In this embodiment, the processor 210 adopts the center value of the first angle range AC1R (i.e., the average value of the lower limit AC1L and the upper limit AC1H) as the first specific angle AC1Z.
[0149] S150c is the same as S150c in Fig. 12. The processor 210 corrects the angle of each block image so that the first angle AC1 becomes the first specific angle AC1Z.
[0150] S160c is the same as S160c in Fig. 12. The processor 210 estimates the second angle AC2 of each corrected block image.
[0151] In S162d, the processor 210 determines a second angle range AC2R using the distribution of the multiple second angles AC2. FIG. 18(B) is a histogram showing an example of the distribution of the second angles AC2. The processor 210 determines the second angle range AC2R (specifically, a lower limit AC2L and an upper limit AC2H) so as to include most of the multiple second angles AC2. The method of determining the second angle range AC2R is the same as the method of determining the first angle range AC1R in S142d. The second angle range AC2R may be determined using an approximation curve Cd2 that approximates the histogram of the second angles AC2.
[0152] In S164d (FIG. 17), the processor 210 determines the second specific angle AC2Z. In this embodiment, the processor 210 adopts the center value of the second angle range AC2R (i.e., the average value of the lower limit AC2L and the upper limit AC2H) as the second specific angle AC2Z.
[0153] S170c is the same as S170c in Fig. 12. The processor 210 corrects the angle of each corrected block image so that the second angle AC2 becomes the second specific angle AC2Z.
[0154] In S172d, the processor 210 generates data of a plurality of training block images by data augmentation. In this embodiment, the processor 210 generates data of a plurality of training block images by performing one or both of a rotation and a shear transformation on the corrected block image generated in S170c.
[0155] Here, the processor 210 generates data of the multiple training block images so that the multiple first angles AC1 of the multiple training block images finally obtained are approximately uniformly distributed within the first angle range AC1R, and the multiple second angles AC2 are approximately uniformly distributed within the second angle range AC2R. For example, the processor 210 randomly determines the angle of rotation of each of the multiple training block images so that the multiple angles (corresponding to the multiple first angles AC1) obtained from the first specific angle AC1Z by the rotation are approximately uniformly distributed within the first angle range AC1R. The processor 210 also randomly determines the amount of change in angle by the shear transformation of each of the multiple training block images so that the multiple angles (corresponding to the multiple second angles AC2) obtained from the second specific angle AC2Z by the shear transformation are approximately uniformly distributed within the second angle range AC2R.
[0156] As described above, in S170c, data of a plurality of corrected block images is generated. The processor 210 uses the plurality of corrected block images to generate data of a plurality of training block images. When a missing portion (e.g., the portion P5c in FIG. 14(J)) occurs due to the rotation or shear transformation, the processor 210 may use the original captured image (S110) to fill in the missing portion.
[0157] Also, the processor 210 may acquire data of a plurality of captured images in S110 in order to generate N training block images, similarly to the embodiment of FIG.
[0158] S290 is the same as S290 in Fig. 3. The processor 210 trains the anomaly detection model 310 (here, the Gaussian matrix GM) using the N training block images. Then, the processor 210 ends the processing in Fig. 17.
[0159] The inspection process of this embodiment is the same as that of the embodiment of FIG. 11. Thus, in this embodiment, the processor 210 executes the following processes to determine the presence or absence of defects in the target textile 700. In S720 and S730 (FIG. 11), the processor 210 acquires data of block images representing the captured images of the target textile 700. In S890b, the processor 210 inputs the block images to the anomaly detection model 310 to determine the presence or absence of defects in the target fabric. As described in FIG. 17, the anomaly detection model 310 is trained using a plurality of training block images representing the training textile 700t. The data of the plurality of training block images is generated by data augmentation in S172d.
[0160] In this embodiment, the first angle AC1 of the training block image is changed by data augmentation. As described in FIG. 14(A) and FIG. 14(E), the first angle AC1 indicates the direction in which the multiple first patterns PTc1 are regularly arranged in the image of the textile 700 (i.e., the first direction Dc1 in which the line PTLc1 formed by the multiple first patterns PTc1 extends). The first pattern PTc1 is formed by the yarns Wf and Wp. As described in S172d, the multiple training block images are configured such that the distribution range of the first angles AC1 of the multiple first directions Dc1 of the multiple training block images with respect to the reference direction (here, the +Dx direction) is the first angle range AC1R. As described in S172d and FIG. 18(A), the first angle range AC1R is determined using the distribution of the first angles AC1 estimated using the captured image of the training textile 700t. Here, the training fabric 700t is supported by a conveying device 900 that conveys the target fabric 700 for photographing the target fabric 700. Also, a first angle AC1 indicates an angle of a first direction Dc1 based on the rotation of the fabric 700 (e.g., the training fabric 700t).
[0161] According to this configuration, a plurality of training block images are configured such that the distribution range of a plurality of first angles AC1 of the plurality of training block images is a first angle range AC1R, and the first angle range AC1R is determined using the distribution of the first angle AC1 estimated using the photographed image. Therefore, even if the first angle AC1 indicated by the target textile 700 represented by the photographed image changes, the processor 210 can appropriately determine the presence or absence of a defect in the target textile 700 by using the anomaly detection model 310. In addition, the use of a first angle range AC1R that is wider than the width of the actual distribution of the first angle AC1 can be easily avoided. As a result, the burden of training the anomaly detection model 310 can be reduced.
[0162] In this embodiment, the second angle AC2 of the training block image is changed by data augmentation. As described in FIG. 14(F) and FIG. 14(J), the second angle AC2 indicates the direction in which the multiple second patterns PTc2 are regularly arranged in the image of the textile 700 (i.e., the second direction Dc2 in which the line PTLc2 formed by the multiple second patterns PTc2 extends). The second pattern PTc2 is formed by the yarns Wf and Wp. As described in S172d, the multiple training block images are configured such that the distribution range of the second angles AC2 of the multiple second directions Dc2 of the multiple training block images with respect to the reference direction (here, the -Dy direction) is the second angle range AC2R. As described in S172d and FIG. 18(B), the second angle range AC2R is determined using the distribution of the second angles AC2 estimated using the captured image of the training textile 700t. Here, the training fabric 700t is supported by a conveying device 900 that conveys the target fabric 700 for photographing the target fabric 700. Also, a second angle AC2 indicates the angle of a second direction Dc2 based on the skew of the fabric 700 (e.g., the training fabric 700t).
[0163] According to this configuration, a plurality of training block images are configured such that the distribution range of the plurality of second angles AC2 of the plurality of training block images is the second angle range AC2R, and the second angle range AC2R is determined using the distribution of the second angle AC2 estimated using the photographed image. Therefore, even if the second angle AC2 indicated by the target textile 700 represented by the photographed image changes, the processor 210 can appropriately determine the presence or absence of a defect in the target textile 700 by using the anomaly detection model 310. In addition, the use of a second angle range AC2R wider than the width of the actual distribution of the second angle AC2 can be easily avoided. As a result, the burden of training the anomaly detection model 310 can be reduced.
[0164] E. Fifth Example: E1. Training process: FIG. 19 is a flowchart showing an example of the training process of the fifth embodiment. In this embodiment, the processor 210 trains the distortion correction model 320 in addition to the anomaly detection model 310. The textile 700 is soft and may be locally deformed. If the captured image shows the textile 700 that is locally deformed, the deformation may reduce the accuracy of the inspection using the anomaly detection model 310. Therefore, in this embodiment, the processor 210 uses the distortion correction model 320 in the inspection process to correct the local deformation (i.e., distortion) of the textile 700 in the captured image. In the training process, the distortion correction model 320 is trained in addition to the anomaly detection model 310.
[0165] The operator attaches the training fabric 700t, which is free of abnormalities and has small distortion, to the conveying device 900. The operator may straighten the distortion of the training fabric 700t (also called straightening) before attaching the training fabric 700t to the conveying device 900. The operator may also straighten the distortion of the training fabric 700t after attaching the training fabric 700t to the conveying device 900. In either case, small distortions that have an acceptable effect on the inspection may remain. Then, the operator operates the operation unit 250 of the data processing device 200 (FIG. 1) to input an instruction to start the training process. In response to the instruction, the processor 210 starts the training process according to the training program 232.
[0166] In S110e, the processor 210 provides a shooting instruction to the digital camera 110. The digital camera 110 shoots the training fabric 700t. The processor 210 acquires shot image data from the digital camera 110.
[0167] S130 and S290 are the same as S130 and S290 in Fig. 3, respectively. In this way, the anomaly detection model 310 is trained. In this embodiment, distortions including rotation and skew can be corrected by the distortion correction model 320. Therefore, the corrections of rotation and skew are omitted.
[0168] In S310e, the processor 210 pastes a defect image onto a portion of the block image obtained in S130 (the block image generated by pasting a defect image is called a first type image). Figures 20(A) to 20(E) are diagrams showing information processed in the training process. Figure 20(A) shows an example of a block image. This block image Beo represents a woven fabric 700t that is free of abnormalities and has little distortion. Figure 20(B) shows an example of a first type image. Data for the first type image Be1 is generated by pasting an image of a defect De onto the block image Beo. A grid is shown by dotted lines on the first type image Be1. A plurality of grid points CP are used in the distortion adding process described later.
[0169] Although not shown in the figure, in this embodiment, data of a plurality of defect images representing defects different from each other is stored in advance in the non-volatile storage device 230. In S310e, the processor 210 executes the following processes. The processor 210 randomly selects a defect image from a plurality of available defect images. The processor 210 randomly selects a block image to be processed from a plurality of block images. The processor 210 randomly determines a position within the selected block image. Then, the processor 210 generates data of a first type image by pasting the selected defect image at the determined position.
[0170] In S320e (FIG. 19), the processor 210 randomly generates a displacement vector field (for example, using random numbers). FIG. 20(C) shows an example of the displacement vector field. In this embodiment, the displacement vector field VFe represents the displacement vectors Ve of each of a plurality of grid points CP (also called control points CP) on the first type image Be1. In this embodiment, the plurality of control points CP are arranged in a grid pattern along the first direction Dx and the second direction Dy. The processor 210 randomly determines the plurality of displacement vectors Ve of the plurality of control points CP. The positions of each of the plurality of control points CP on the first type image Be1 (and thus the block image) are determined in advance. In addition, the density of the control points CP (for example, the number of control points CP per unit area) is experimentally determined in advance so that the displacement vector field VFe appropriately represents the distortion that may actually occur in the target fabric 700.
[0171] If multiple displacement vectors Ve are determined independently and randomly, the displacement vector field VFe may represent unrealistic distortion for the woven fabric 700. For example, the difference between the two displacement vectors Ve of two adjacent control points CP may be excessively large. Therefore, in this embodiment, the processor 210 adjusts the displacement vectors Ve so that the difference between the two displacement vectors Ve of two adjacent control points CP is equal to or smaller than a reference value. As the control points CP adjacent to the control point CP of interest, so-called four neighboring control points CP may be adopted.
[0172] Fig. 20(D) shows the difference in displacement vector Ve. In the figure, two adjacent control points CPa, CPb and the displacement vectors Vea, Veb of the control points CPa, CPb are shown. The difference vector dVe is calculated by Vea-Veb. The processor 210 determines multiple displacement vectors Ve so that the following displacement conditions are satisfied. (Displacement condition) The length of each difference vector dVe of all combinations of two adjacent control points CP is equal to or less than a reference THe.
[0173] The method of determining the multiple displacement vectors Ve may be various methods. For example, the processor 210 first randomly determines each of the multiple displacement vectors Ve of the multiple control points CP. The length of each displacement vector Ve is limited to an upper limit value or less. The upper limit value is experimentally determined in advance so that the upper limit value represents a displacement (particularly, a maximum displacement) that can be formed by the actual distortion of the target fabric 700. Next, the processor 210 selects a pair of two adjacent control points CP to be processed (referred to as a target pair). When the length of the difference vector dVe of the target pair exceeds a reference THe, the processor 210 adjusts one displacement vector Ve of one control point CP of the target pair so that the length of the difference vector dVe is equal to or less than the reference THe. The adjustment of the displacement vector Ve may include one or both of changing the direction of the displacement vector Ve and changing the length of the displacement vector Ve. The processor 210 adjusts the displacement vector Ve while changing the target pair until the displacement condition is satisfied.
[0174] Alternatively, the processor 210 may execute the following process. The processor 210 selects a plurality of control points CP from the plurality of control points CP in a low density (i.e., roughly) manner (referred to as target control points CP). For example, the processor 210 may select one control point CP at intervals of 10 control points CP along the first direction Dx, and one control point CP at intervals of 10 control points CP along the second direction Dy. The processor 210 randomly determines a provisional displacement vector for each of the selected plurality of target control points CP. The length of the provisional displacement vector is limited to an upper limit value or less. The processor 210 determines a plurality of displacement vectors Ve of the plurality of control points CP by interpolation using the plurality of provisional displacement vectors (e.g., interpolation by cubic B-spline). Here, the components in the first direction Dx and the components in the second direction Dy are respectively interpolated. The possibility of generating a difference vector dVe having an excessively large length is reduced by the interpolation using multiple tentative displacement vectors of multiple roughly selected target control points CP. The processor 210 may repeat the adjustment of the upper limit of the length of the tentative displacement vector, the determination of the tentative displacement vector, and the interpolation until the displacement condition is satisfied.
[0175] In S330e (FIG. 19), the processor 210 applies distortion to the first type image using the displacement vector field VFe (the image generated by applying distortion is called the second type image). FIG. 20(E) shows an example of the second type image. Data of the second type image Be2 is generated by applying distortion to the first type image Be1 (FIG. 20(B)). The second type image Be2 generated in this manner represents the same defect De as the defect De in the first type image Be1 (FIG. 20(B)) (however, the defect De may be distorted on the second type image Be2). A control point CP moved according to the displacement vector field VFe is shown on the second type image Be2.
[0176] The method of applying distortion using the displacement vector field VFe may be various. The processor 210 may determine the color value of each pixel of the second type image Be2 by interpolation (e.g., interpolation by cubic B-spline) using a plurality of displacement vectors Ve represented by the displacement vector field VFe. For example, the processor 210 may displace each of a plurality of pixels of the first type image Be1 according to the displacement vector field VFe. Then, the processor 210 may calculate the color value of a pixel of interest of the second type image Be2 by interpolation using a plurality of displaced pixels located in the vicinity of the pixel of interest.
[0177] In this embodiment, the processor 210 finally generates data of M pairs (M is an integer equal to or greater than 2) of pairs of a first type image and a second type image (called training pairs). The number M of training pairs may be any number suitable for training the distortion correction model 320 (for example, about several hundred or several thousand). The processor 210 may execute the processes of S310e-S330e on M block images to generate M sets of training pairs. Furthermore, the processor 210 may acquire data of a plurality of captured images in S110e to acquire M block images. Then, in S130, the processor 210 may acquire a plurality of block images from each of the plurality of captured images.
[0178] In S380e (FIG. 19), the processor 210 trains the distortion correction model 320 using the M training pairs. FIG. 21 is a block diagram showing an example of the distortion correction model 320. In this embodiment, the distortion correction model 320 is configured using a distortion correction technique called DIRNet (End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network). DIRNet is disclosed in, for example, the paper "Bob D. de Vos, Floris F. Berendsen, Max A. Viergever, Marius Staring, Ivana Isgum, "End-to-end unsupervised deformable image registration with a convolutional neural network", arXiv:1704.06065(2017), https: / / arxiv.org / abs / 1704.06065".
[0179] The DIRNet technology generates a displacement vector field DVF for warping (i.e., transforming) the moving image E2 to the fixed image E1. This displacement vector field DVF corrects the distortion of the moving image E2 relative to the fixed image E1. In the training process of this embodiment, a second type of image (e.g., a second type of image Be2) is used as the moving image E2, and a first type of image (e.g., a first type of image Be1) is used as the fixed image E1.
[0180] The distortion correction model 320 includes a convolution net regressor 320a, a spatial transformer 320b, and a resampler 320c.
[0181] The convolution net regressor 320a includes multiple convolution networks (not shown). The convolution net regressor 320a receives data of the fixed image E1 and the moved image E2, which are concatenated. The convolution net regressor 320a analyzes pairs of patches EP1, EP2, which are spatially corresponding (i.e., at the same location), where the pair is a patch EP1 from the fixed image E1 and a patch EP2 from the moved image E2. The convolution net regressor 320a is trained to output two-dimensional displacements of control points at locations corresponding to the pair of patches EP1, EP2. The convolution net regressor 320a outputs the displacements of each of the multiple control points of the multiple pairs of patches EP1, EP2. The set 320ao of multiple displacements of the multiple control points is used by the spatial transformer 320b.
[0182] The spatial transformer 320b uses the displacement set 320ao to generate a displacement vector field DVF that represents dense displacements (e.g., displacements of all pixels) that allow the resampler 320c to warp the image. The displacement vector field DVF may be generated in various ways. In this embodiment, the spatial transformer 320b generates the displacement vector field DVF by cubic B-spline interpolation.
[0183] The resampler 320c generates data of the corrected image E3 by warping the moved image E2 according to the displacement vector field DVF. The image warping method using the displacement vector field DVF may be various methods. For example, the processor 210 may displace each of a plurality of pixels of the moved image E2 according to the displacement vector field DVF. Then, the processor 210 may calculate the color value of a pixel of interest in the corrected image E3 by interpolation using a plurality of displaced pixels located in the vicinity of the pixel of interest.
[0184] The training method of the distortion correction model 320 may be any method suitable for the distortion correction model 320, for example, the method described in the above-mentioned paper on DIRNet. The processor 210 may train the distortion correction model 320, for example, as follows. The processor 210 obtains a subset consisting of a plurality of training pairs from M sets of training pairs of a first type image and a second type image. The processor 210 generates data of the corrected image E3 by executing the operation of the distortion correction model 320 using data of the training pairs included in the subset. The processor 210 calculates the image similarity SM using data of the first type image (i.e., the fixed image E1) and data of the corrected image E3. The image similarity SM may be various values indicating the similarity between the two images E1 and E3. For example, the image similarity SM may be a normalized cross correlation.
[0185] The processor 210 calculates a plurality of image similarities SM using a plurality of training pairs of the subset. The processor 210 calculates a loss using the plurality of image similarities SM. The loss may be various values that are larger as the image similarities SM are smaller. For example, the loss may be a total value of the difference between the maximum value of the image similarities SM (here, 1) and the image similarities SM.
[0186] The processor 210 adjusts a number of calculation parameters (convolution filter, bias, etc.) of the convolution net regressor 320a so as to reduce the loss. For example, the backpropagation method and the gradient descent method may be used for the adjustment.
[0187] After the adjustment, the processor 210 determines whether a training end condition is satisfied. The training end condition may be any condition that indicates that the distortion correction model 320 has been properly trained. For example, the end condition may be that the loss calculated using a predetermined number of training pairs not used in training is equal to or less than a predetermined loss threshold. Alternatively, the training end condition may be that an end instruction is input from an operator. The operator may input an end instruction or a continuation instruction to the operation unit 250 by comparing the corrected image E3 obtained from the training pair not used in training with the fixed image E1.
[0188] The processor 210 repeats the adjustment using the new subset until the training end condition is satisfied. When the training end condition is satisfied, the processor 210 stores the trained data of the distortion correction model 320 in the storage device 215 (here, the non-volatile storage device 230). This ends the training of the distortion correction model 320, i.e., S380e (FIG. 19). Then, the processing of FIG. 19 ends.
[0189] E2. Inspection process: Fig. 22 is a flowchart showing an example of the inspection process of Example 5. The only difference from the inspection process of Fig. 7 is that distortion correction (S810e) is performed instead of the correction of the first angle AA1 (S740, S750).
[0190] S710, S720, and S730 are the same as S710, S720, and S730, respectively, in Fig. 7. The processor 210 divides the captured image into a plurality of block images.
[0191] In S810e, the processor 210 corrects the distortion of each of the multiple block images using the distortion correction model 320 (FIG. 21). In this embodiment, data of a reference block image representing the undistorted textile 700 is stored in advance in the non-volatile storage device 230 (not shown). The processor 210 uses a pair of a block image and a reference block image to generate data of a corrected image E3. The block image is used as a moving image E2, and the reference block image is used as a fixed image E1. The distortion correction model 320 generates data of the corrected image E3 by correcting the distortion of the block image relative to the reference block image. If the moving image E2 (block image) represents a defect, the corrected image E3 represents the same defect. If the moving image E2 (block image) does not represent a defect, the corrected image E3 does not represent a defect.
[0192] S890 is the same as S890 in Fig. 7. The processor 210 inputs each of the multiple corrected images obtained in S810e to the anomaly detection model 310, and determines whether the target textile 700 has an anomaly. S894, S896, and S898 after S890 are the same as S894, S896, and S898 in Fig. 7, respectively. If the determination result of S896 is No, the processor 210 proceeds to S710.
[0193] As described above, in this embodiment, the processor 210 executes the following processes to determine whether or not the target textile 700 has a defect. In S720 and S730 (FIGS. 22 and 11), the processor 210 acquires data of a block image representing a captured image of the target textile 700. In S810e, the processor 210 performs a correction process for the block image. In S890, the processor 210 inputs the image processed by the correction process to the anomaly detection model 310 to determine whether or not the target textile 700 has a defect. Here, the correction process (S810e) includes a distortion correction for correcting distortion of the target textile 700 in the image representing the target textile 700, as described in FIG. 21. Therefore, even if the target textile 700 in the captured image has a distortion, the processor 210 can appropriately determine whether or not the target textile 700 has a defect by using the anomaly detection model 310.
[0194] In this embodiment, the distortion correction (S810e) includes the correction of distortion by the distortion correction model 320 (FIG. 21), which is a machine learning model that corrects distortion. As described in S330e, S380e of FIG. 19, and FIG. 21, the distortion correction model 320 is trained to generate a corrected image E3 having a high image similarity SM with the first type image Be1 from a second type image Be2 obtained by applying distortion to the first type image Be1 representing the training textile 700t. That is, the distortion correction model 320 is trained to generate the first type image Be1 from the second type image Be2. Therefore, the distortion correction model 320 can appropriately correct the distortion of the image.
[0195] In this embodiment, as described in S310e of Fig. 19 and Fig. 20(B), the first type image Be1 is obtained by adding an image of a defect (e.g., an image of a defect De) to a portion of the block image representing the training fabric 700t. As described in Fig. 20(E), the second type image Be2 represents the same defect De as the defect De represented by the first type image Be1. Therefore, the distortion correction model 320 can appropriately correct the distortion of the captured image for inspection even when the captured image represents the target fabric 700 having a defect.
[0196] In this embodiment, the distortion correction model 320 does not directly generate the corrected image E3, but generates the corrected image E3 by deforming the moved image E2. Therefore, if the moved image E2 does not represent a defect, a corrected image E3 that does not represent the defect is generated. If the moved image E2 represents a defect, a corrected image E3 that represents the same defect is generated. Therefore, adding an image of the defect to the first type image Be1 for training (S310e) may be omitted.
[0197] In this embodiment, as described in S320e and S330e of FIG. 19, the distortion is applied according to each displacement vector Ve of the plurality of control points CP arranged in the first type image Be1 (FIG. 20(C)). Then, as described in FIG. 20(D), the plurality of displacement vectors Ve of the plurality of target control points CP are configured so that the difference between the displacement vectors Ve of the adjacent control points CP is equal to or smaller than a reference. In this embodiment, the plurality of displacement vectors Ve are configured so that the length of the difference vector dVe between two adjacent control points CP is equal to or smaller than a reference THe. Therefore, the processor 210 can apply the distortion that may actually occur in the textile 700 to the first type image Be1. As a result, the distortion correction model 320 is trained to correct the distortion that may actually occur.
[0198] F. Sixth Example: F1. Training process: Fig. 23 is a flowchart showing an example of the training process of the sixth embodiment. In this embodiment, the distortion correction model 320 is omitted. The processor 210 corrects the distortion of the fabric 700 in the captured image in the inspection process described below without using the distortion correction model 320. The training process of this embodiment is the same as the training process of Fig. 19 except that S310e-S380e for the distortion correction model 320 are omitted. The processor 210 trains the anomaly detection model 310 by S110e, S130, and S290.
[0199] F2.Inspection process: Fig. 24 is a flowchart showing an example of the inspection process of the sixth embodiment. The only difference from the inspection process of Fig. 22 is that the distortion correction process by the distortion correction model 320 (S810e) is replaced with distortion correction process by keypoint matching (S810f).
[0200] S710, S720, and S730 are the same as S710, S720, and S730, respectively, in Fig. 22. The processor 210 divides the captured image into a plurality of block images.
[0201] In S810f, the processor 210 corrects the distortion of each of the multiple block images using keypoint matching. The distortion correction using keypoint matching may be various processes. In this embodiment, S810f includes S812f, S814f, and S816f.
[0202] In S812f, the processor 210 forms a plurality of matching pairs between the block image and the reference block image by keypoint matching. FIG. 25 is a diagram showing an example of distortion correction by keypoint matching. In the diagram, a block image Bf1 and a reference block image Bfs are shown. The block image Bf1 is a block image generated in S730 (FIG. 24) and represents the target textile 700. In the block image Bf1, the target textile 700 may have distortion. The block image Bf1 may also represent a defect (e.g., defect Df). The reference block image Bfs is an image representing the textile 700 without a defect and with small distortion (e.g., a photographed image of the textile 700). In this embodiment, data of the reference block image Bfs is stored in the non-volatile storage device 230 in advance (not shown). The reference block image Bfs may include a small distortion that has an acceptable effect on the inspection.
[0203] The processor 210 detects key points from each of the block image Bf1 and the reference block image Bfs. A key point is a point that is expected to be highly likely to be unique in an image. A plurality of lines RL, RLz are shown in the figure. Both ends of these lines RL, RLz indicate key points. Hereinafter, the key point K1 detected from the block image Bf1 is called the photographed key point K1. Also, the key point Ks detected from the reference block image Bfs is called the reference key point Ks. In this embodiment, points representing characteristic parts such as corners and edges of a pattern formed by the yarns Wf, Wp of the woven fabric 700 (FIG. 2) are detected as the key points K1, Ks. Although not shown in the figure, in reality, many more key points K1, Ks are detected (for example, about tens or hundreds).
[0204] The keypoint detection method may be various methods. For example, the detection method may be selected in advance from a search for extreme values (maximum and minimum values) using DoG (Difference-of-Gaussian), Harris corner detection, and FAST (Features from Accelerated Segment Test) corner detection. For the detection of keypoints, an algorithm for detecting keypoints and calculating feature descriptors may be adopted. The feature descriptor is information describing the features of the keypoints. The algorithm for detecting keypoints and calculating feature descriptors may be selected in advance from, for example, SIFT (Scale Invariant Feature Transform), SURF (Speed Up Robust Features), and ORB (Oriented FAST and Rotated BRIEF). In this embodiment, the processor 210 detects keypoints according to the ORB algorithm. For example, the function "detect" or the function "detectAndCompute" of OpenCV (Open Source Computer Vision Library) can be used to detect keypoints using the ORB algorithm.
[0205] The processor 210 performs matching between the captured keypoint K1 and the reference keypoint Ks. Through keypoint matching, keypoints K1 and Ks that indicate the same part of the same subject (for example, the same corner of a pattern formed by threads Wf and Wp) are associated with each other. A pair of keypoints K1 and Ks that are associated with each other is also called a matching pair.
[0206] A plurality of lines RL in the figure respectively indicate suitable matching pairs K1, Ks. For suitable matching pairs K1, Ks, the position of the photographed keypoint K1 in the block image Bf1 is close to the position of the reference keypoint Ks in the reference block image Bfs. A plurality of lines RLz indicate inappropriate matching pairs K1, Ks. For inappropriate matching pairs K1, Ks, the position of the photographed keypoint K1 in the block image Bf1 is far from the position of the reference keypoint Ks in the reference block image Bfs.
[0207] The method of keypoint matching may be various methods. For example, the processor 210 may perform keypoint matching using a feature descriptor of the keypoint. The feature descriptor may be various information describing the features of the keypoint. The feature descriptor is calculated so as to change according to the distribution of color values of multiple pixels around the keypoint, for example. The method of calculating the feature descriptor may be various methods. The algorithm for calculating the feature descriptor may be selected in advance from, for example, BRIEF (Binary Robust Independent Elementary Features), BRISK (Binary Robust Invariant Scalable Keypoints), SIFT, SURF, and ORB. In this embodiment, the processor 210 uses a feature descriptor based on the ORB algorithm.
[0208] The processor 210 calculates the feature descriptor of each captured key point K1 by analyzing the data of the block image Bf1. For example, the OpenCV function "compute" can be used to calculate the feature descriptor based on the ORB algorithm. The processor 210 may detect multiple key points K1 and calculate the feature descriptor of each key point K1 by executing the OpenCV function "detectAndCompute".
[0209] In addition, the feature descriptor of each reference keypoint Ks is calculated in advance. In this embodiment, data representing the multiple reference keypoints Ks and the feature descriptor of each reference keypoint Ks is stored in advance in the non-volatile storage device 230 (not shown).
[0210] The method of keypoint matching using the feature descriptor may be various methods. For example, the matching method may be selected in advance from FLANN (Fast Library for Approximate Nearest Neighbor) based matching and brute force matching. For FLANN based matching, for example, matching by "cv2.FlannBasedMatcher" of OpenCV can be used. For brute force matching, for example, matching by "cv2.BFMatcher" of OpenCV can be used. In this embodiment, the processor 210 performs brute force matching.
[0211] In this embodiment, as described in FIG. 2, the woven fabric 700 is configured so that the same patterns formed by the threads Wf and Wp are regularly arranged. Therefore, keypoint pairs K1 and Ks that are significantly different in position in the images Bf1 and Bfs and represent similar parts of the patterns can be associated with each other (for example, a matching pair represented by a line RLz). Various methods can be adopted as a method for reducing the possibility of forming such an inappropriate matching pair. For example, the processor 210 may calculate a homography matrix using multiple matching pairs and remove matching pairs classified as outliers based on the homography matrix. For example, the OpenCV function "cv2.findHomography" can be used for this process. In this function, an algorithm called RANSAC or LEAST_MEDIAN can be used. Either algorithm classifies multiple matching pairs into good pairs (inliers) and outliers. In FIG. 25, the appropriate matching pair K1 and Ks represented by the line RL is likely to be classified as a good pair. The poor matching pairs K1, Ks as represented by the line RLz are likely to be classified as outliers. The processor 210 may adopt the multiple keypoint pairs classified as good pairs as the final matching pairs.
[0212] Also, the method of keypoint matching may be the following method. In this embodiment, the distortion of the textile 700 is caused by a local deformation of the textile 700. Therefore, there is a limit to the displacement (for example, the displacement of the photographed keypoint K1) that can be formed by the actual distortion of the target textile 700. Therefore, the processor 210 may perform keypoint matching (for example, keypoint matching based on a feature descriptor) using the target photographed keypoint K1 and a reference keypoint Ks included in a search area including the target photographed keypoint K1. The search area may be, for example, a circular area centered on the position of the target photographed keypoint K1. The radius is experimentally determined in advance so that the radius represents the displacement (particularly, the maximum displacement) that can be formed by the actual distortion of the target textile 700. The processor 210 can form an appropriate matching pair by performing such keypoint matching for a plurality of photographed keypoints K1. After this keypoint matching, the processor 210 may remove matching pairs that are classified as outliers based on the homography matrix.
[0213] In S814f (FIG. 24), the processor 210 generates a displacement vector field using multiple matching pairs. FIG. 25 shows an example of the displacement vector field VFf. In this embodiment, the processor 210 adopts the photographic key point K1 of the matching pair K1, Ks as the control point CPf. The processor 210 adopts the vector from the coordinate of the photographic key point K1 of the matching pair K1, Ks to the coordinate of the reference key point Ks as the displacement vector Vf. As shown in the figure, the displacement vector field VFf represents the respective displacement vectors Vf of the multiple control points CPf arranged on the block image Bf1.
[0214] In S816f (FIG. 24), the processor 210 corrects the distortion of the block image Bf1 using the displacement vector field VFf. The correction method is the same as the distortion applying method using the displacement vector field VFe in S330e (FIG. 19). Applying the distortion based on the displacement vector field VFf reduces the distortion contained in the block image Bf1. FIG. 25 shows an example of a corrected block image Bf2. In the corrected block image Bf2, the distortion is reduced compared to that in the block image Bf1. If the block image Bf1 represents a defect (e.g., defect Df), the corrected block image Bf2 represents the same defect (however, the defect may be deformed from the defect on the block image Bf1).
[0215] S890 (FIG. 24) is the same as S890 in FIG. 22. The processor 210 inputs each of the multiple corrected block images obtained in S810f into the anomaly detection model 310, and determines whether the target textile 700 has an anomaly. S894, S896, and S898 after S890 are the same as S894, S896, and S898 in FIG. 7, respectively. If the determination result of S896 is No, the processor 210 proceeds to S710.
[0216] As described above, in this embodiment, the distortion correction includes the correction of distortion using the result of keypoint matching using the reference image (here, the reference block image Bfs). Therefore, even if the target textile 700 in the captured image has distortion, the processor 210 can appropriately determine the presence or absence of a defect in the target textile 700 by using the anomaly detection model 310.
[0217] G. Seventh Example: G1. Training process: Fig. 26 is a flowchart showing an example of a training process of the seventh embodiment. The difference from the training process of Fig. 19 is that distortion is imparted to a training block image for training the anomaly detection model 310. In this embodiment, the distortion correction model 320 is omitted.
[0218] S110e and S130 are the same as S110e and S130, respectively, in Fig. 19. The processor 210 acquires data of a plurality of block images representing a normal and less distorted training fabric 700t.
[0219] S310g is the same as S310e in Fig. 19. The processor 210 pastes the defective image into a portion of the block image obtained in S130. S310g is executed when a training image having a defect is used to train the anomaly detection model 310. In this embodiment, S310g is omitted.
[0220] S320g and S330g are the same as S320e and S330e in FIG. 19, respectively. The processor 210 randomly generates a displacement vector field and applies distortion to the block image using the displacement vector field. In this way, the processor 210 generates data of a training block image having distortion. The processor 210 may execute the processes of S320g and S330g for N block images in order to generate N training block images. In addition, the processor 210 may acquire data of a plurality of captured images in S110e in order to acquire N block images. Then, in S130, the processor 210 may acquire a plurality of block images from each of the plurality of captured images.
[0221] In S340g, the processor 210 uses the N training block images to train the anomaly detection model 310. The training method is the same as the training method in S290 in Fig. 19 etc. After S340g, the processor 210 ends the processing in Fig. 26.
[0222] The inspection process of this embodiment is the same as the inspection process of Fig. 11. Thus, in this embodiment, the anomaly detection model 310 is trained using a plurality of training block images having distortion. Therefore, even if the target textile 700 represented by the captured image has distortion, the processor 210 can appropriately determine the presence or absence of a defect in the target textile 700 by using the anomaly detection model 310.
[0223] H. Eighth Example: H1. Training process: Fig. 27 is a flowchart showing an example of the training process of the eighth embodiment. The training process of this embodiment includes S110e and S130 in Fig. 19, process SC1 of S140c-S290 in Fig. 12, and process SE1 of S310e-S380e in Fig. 19. In this way, the training method of the anomaly detection model 310 is the same as the training method of the embodiment in Fig. 12. The training method of the distortion correction model 320 is the same as the training method of the embodiment in Fig. 19.
[0224] H2. Inspection process: FIG. 28 is a flowchart showing an example of the inspection process of the eighth embodiment. The inspection process of this embodiment includes S710, S720, and S730 in FIG. 16, the process SC2 of S740c-S770c, and the process SE2 of S810e-S890 in FIG. 22. In the process SE2, the block image processed by the process SC2 is processed. In this manner, the processor 210 corrects the first angle AC1 (S750c), the second angle AC2 (S770c), and the distortion (S810e) for the block image representing the captured image of the target textile 700. In S890, the processor 210 inputs each of the multiple corrected block images generated by the processes SC2 and SE2 to the anomaly detection model 310 and judges whether the target textile 700 has an anomaly.
[0225] As described above, in this embodiment, the processor 210 executes the following process to determine whether or not the target textile 700 has a defect. In S720 and S730 (FIG. 28), the processor 210 acquires data of a block image representing a captured image of the target textile 700. In S750c, S770c, and S810e, the processor 210 performs a correction process for the block image. In S890, the processor 210 inputs the image processed by the correction process to the anomaly detection model 310 to determine whether or not the target textile 700 has a defect. Here, the correction process (S750c, S770c, and S810e) includes a distortion correction (S810e) that corrects distortion of the target textile 700 in the image representing the target textile 700. Therefore, even if the target textile 700 in the captured image has a distortion, the processor 210 can appropriately determine whether or not the target textile 700 has a defect by using the anomaly detection model 310.
[0226] In this embodiment, the correction process (S750c, S770c, S810e) further includes angle correction (S750c, S770c). As shown in FIG. 14(E), the correction of S750c is a correction of the first angle AC1 by image rotation (a type of affine transformation) (called the first correction). Specifically, the first correction brings the first angle AC1 of the first direction Dc1 with respect to the reference direction Dx closer to the first specific angle AC1Z by the rotation of the affine transformation. As described in FIG. 14(A), the first angle AC1 indicates the first direction Dc1 in which a plurality of patterns PTc1 in the image representing the target textile 700 are regularly arranged. The first pattern PTc1 is formed by the yarns Wf and Wp. As shown in FIG. 14(J), the correction of S770c is a correction of the second angle AC2 by image shear transformation. Specifically, the second correction brings the second angle AC2 of the second direction Dc2 relative to the second reference direction (here, the -Dy direction) closer to the second specific angle AC2Z by shearing of the affine transformation. As described in FIG. 14(F), the second angle AC2 indicates the second direction Dc2 in which the multiple second patterns PTc2 in the image representing the target textile 700 are regularly arranged. The second patterns PTc2 are formed by the yarns Wf and Wp. Then, the distortion correction in S810e corrects the distortion of the target textile 700 in the processed image by the angle correction (S750c, S770c).
[0227] According to this configuration, among the distortions of the target textile 700 in the block image (i.e., the captured image), the angular deviation is appropriately corrected by angle correction (here, S750c, S770c). The distortion correction in S810e is performed on the block image whose angle has been corrected. Therefore, the effect of the angular deviation on the distortion correction in S810e is mitigated. As described above, the processor 210 can appropriately determine the presence or absence of a defect in the target textile 700 by using the anomaly detection model 310.
[0228] In addition, when a machine learning model (e.g., the distortion correction model 320) is used for the distortion correction of S810e, the machine learning model can be trained without assuming the angular misalignment, and thus the burden of training the machine learning model can be reduced (e.g., multiple training images corresponding to different angles can be omitted).
[0229] Note that either the first correction (S750c) or the second correction (S770c) may be omitted from the inspection process (FIG. 28). The correction (S150c or S170c) corresponding to the omitted angle correction may be omitted from the training process (FIG. 27). Also, instead of the distortion correction of S810e, the distortion correction of S810f (FIG. 24) may be executed. In this case, the distortion correction model 320 may be omitted. Process SE1 may be omitted from the training process (FIG. 27).
[0230] I. Ninth Example: I1. Training process: FIG. 29 is a flowchart showing an example of the training process of the ninth embodiment. The training process of this embodiment includes S110e and S130 in FIG. 26, process SD1 of S140c-S172d in FIG. 17, and process SG1 of S310g-S340g in FIG. 26. In process SG1, the block image processed by process SD1 is processed. In this manner, the processor 210 trains the anomaly detection model 310 using a plurality of training block images having various first angles AC1, various second angles AC2, and various distortions. In this embodiment, the distortion correction model 320 is omitted.
[0231] The inspection process of this embodiment is the same as the inspection process of FIG. 11. Thus, in this embodiment, the processor 210 executes the following processes to determine the presence or absence of a defect in the target textile 700. In S720 and S730 (FIG. 11), the processor 210 acquires data of a block image representing a photographed image of the target textile 700. In S890b, the processor 210 inputs the block image to the anomaly detection model 310 to determine the presence or absence of a defect in the target fabric. As shown in S320g-S340g of FIG. 29, the anomaly detection model 310 is trained using a second type of image (here, a training block image) obtained by performing image processing SD1 and SG1 including the addition of distortion (S330g) on a first type of image (here, a block image obtained in S130) representing the training textile 700t. Therefore, even if the target textile 700 in the photographed image has distortion, the processor 210 can appropriately determine the presence or absence of a defect in the target textile 700 by using the anomaly detection model 310.
[0232] In this embodiment, the image processing SD1, SG1 for generating the training block image includes an angle changing process (S172d). The angle changing process (S172d) includes a first changing process for changing the first angle AC1 and a second changing process for changing the second angle AC2. The first changing process changes the first angle AC1 of the first direction Dc1 (FIG. 14(E)) of the image to be processed with respect to the first reference direction (here, +Dx direction) by affine transformation (here, rotation). The second changing process changes the second angle AC2 of the second direction Dc2 (FIG. 14(J)) of the image to be processed with respect to the second reference direction (here, -Dy direction) by shearing of the affine transformation. The distortion applying process (S330g) applies distortion to the image processed by the angle changing process (S172d). The training block image obtained by such image processing can appropriately represent the actual woven fabric 700. Therefore, the processor 210 can appropriately determine whether or not there is a defect in the target fabric 700 by using the anomaly detection model 310 trained using such training block images.
[0233] It should be noted that the first change process of the first angle AC1 or the second change process of the second angle AC2 may be omitted from the angle change process (S172d).
[0234] J. Tenth Example: The fabric to be inspected may be a knitted fabric instead of a woven fabric (for example, the woven fabric 700 (FIG. 2)). FIG. 30 is a diagram showing an example of a knitted pattern. The knitted fabric 700J is knitted using a yarn Wj. The knitted fabric 700J may be knitted in various ways. In any case, the knitted fabric 700J is configured so that the patterns associated with the knitting methods are regularly arranged. The captured image of the knitted fabric 700J generated by the digital camera 110 shows the fine pattern of the knitted fabric 700J. The method of specifying the direction in which the patterns are arranged may be various methods. For example, the gap PJ1 surrounded by the yarn Wj forms an island-like portion similar to the first portions Pa1 and Pc1 (FIG. 4(B) and FIG. 14(A)). Therefore, similar to each of the above embodiments, the processor 210 can estimate the direction in which the pattern composed of a plurality of gaps PJ1 is arranged (for example, the direction in which a plurality of patterns PTj1 are arranged) by using the binarized image. Also, as the pattern, the shape of the yarn Wj (e.g., a loop, a knot, etc.) may be used, such as the pattern PTj2. In this case, the processor 210 may detect a plurality of patterns PTj2 from the captured image by template matching using a template image of the pattern PTj2. Then, the processor 210 may estimate the direction in which the patterns PTj2 are arranged, using the detected plurality of patterns PTj2. In this manner, the direction of the fabric of the knitted fabric 700J, woven fabric 700, etc., represented by the captured image (i.e., the direction in which the patterns formed by the yarn are regularly arranged) may be determined by template matching.
[0235] K. Variations: (1) In the embodiment of FIG. 3 and FIG. 7, a plurality of training block images showing various second angles AC2 may be further used in training the anomaly detection model 310. For example, the processor 210 may execute S160c, S162d, S164d, S170c, and modified S172d in FIG. 17 using a plurality of training block images generated in S150 in FIG. 3. In the modified S172, the change of the first angle AC1 is omitted, and the processor 210 changes the second angle AC2. In S290, a plurality of training block images generated in the modified S172d are used. In this modified example, the plurality of training block images are configured such that the distribution range of the second angle AC2 with respect to the reference direction (here, the -Dy direction) of the plurality of second directions Dc2 (FIG. 14(J)) of the plurality of training block images is the second angle range AC2R. As described in FIG. 18B, the second angle range AC2R is determined using the distribution of the second angle AC2 estimated using the captured image of the training fabric 700t. Here, the training fabric 700t is supported by a conveying device 900 that conveys the target fabric 700 for capturing the image of the target fabric 700. The second angle AC2 indicates the angle of the second direction Dc2 based on the skew of the fabric 700 (e.g., the training fabric 700t). According to this configuration, as in the embodiment of FIG. 17 and FIG. 18, even if the second angle AC2 indicated by the target fabric 700 represented by the captured image changes, the processor 210 can appropriately determine the presence or absence of a defect in the target fabric 700 by using the anomaly detection model 310.
[0236] In the embodiments of Figs. 12 and 16, a plurality of training block images showing various second angles AC2 may be used for training the anomaly detection model 310. For example, the processor 210 may execute S160c, S162d, S164d, S170c, and modified S172d in Fig. 17 using a plurality of training block images generated in S170c in Fig. 12. In the modified S172, the change in the first angle AC1 is omitted, and the processor 210 changes the second angle AC2. In S290, a plurality of training block images generated in the modified S172d are used. In the inspection process of Fig. 16, S760c and S770c are omitted. In this case, as in the embodiments of Figures 17 and 18, even if the second angle AC2 indicated by the target textile 700 represented by the captured image changes, the processor 210 can appropriately determine the presence or absence of a defect in the target textile 700 by using the anomaly detection model 310.
[0237] (2) In the embodiment of FIG. 3 and FIG. 7, the processor 210 may determine the first specific angle AA1Z using a plurality of captured images of the textile 700. For example, the processor 210 may execute S142b and S144b of FIG. 9 between S140 and S150. In this modification, the first specific angle AA1Z is determined using a distribution of the first angle AA1 (FIG. 10) estimated using a block image (i.e., a captured image) of the training textile 700t. Here, the training textile 700t is supported by a conveying device 900 that conveys the target textile 700 for capturing the target textile 700. Also, the first angle AA1 indicates an angle based on the rotation of the textile 700 (e.g., the training textile 700t) as described in FIG. 4(F) and the like. According to this configuration, the difference between the first specific angle AA1Z and the first angle AA1 indicated by the block image used in the inspection process is reduced. Therefore, the correction amount (e.g., rotation angle) by the correction of the first angle AA1 (S150 (FIG. 3), S750 (FIG. 7)) is reduced. As a result, the negative impact of the correction of the first angle AA1 on the determination of the presence or absence of a defect is mitigated.
[0238] In the embodiment of FIG. 12 and FIG. 16, the processor 210 may determine one or both of the first specific angle AC1Z and the second specific angle AC2Z using a plurality of captured images of the textile 700. For example, the processor 210 may execute S142d and S144d of FIG. 17 between S140c and S150c of FIG. 12. The processor 210 may execute S162d and S164d of FIG. 17 between S160c and S170c of FIG. 12. When the first specific angle AC1Z is determined as described above, the amount of correction (e.g., rotation angle) by the correction of the first angle AC1 (S150c (FIG. 12), S750c (FIG. 16)) is reduced. Therefore, the possibility of erroneous detection of a defect due to the correction of the first angle AC1 can be reduced. When the second specific angle AC2Z is determined as described above, the correction amount (e.g., the amount of change in angle due to shear transformation) by the correction of the second angle AC2 (S170c (FIG. 12), S770c (FIG. 16)) is reduced, and therefore the negative impact of the correction of the second angle AC2 on the determination of the presence or absence of a defect is mitigated.
[0239] (3) In the above embodiment and modified example, the first specific angles AA1Z, AC1Z and their angle ranges AA1A, AA1R, AC1R are determined (e.g., S152 (FIG. 3), S142b, S144b (FIG. 9), S142d, S144d (FIG. 17)) using a distribution of a plurality of first angles AA1, AC1 obtained from a plurality of block images. As described in FIG. 4(D), FIG. 4(C), FIG. 14(A), and FIG. 14(C), a plurality of vectors Va, Vc1 indicating angles AA1v, AC1v corresponding to the first angles AA1, AC1 are obtained from one block image. Such angles AA1v, AC1v also indicate the directions Da, Dc1 of the fabric 700. The processor 210 may determine the first specific angles AA1Z, AC1Z and ranges AA1A, AA1R, AC1R using the distribution of the multiple angles AA1v, AC1v of the multiple vectors Va, Vc1 obtained from one image.
[0240] In the above embodiment and modified example, the second specific angle AC2Z and the second angle range AC2R are determined (for example, S162d, S164d (FIG. 17)) using the distribution of the second angles AC2 obtained from the multiple block images. As described in FIG. 14(F) and FIG. 14(H), multiple vectors Vc2 indicating an angle AC2v equivalent to the second angle AC2 are obtained from one block image. Such an angle AC2v also indicates the direction Dc2 of the textile 700. The processor 210 may determine the second specific angle AC2Z and the second angle range AC2R using the distribution of the multiple angles AC2v of the multiple vectors Vc2 obtained from one image.
[0241] (4) In the above embodiment and modified example, the first angles AA1 and AC1 may indicate angles of the first directions Da and Dc1 based on the skew of the fabric (e.g., the woven fabric 700 or the knitted fabric 700J). In this case, the correction of the first angles AA1 and AC1 (e.g., S150 (FIG. 3), S750 (FIG. 7), S150c (FIGS. 12, 17, 27), S750c (FIGS. 16, 28)) may be performed by shear transformation in a direction intersecting the first directions Da and Dc1 (e.g., the second direction Dy). The change of the first angles AA1 and AC1 by data augmentation (e.g., S154 (FIG. 3), S154b (FIGS. 9), S172d (FIGS. 17, 29)) may be performed by shear transformation.
[0242] In the above embodiment and modified example, the first angles AA1 and AC1 may be angles based on both the rotation and skew of the fabric. For example, the first angles AA1 and AC1 may be determined by the addition of the rotation and the shear transformation.
[0243] In the above embodiment and modified example, the second angle AC2 may indicate the angle of the second direction Dc2 based on the rotation of the fabric (e.g., the woven fabric 700 or the knitted fabric 700J). In this case, the correction of the second angle AC2 (e.g., S170c (FIGS. 12, 17, 27), S770c (FIGS. 16, 28)) may be performed by the rotation. The change of the second angle AC2 by data augmentation (e.g., S172d (FIGS. 17, 29)) may be performed by the rotation.
[0244] In the above embodiment and modified example, the second angle AC2 may be an angle based on both the rotation and the skew of the fabric. For example, the second angle AC2 may be determined by the addition of the rotation and the shear transformation.
[0245] (5) In the above embodiment and modified examples, the anomaly detection model may be various models for detecting anomalies, instead of the anomaly detection model 310 in FIG. 6. For example, the anomaly detection model may include, in addition to the model 310 in FIG. 6, an image generation model (e.g., an autoencoder, etc.) trained to generate an image of an object (here, fabric) without defects from an image of the same object having defects. In the inspection process, the processor 210 generates an intermediate block image, which is a block image without defects, by inputting a block image representing a photographed image of the fabric to the image generation model. The processor 210 generates a difference image between the block image and the intermediate block image. The processor 210 inputs the difference image to the model 310 to determine whether the fabric has an anomaly (i.e., a defect).
[0246] The anomaly detection model may be configured using a technique called FastFlow (FastFlow: Unsupervised Anomaly Detection and Localization via 2D Normalizing Flows). FastFlow is disclosed in, for example, the following paper: Jiawei Yu, Ye Zheng, Xiang Wang, Wei Li, Yushuang Wu, Rui Zhao, Liwei Wu, “Fastflow: Unsupervised Anomaly Detection and Localization via 2d Normalizing Flows.” arXiv:2111.07677, http: / / arxiv.org / abs / 2111.07677
[0247] The anomaly detection model may also be an image generation model (e.g., an autoencoder) trained to generate a defect-free image from an image with defects. The difference image between the captured image and the generated image obtained from the captured image can be used as an image indicating the defect.
[0248] The method for training the anomaly detection model may be any method suitable for the anomaly detection model. In any case, the various training block images described in the above embodiment and modified examples may be used for training the anomaly detection model.
[0249] (6) In the above embodiment and modified examples, the distortion correction model may be various models for correcting distortion, instead of the distortion correction model 320 in FIG. 21. For example, a set of multiple displacements of multiple control points 320ao (FIG. 21) indicates the respective displacement vectors of multiple control points arranged in a matrix, similar to the displacement vector field VFe in FIG. 20(C). Here, the processor 210 may transform the image of the minimum rectangle by homography transformation using the four displacement vectors of the four control points forming the minimum rectangle. The processor 210 performs such homography transformation on each of the multiple minimum rectangles forming the block image.
[0250] The distortion correction model may be an image generation model (e.g., an autoencoder) trained to generate an image of an object (here, a texture) with reduced distortion from an image of the same object having distortion. In this case, it is preferable that the distortion correction model is trained to generate an image of an object having a defect from an image of the object having the same defect. The training process of such a distortion correction model preferably includes a process of adding defects to the training image, as in S310e of FIG. 19.
[0251] (6) In the training process of the above embodiment and modified examples, as in S152 and S154 of FIG. 3, data augmentation in a range narrower than the distribution range of the angles may be applied to one or both of the first angle AC1 and the second angle AC2.
[0252] (7) In the above embodiment and modified examples, the device for optically reading the fabric may be a stationary reading device (e.g., a scanner) instead of a portable reading device such as the digital camera 110. In this way, the optically read image of the fabric is not limited to a photographed image taken by a camera, and may be various read images such as a scanned image read by a scanner.
[0253] (8) In the above embodiment and modified examples, the processor 210 may cause the GPU 260 to execute various calculations. For example, the processor 210 may cause the GPU 260 to execute a part or all of the calculations of a machine learning model (e.g., the anomaly detection model 310 or the distortion correction model 320).
[0254] (9) In the above embodiment and modified examples, the training process may be executed by a device (e.g., a computer) different from the judgment device (e.g., the data processing device 200) that judges whether or not there is a defect in the fabric. The judgment device may also be a type of device different from a personal computer (e.g., a digital camera, a scanner, a smartphone, a tablet computer). Furthermore, multiple devices (e.g., computers) that can communicate with each other via a network may share part of the functions of the inspection process by the judgment device, and collectively provide the inspection process functions (a system including these devices corresponds to the judgment device).
[0255] In each of the above embodiments, a part of the configuration realized by hardware may be replaced by software, and conversely, a part or the whole of the configuration realized by software may be replaced by hardware. For example, the function of the anomaly detection model 310 (FIG. 6) may be realized by a dedicated hardware circuit.
[0256] Furthermore, when some or all of the functions of the present disclosure are realized by a computer program, the program can be provided in a form stored in a computer-readable recording medium (e.g., a non-transitory recording medium). The program can be used in a state stored in the same or a different recording medium (computer-readable recording medium) from when it was provided. The "computer-readable recording medium" is not limited to portable recording media such as memory cards and CD-ROMs, but may also include internal storage devices within a computer, such as various ROMs, and external storage devices connected to a computer, such as a hard disk drive.
[0257] The above-mentioned examples and modifications can be combined as appropriate. The above-mentioned examples and modifications are provided to facilitate understanding of the present disclosure, and do not limit the present invention. The present invention may be modified or improved without departing from the spirit of the present invention, and the present invention includes equivalents thereof. [Explanation of symbols]
[0258] 110...digital camera, 120...light source, 200...data processing device, 210...processor, 215...storage device, 220...volatile storage device, 230...non-volatile storage device, 232...training program, 233...inspection program, 240...display unit, 250...operation unit, 260...graphics processing unit, 270...communication interface, 310...anomaly detection model, 310a...feature extraction model, 310t...image Classification model, 320... distortion correction model, 320a... convolutional net regressor, 320b... spatial transformer, 320c... resampler, 320ao... displacement set, 700... fabric, 700a... target fabric, 700b... target fabric, 700c... target fabric, 700d... target fabric, 700t... training fabric, 700J... knitted fabric, 900... conveyor, 910... first roller, 912... rotary encoder, 920... second roller, 922... drive motor
Claims
1. A program for a computer that judges the presence or absence of defects in a woven or knitted fabric, comprising: A function for acquiring data of a read image, which is an optically read image of the target fabric; a correction function for performing a correction process of the read image, the correction process including a first correction for making an angle of a first direction, which is a direction in which a first pattern formed by threads in an image representing the target fabric is regularly arranged, with respect to a first reference direction closer to a first specific angle; A judgment function that inputs the processed image by the correction process into a machine learning model to judge the presence or absence of a defect in the target fabric, the machine learning model being trained using an image of the fabric whose angle in the first direction is the first specific angle; A program that enables a computer to achieve the above.
2. The program according to claim 1, the machine learning model is trained using a plurality of training images representative of fabrics; The plurality of training images are configured such that a distribution range of angles of a plurality of first directions of the plurality of training images is a specific range including the first specific angle; The specific range is determined using a distribution of angles of the first direction based on one or both of a rotation and a skew of the fabric, which are estimated using a read image of the fabric supported by a conveying device that conveys the target fabric for reading the target fabric. program.
3. The program according to claim 1, the correction process includes a second correction for making an angle of a second direction, which is a direction in which a second pattern formed by threads is regularly arranged in the image representing the target fabric, with respect to a second reference direction approach a second specific angle by shearing of an affine transformation; the machine learning model is trained using a plurality of training images representative of fabrics; The plurality of training images include a plurality of training images that have been corrected by a correction in which an angle of a second direction of the training images with respect to the second reference direction is made to approach the second specific angle by shearing the affine transformation. program.
4. The program according to claim 1, the machine learning model is trained using a plurality of training images representative of fabrics; The plurality of training images are configured such that a distribution range of angles of a plurality of second directions of the plurality of training images is an angle range, the second directions being directions in which a second pattern formed by threads in an image of a fabric is regularly arranged; The angle range is determined using a distribution of angles in the second direction based on one or both of a rotation and a skew of the fabric estimated using a read image of the fabric supported by a conveying device that conveys the target fabric for reading the target fabric. program.
5. The program according to claim 1, The first specific angle is determined using a distribution of angles of the first direction based on one or both of a rotation and a skew of the fabric estimated using a read image of the fabric supported by a conveying device that conveys the target fabric for reading the target fabric. program.
6. A program for a computer that judges the presence or absence of defects in a woven or knitted fabric, comprising: A function for acquiring data of a read image, which is an optically read image of the target fabric; A judgment function of inputting the read image into a machine learning model to judge whether or not the target fabric has a defect; This is realized on a computer. the machine learning model is trained using a plurality of training images representative of fabrics; the plurality of training images are configured such that a distribution range of angles of a plurality of first directions of the plurality of training images with respect to a first reference direction is a first angle range, the first directions being a direction in which a first pattern formed by threads in an image of a fabric is regularly arranged; The first angle range is determined using a distribution of angles of the first direction based on one or both of a rotation and a skew of the fabric estimated using a read image of the fabric supported by a conveying device that conveys the target fabric for reading the target fabric. program.
7. A program for a computer that judges the presence or absence of defects in a woven or knitted fabric, comprising: A function for acquiring data of a read image, which is an optically read image of the target fabric; a correction function for performing a correction process of the read image, the correction process including a distortion correction for correcting a distortion of the target fabric in an image representing the target fabric; A judgment function of inputting the processed image by the correction process into a machine learning model to judge whether or not the target fabric has a defect; A program that enables a computer to achieve the above.
8. The program according to claim 7, The distortion correction includes correcting distortion using a distortion correction model that is a machine learning model that corrects distortion, The distortion correction model is trained to generate a first type of image from a second type of image obtained by applying distortion to a first type of image representing a texture. program.
9. The program according to claim 8, the first type of image is obtained by adding an image of a defect to a portion of an image representing a texture; the second type of image represents the same defect as the defect represented by the first type of image; program.
10. The program according to claim 8 or 9, the distortion is applied in accordance with a displacement vector of each of a plurality of control points arranged in the first type image; The plurality of displacement vectors of the plurality of control points are configured such that a difference between the displacement vectors of adjacent control points is equal to or smaller than a reference value. program.
11. The program according to claim 7, The distortion correction includes correcting the distortion using a result of keypoint matching using a reference image. program.
12. The program according to claim 7, The correction process further includes an angle correction. The angle correction is a first correction for adjusting an angle of a first direction, which is a direction in which a first pattern formed by threads is regularly arranged in an image representing the target fabric, with respect to a first reference direction to approach a first specific angle by affine transformation; a second correction for adjusting an angle of a second direction, which is a direction in which a second pattern formed by threads is regularly arranged in the image representing the target fabric, with respect to a second reference direction to approach a second specific angle by shearing an affine transformation; Including either or both of The distortion correction corrects distortion of the target texture in the processed image caused by the angle correction. program.
13. A program for a computer that judges the presence or absence of defects in a woven or knitted fabric, comprising: A function for acquiring data of a read image, which is an optically read image of the target fabric; A judgment function of inputting the read image into a machine learning model to judge whether or not the target fabric has a defect; This is realized on a computer. The machine learning model is trained using a second type of image obtained by performing image processing including the addition of distortion on a first type of image representing a fabric. program.
14. The program according to claim 13, The image processing further includes an angle change process, The angle change process includes: a first modification process for modifying an angle of a first direction of an image to be processed with respect to a first reference direction by affine transformation, the first direction being a direction in which a first pattern formed by threads in an image of a fabric is regularly arranged; a second modification process for modifying an angle of the second direction of the image to be processed with respect to a second reference direction by shearing an affine transformation, the second direction being a direction in which a second pattern formed by threads in an image of the fabric is regularly arranged; Including either or both of The application of distortion includes applying distortion to the image that has been processed by the angle change process. program.
15. A device for determining the presence or absence of defects in a woven or knitted fabric, comprising: an acquisition unit that acquires data of a read image, which is an optically read image of the target fabric; a correction unit that performs a correction process of the read image, the correction process including a first correction that makes an angle of a first direction, which is a direction in which a first pattern formed by threads in an image representing the target fabric is regularly arranged, with respect to a first reference direction approach a first specific angle; A judgment unit that inputs the processed image by the correction process into a machine learning model to judge the presence or absence of a defect in the target fabric, the machine learning model being trained using an image of the fabric whose angle in the first direction is the first specific angle; A determination device comprising:
16. A device for determining the presence or absence of defects in a woven or knitted fabric, comprising: an acquisition unit that acquires data of a read image, which is an optically read image of the target fabric; A determination unit that inputs the read image into a machine learning model to determine whether or not the target fabric has a defect; Equipped with the machine learning model is trained using a plurality of training images representative of fabrics; the plurality of training images are configured such that a distribution range of angles of a plurality of first directions of the plurality of training images with respect to a first reference direction is a first angle range, the first directions being a direction in which a first pattern formed by threads in an image of a fabric is regularly arranged; The first angle range is determined using a distribution of angles of the first direction based on one or both of a rotation and a skew of the fabric estimated using a read image of the fabric supported by a conveying device that conveys the target fabric for reading the target fabric. Judgment device.
17. A device for determining the presence or absence of defects in a woven or knitted fabric, comprising: an acquisition unit that acquires data of a read image, which is an optically read image of the target fabric; a correction unit that performs a correction process of the read image, the correction process including a distortion correction that corrects a distortion of the target fabric in an image representing the target fabric; A determination unit that inputs the processed image by the correction process into a machine learning model to determine whether or not there is a defect in the target fabric; A determination device comprising:
18. A device for determining the presence or absence of defects in a woven or knitted fabric, comprising: an acquisition unit that acquires data of a read image, which is an optically read image of the target fabric; A determination unit that inputs the read image into a machine learning model to determine whether or not the target fabric has a defect; Equipped with The machine learning model is trained using a second type of image obtained by performing image processing including the addition of distortion on a first type of image representing a fabric. Judgment device.
Citation Information
Patent Citations
Surface defect identification method and apparatus
JP2020187735A
