Program and determination device
The program corrects fabric image orientation and distortion before analyzing it with a machine learning model, addressing accuracy issues in defect detection in woven or knitted fabrics by ensuring accurate defect identification despite orientation and deformation challenges.
Patent Information
- Application Number
- PCT/JP2024/016439
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-12
- Filing Date
- 2024-04-26
- Publication Date
- 2024-11-21
AI Technical Summary
Existing methods for determining defects in woven or knitted fabrics using machine learning models face accuracy issues due to fabric orientation deviations and deformations, such as skewing and distortion, which can occur during the reading process.
A program that corrects fabric image orientation and distortion before inputting it into a machine learning model for defect detection, using techniques like angle correction and distortion correction models to ensure accurate defect identification.
The solution enables accurate determination of defects in fabrics even when the fabric orientation deviates or is distorted, improving the reliability of defect detection in woven or knitted fabrics.
Smart Images

Figure JP2024016439_21112024_PF_FP_ABST
Abstract
Description
Program and determination device
[0001] The present specification relates to a technique for determining the presence or absence of defects in fabrics, such as woven or knitted fabrics.
[0002] Techniques using machine learning to identify defects on a material surface have been proposed. For example, in the technique described in Patent Literature 1, an image of a target material surface is input to a neural network to generate a defect-reduced version, the defect-reduced version is compared with the input image to identify differences, and defects on the target material surface are identified at the positions of the identified differences.
[0003] Japanese Patent Application Laid-Open No. 2020-187735
[0004] A scanned image of the fabric and a machine learning model may be used to determine whether or not there is a defect in a woven or knitted fabric. However, when a scanned image of the fabric is used, the accuracy of the determination result obtained using the machine learning model may be reduced due to various causes. For example, the orientation of the fabric in the scanned image may deviate from the orientation suitable for the machine learning model. Such a deviation in the orientation of the fabric may reduce the accuracy of the determination result. Furthermore, fabric is more susceptible to deformation than hard materials such as metal, hard resin, and glass. For example, when the fabric is transported, deformations such as skew and distortion may occur in the fabric. Deformation of the fabric in the scanned image may reduce the accuracy of the determination result.
[0005] This specification discloses a technique for appropriately determining whether or not a fabric has a defect.
[0006] The techniques disclosed in this specification can be implemented in the following application examples.
[0007] [Application Example 1] A program for a computer that determines whether or not there is a defect in a woven or knitted fabric, the program causing a computer to realize the following functions: a function for acquiring data of a read image, which is an optically read image of the target fabric; a correction function for performing a correction process on the read image, the correction process including a first correction that brings the angle of a first direction, which is a direction in which a first pattern formed by threads in the image representing the target fabric is regularly arranged, relative to a first reference direction closer to a first specific angle; and a judgment function that inputs the image processed by the correction process into a machine learning model to determine whether or not there is a defect in the target fabric, the machine learning model being trained using an image of fabric in which the angle of the first direction is the first specific angle.
[0008] According to this configuration, even if the angle of the first direction indicated by the fabric represented in the scanned image deviates from the first specific angle, it is possible to appropriately determine whether or not there is a defect in the fabric.
[0009] [Application Example 2] A program for a computer that determines whether or not there is a defect in a woven or knitted fabric, the program causing the computer to achieve a function of acquiring data on a read image, which is an optically read image of the target fabric, and a judgment function of inputting the read image into a machine learning model to determine whether or not there is a defect in the target fabric, wherein the machine learning model has been trained using a plurality of training images representing the fabric, the plurality of training images being configured so that a distribution range of angles of a plurality of first directions of the plurality of training images relative to a first reference direction is a first angle range, the first direction being a direction in which a first pattern formed by yarns in the image of the fabric is regularly arranged, and the first angle range is determined using a distribution of angles of the first directions based on one or both of the rotation and skew of the fabric, which is estimated using read images of the fabric supported by a conveying device that conveys the target fabric for reading the target fabric.
[0010] According to this configuration, even if the angle of the first direction indicated by the texture represented by the scanned image changes, it is possible to appropriately determine whether or not there is a defect in the texture.
[0011] [Application Example 3] A program for a computer that determines whether or not there is a defect in a woven or knitted fabric, the program causing the computer to realize the following functions: a function for acquiring data of a read image, which is an optically read image of the target fabric; a correction function for performing a correction process on the read image, the correction process including distortion correction for correcting distortion of the target fabric in an image representing the target fabric; and a judgment function for inputting the image processed by the correction process into a machine learning model to determine whether or not there is a defect in the target fabric.
[0012] According to this configuration, even if the texture shown in the scanned image has distortion, it is possible to appropriately determine whether or not there is a defect in the texture.
[0013] [Application Example 4] A program for a computer that determines whether or not there is a defect in a woven or knitted fabric, the program causing the computer to achieve the following functions: a function of acquiring data on a read image, which is an optically read image of the target fabric; and a judgment function of inputting the read image into a machine learning model to determine whether or not there is a defect in the target fabric; the machine learning model is trained using a second type of image obtained by performing image processing, including the addition of distortion, on a first type of image representing the fabric.
[0014] According to this configuration, even if the texture shown in the scanned image has distortion, it is possible to appropriately determine whether or not there is a defect in the texture.
[0015] The technology disclosed in this specification can be realized in various forms, such as a method and apparatus for determining whether or not there is a defect, a method and apparatus for generating a machine learning model, a computer program for realizing the functions of those methods or apparatus, a recording medium (e.g., a non-temporary recording medium) on which that computer program is recorded, and the like.
[0016] FIG. 1 is an explanatory diagram showing a data processing device according to an embodiment. FIG. 1 is a perspective view of a digital camera 110, a fabric 700, and a conveying device 900. FIG. 2 is a flowchart showing an example of a training process. (A)-(G) are diagrams showing information processed in the training process. (A)-(G) are a flowchart showing an example of a process for estimating a first angle AA1. (A)-(G) are a block diagram showing an example of an anomaly detection model 310. (A)-(G) are a flowchart showing an example of an inspection process. (A)-(D) are diagrams showing examples of a block image and an anomaly map. (A)-(D) are a flowchart showing an example of a training process according to a second embodiment. (A)-(D) are a histogram showing an example of a distribution of the first angle AA1. (A)-(D) are a flowchart showing an example of an inspection process according to a second embodiment. (A)-(D) are a flowchart showing an example of a training process according to a third embodiment. (A)-(D) are a flowchart showing an example of a process for estimating a first angle AC1. (A)-(E) are diagrams showing information processed in relation to the first angle AC1. (F)-(J) are diagrams showing information processed in relation to the second angle AC2. (A)-(E) are a flowchart showing an example of a process for estimating a second angle AC2. (F)-(J) are diagrams showing information processed in relation to the second angle AC2. (F)-(J) are a flowchart showing an example of a process for estimating a second angle AC2. (F)-(J) are a flowchart showing an example of an inspection process according to a third embodiment. 10A is a histogram showing an example of the distribution of the first angle AC1. FIG. 10B is a histogram showing an example of the distribution of the second angle AC2. FIG. 10E is a flowchart showing an example of training processing of the fifth embodiment. FIG. 10A-10E are diagrams showing information processed in the training processing. FIG. 10B is a block diagram showing an example of a distortion correction model 320. FIG. 10C is a flowchart showing an example of an inspection processing of the fifth embodiment. FIG. 10E is a flowchart showing an example of a training processing of the sixth embodiment. FIG. 10F is a flowchart showing an example of an inspection processing of the sixth embodiment. FIG. 10C is a diagram showing an example of distortion correction by keypoint matching. FIG. 10F is a flowchart showing an example of a training processing of the seventh embodiment. FIG. 10F is a flowchart showing an example of a training processing of the eighth embodiment. FIG. 10F is a flowchart showing an example of an inspection processing of the eighth embodiment. FIG. 10F is a flowchart showing an example of a training processing of the ninth embodiment. FIG. 10C is a diagram showing an example of a knitting pattern.
[0017] A. First Example: A1. Device Configuration: Fig. 1 is an explanatory diagram showing a data processing device as one example. Data processing device 200 is, for example, a personal computer. Data processing device 200 performs various data processing for inspecting the appearance of an object (for example, a woven or knitted fabric). Hereinafter, it is assumed that the appearance of a woven fabric 700 is to be inspected.
[0018] The data processing device 200 includes a processor 210, a storage device 215, a display unit 240, an operation unit 250, a graphics processing unit 260 (referred to as GPU 260), and a communication interface 270. These elements are connected to each other via a bus. The storage device 215 includes a volatile storage device 220 and a non-volatile storage device 230.
[0019] The processor 210 is a device configured to process data, such as a central processing unit (CPU) or a system on a chip (SoC). The volatile storage device 220 is, for example, a DRAM, and the nonvolatile storage device 230 is, for example, a flash memory. The nonvolatile storage device 230 stores data for a training program 232, an inspection program 233, an anomaly detection model 310, and a distortion correction model 320. The anomaly detection model 310 and the distortion correction model 320 are program modules that form machine learning models. Details of the programs 232-233 and the models 310 and 320 will be described later. The distortion correction model 320 will be used in other embodiments described later (omitted in this embodiment).
[0020] The display unit 240 is a device configured to display images, such as a liquid crystal display or an organic EL display. The operation unit 250 is a device configured to receive operations by a user, such as a button, a lever, or a touch panel overlaid on the display unit 240. The display unit 240 and the operation unit 250 may form a so-called touch screen. The user can input various requests and instructions to the data processing device 200 by operating the operation unit 250.
[0021] The GPU 260 is a computing device configured to perform various numerical calculations such as image processing and machine learning. The GPU 260 performs various calculations in accordance with instructions from the processor 210. A driver program (not shown) for controlling the GPU 260 may be provided by the manufacturer of the GPU 260.
[0022] The communication interface 270 is an interface for communicating with other devices (for example, it includes one or more of a USB interface, a wired LAN interface, an IEEE802.11 wireless interface, and an industrial camera interface (for example, CameraLink, CoaXPress, etc.)). The digital camera 110 and the conveying device 900 are connected to the communication interface 270. The conveying device 900 is a device that conveys the textile 700. The digital camera 110 is used to photograph the textile 700.
[0023] 2 is a perspective view of the digital camera 110, the fabric 700, and the conveying device 900. In this embodiment, the fabric 700 is conveyed by the conveying device 900. The conveying device 900 has two rollers 910 and 920. The fabric 700 is guided by the two rollers 910 and 920. Between the two rollers 910 and 920, the fabric 700 forms a flat portion. The light source 120 irradiates light onto the flat portion of the fabric 700. The shooting direction D110 of the digital camera 110 faces the flat portion of the fabric 700.
[0024] The conveying device 900 has a rotary encoder 912 connected to the first roller 910 and a drive motor 922 connected to the second roller 920. The drive motor 922 rotates the second roller 920 to convey the fabric 700 from the first roller 910 to the second roller 920. The data processing device 200 can obtain the conveying position of the fabric 700 using data from the rotary encoder 912. The data processing device 200 can control the conveyance of the fabric 700 by controlling the drive motor 922.
[0025] An example of a pattern of the woven fabric 700 is shown at the bottom of FIG. 2. The woven fabric 700 is woven using warp threads Wp and weft threads Wf. The woven fabric 700 can be woven in various ways. Patterns W1, W2, and W3 in the figure represent "plain weave," "twill weave," and "satin weave," respectively. In either case, the woven fabric 700 is configured so that patterns associated with the weave are regularly arranged. When multiple specific patterns are arranged at approximately equal intervals in a specific direction, it can be said that the specific patterns are regularly arranged in a specific direction. The captured image of the woven fabric 700 generated by the digital camera 110 shows the detailed pattern of the woven fabric 700.
[0026] A2. Training Process: Figure 3 is a flow chart showing an example of the training process for the anomaly detection model 310 (Figure 1). The anomaly detection model 310 is used to detect anomalies in the textile 700 in the inspection process described below. The anomaly detection model 310 may be any model that detects anomalies. Furthermore, the training process for the anomaly detection model 310 may be any process suitable for the anomaly detection model 310.
[0027] In this embodiment, the anomaly detection model 310 is trained using training images of a textile without any abnormalities (details of the anomaly detection model 310 will be described later). Here, it is assumed that photographed images of the textile for training are used. In the photographed images, the textile image may be tilted (i.e., the textile image may be rotated) due to various causes. Causes of the rotation of the textile image include, for example, an error in adjusting the orientation of the digital camera 110 relative to the direction in which the textile is conveyed by the conveying device 900, an error in the attachment position of the textile 700 relative to the conveying device 900, and deformation of the soft textile. Unintended rotation of the textile image in the photographed images may reduce the accuracy of inspection using the anomaly detection model 310. Therefore, in this embodiment, the processor 210 corrects the angle of the textile image in the photographed images in both the training process and the inspection process.
[0028] The operator attaches a training fabric 700t (also referred to as a training fabric 700t) to the conveying device 900. The training fabric 700t is a fabric 700 without any abnormalities. The operator then operates the operation unit 250 of the data processing device 200 (FIG. 1) to input an instruction to start the training process. In response to the instruction, the processor 210 starts the training process in accordance with the training program 232.
[0029] In S110, the processor 210 provides a photographing instruction to the digital camera 110. The digital camera 110 photographs the training fabric 700t. The processor 210 acquires photographed image data from the digital camera 110.
[0030] 4(A)-4(G) are diagrams illustrating information processed in the training process. FIG. 4(A) shows an example of a captured image of a textile 700. The figure shows a captured image Ba1 and an enlarged view of a portion Ba1P of the captured image. The captured image Ba1 is a rectangular image having two sides parallel to a first direction Dx and two sides parallel to a second direction Dy perpendicular to the first direction Dx. The data of the captured image Ba1 is color bitmap data representing the color values of the R (red), G (green), and B (blue) components for each pixel. Note that the color components represented by the captured image data may be other color components (e.g., luminance values) instead of R, G, and B.
[0031] As shown in the figure, the captured image Ba1 includes a first portion Pa1 and a second portion Pa2, each having a different brightness. Here, the first portion Pa1 is a bright, island-like portion, and the second portion Pa2 is a dark, remaining portion. The captured image Ba1 shows a plurality of regularly-spaced first portions Pa1. The pattern represented by the first portions Pa1 and the second portions Pa2 represents a pattern formed by the weft threads Wf and the warp threads Wp ( FIG. 2 ). For example, if the weave of the woven fabric 700 shows the third pattern W3, the weft threads Wf form an island-like portion surrounded by the warp threads Wp. In the captured image Ba1, the first portion Pa1 may represent the weft threads Wf, and the second portion Pa2 may represent the warp threads Wp.
[0032] FIG. 4A shows a pattern PTa consisting of two first portions Pa1 and a second portion Pa2 surrounding the first portions Pa1. This pattern PTa is an example of a pattern formed by threads Wf and Wp. This pattern PTa is regularly arranged toward the lower right. Although not shown, on the captured image Ba1, multiple patterns PTa are arranged linearly toward the lower right to form a line PTL. The first direction Da is the direction in which the line PTL extends. Although not shown, on the captured image Ba1, multiple lines PTL are arranged in the second direction Dy.
[0033] In S130 (FIG. 3), the processor 210 divides the captured image into a plurality of block images, each of which has a size (i.e., width (number of pixels in the first direction Dx) and height (number of pixels in the second direction Dy)) that is acceptable to the anomaly detection model 310 (FIG. 1).
[0034] In S140, the processor 210 estimates a first angle AA1 of each block image. FIG. 5 is a flowchart showing an example of the process of estimating the first angle AA1. In S410, the processor 210 binarizes the block image. Any binarization method may be used (e.g., Otsu's binarization). FIG. 4(B) shows a portion of the binarized block image. As shown in the figure, the binarized image Ba2 represents a first portion Pa1 and a second portion Pa2.
[0035] In S420 (FIG. 5), the processor 210 selects a feature point of interest from a plurality of feature points included in the binarized image. In this embodiment, an island-shaped portion (here, the first portion Pa1) is used as the feature point. The processor 210 selects the unprocessed first portion Pa1 as the feature point of interest.
[0036] In S430, the processor 210 selects the closest feature point from the area to the lower right of the target feature point. For example, if feature point Pa1a (FIG. 4B) is the target feature point, feature point Pa1b, which is located to the lower right of target feature point Pa1a, is selected. These two feature points Pa1a and Pa1b form pattern PTa.
[0037] In S440 ( FIG. 5 ), an angle AA1v of a vector Va extending from the start point to the end point is calculated. FIG. 4C shows an example of the vector Va. The vector Va extends from the start point Ps to the end point Pe. In this embodiment, the start point Ps is the center of gravity of the target feature point selected in S420, and the end point Pe is the center of gravity of the feature point selected in S430. The angle AA1v is the angle of the vector Va with respect to the reference direction. In this embodiment, the reference direction is the first direction Dx (the first direction Dx is also referred to as the reference direction Dx). The angle AA1v is the angle at which the direction obtained by rotating the reference direction Dx clockwise by the angle AA1v is the same as the direction of the vector Va.
[0038] In S450 ( FIG. 5 ), the processor 210 determines whether processing of all feature points has been completed. If unprocessed feature points remain ( S450 : No), the processor 210 proceeds to S420 to process new feature points. As shown in FIG. 4 (D) , the processor 210 calculates the angles AA1v of the multiple vectors Va of the multiple first portions Pa1.
[0039] If processing of all feature points is complete ( FIG. 5 : S450: Yes), in S460, the processor 210 estimates a first angle AA1 of the block image using the multiple angles AA1v of the multiple vectors Va. The first angle AA1 may be various representative angles represented by the multiple angles AA1v. In this embodiment, the processor 210 generates a histogram of the angles AA1v. FIG. 4(E) shows an example of a histogram of the angles AA1v. The horizontal axis represents the angle AA1v, and the vertical axis represents the frequency F. Note that zero degrees and 360 degrees of the angle AA1v represent the same angle. The same applies to other angles described below. The processor 210 adopts the angle associated with the highest frequency F as the first angle AA1.
[0040] The first angle AA1 may be estimated by various other methods. For example, the processor 210 may calculate an approximation curve Cav that approximates the histogram and use the angle of the peak of the approximation curve Cav as the first angle AA1. The approximation curve Cav may be represented by various functions (e.g., a Gaussian function representing a normal distribution). The processor 210 may also use various summary statistics (such as the mean, median, or mode) of the multiple angles AA1v as the first angle AA1.
[0041] As shown in Figures 4(B) and 4(C), vector Va indicates the direction in which two first portions Pa1 (e.g., feature points Pa1a and Pa1b) that form pattern PTa are arranged. The multiple patterns PTa are arranged regularly in approximately the same direction as vector Va. First angle AA1 indicates the direction in which the multiple patterns PTa are arranged regularly (i.e., first direction Da, which is the direction in which line PTL formed by the multiple patterns PTa extends).
[0042] After estimating the first angle AA1 (FIG. 5: S460), the processor 210 ends the processing in FIG. 5. In addition, in S140 (FIG. 3), the processor 210 determines the first angle AA1 for each block image. When the first angles AA1 for all block images have been determined, the processing proceeds to S150.
[0043] In S150, the processor 210 corrects the angle of each block image so that the first angle AA1 becomes a first specific angle AA1Z. In this embodiment, the first specific angle AA1Z is determined in advance. The processor 210 also corrects the first angle AA1 by rotating the block image.
[0044] 4C shows an example of the first specific angle AA1Z. Like the angle AA1v, the first specific angle AA1Z indicates an angle relative to the reference direction Dx. The direction Daz is determined by the first specific angle AA1Z (referred to as the reference direction Daz). It is preferable to adopt a direction associated with a typical first angle AA1 as the reference direction Daz. This allows the rotation angle of the block image for correction to be reduced.
[0045] FIG. 4F shows an example of correcting the first angle AA1. The processor 210 rotates the block image Ba3 so that the first angle AA1 is equal to the first specific angle AA1Z. Through this rotation, the processor 210 generates data for a corrected block image Ba4. The angle of rotation is the difference between the first specific angle AA1Z and the angle AA1v. The center of rotation may be the center of the block image Ba3. The height and width of the corrected block image Ba4 are the same as those of the block image Ba3. A portion P3a of the block image Ba3 that extends beyond the corrected block image Ba4 may be truncated. The color of pixels in a portion P4a of the corrected block image Ba4 that is not assigned to the block image Ba3 may be set to a predetermined color (e.g., black). Alternatively, the processor 210 may acquire a partial image corresponding to the portion P4a from the original captured image acquired in S110 (FIG. 3) (i.e., the captured image before division) and assign the acquired partial image to the portion P4a. Here, the processor 210 acquires a partial image from the captured image that has been rotated by the rotation angle of the block image Ba3. Hereinafter, the corrected block image will be referred to as a training block image.
[0046] As will be described later, in this embodiment, the image is corrected in the inspection process so that the first angle AA1 is the same as the first specific angle AA1Z. A small tilt of the image of the fabric 700 may remain in such an angle-corrected image. For example, because the fabric 700 is soft, the image of the fabric 700 may be tilted in a portion of the image. Therefore, in this embodiment, the processor 210 generates data of a plurality of training images showing various first angles AA1 through data augmentation.
[0047] In S152 (FIG. 3), the processor 210 determines a specific range AA1S using the distribution of the multiple first angles AA1 obtained in S140. FIG. 4G is a histogram showing an example of the distribution of the first angles AA1. The horizontal axis represents the first angle AA1, and the vertical axis represents the frequency F. The processor 210 determines a reference range AA1A so as to include most of the multiple first angles AA1. The processor 210 determines the width (referred to as the first width) of the specific range AA1S by narrowing the width of the reference range AA1A. For example, the processor 210 calculates the first width of the specific range AA1S by multiplying the width of the reference range AA1A by a predetermined first coefficient greater than zero and less than 1 (e.g., 0.1 or greater and 0.9 or less). The processor 210 adopts the range of the first width centered on the first specific angle AA1Z as the specific range AA1S. The first coefficient is experimentally determined in advance so that the first angle AA1 indicated by the tilt of the image of the fabric 700 in the angle-corrected image falls within a specific range AA1S.
[0048] Various methods may be used to determine the reference range AA1A. For example, the processor 210 may sort the multiple first angles AA1 in ascending order and use the distribution range of the remaining first angles AA1, excluding a portion of the top angles (e.g., 5%) and a portion of the bottom angles (e.g., 5%), as the reference range AA1A. Alternatively, the processor 210 may calculate an approximation curve Caa that approximates the histogram and use the range defined by the approximation curve Caa as the reference range AA1A. For example, the approximation curve Caa may be represented by a Gaussian function representing a normal distribution. The reference range AA1A may be a range centered on the mean of the normal distribution and having a width four times the standard deviation. In this way, it is preferable that the processor 210 determine the reference range AA1A so as to include the remaining multiple first angles AA1 excluding outliers. In either case, a range including the first specific angle AA1Z is used as the specific range AA1S.
[0049] In S154 (FIG. 3), the processor 210 generates new data for a plurality of training block images through data augmentation. In this embodiment, the processor 210 generates the plurality of training block images by rotating the training block images (i.e., the corrected block images) generated in S150 at various angles. Here, the processor 210 generates the plurality of training block images so that the plurality of first angles AA1 of the plurality of training block images ultimately obtained are approximately uniformly distributed within a specific range AA1S. In S150, a plurality of corrected block images are generated. The processor 210 generates a plurality of training block images using the plurality of corrected block images. If a missing portion (e.g., portion P4a in FIG. 4(F)) occurs due to the rotation of the image, the processor 210 may fill in the missing portion using the original captured image (S110), as described in S150 (FIG. 3).
[0050] Here, the processor 210 ultimately generates data on N training block images (N is an integer greater than or equal to 2). The number N of training block images may be any number suitable for training the anomaly detection model 310 (e.g., several hundred or several thousand). To generate the N training block images, the processor 210 may acquire data on multiple captured images in S110. Then, in S130, the processor 210 may acquire multiple block images from each of the multiple captured images. Here, it is preferable that the portions of the training fabric 700t represented by the captured images differ between the multiple captured images. For example, the processor 210 may cause the conveying device 900 to convey the fabric 700t each time the fabric 700t is captured.
[0051] In S290 (FIG. 3), the processor 210 trains the anomaly detection model 310 using N training block images. FIG. 6 is a block diagram illustrating an example of the anomaly detection model 310. In this embodiment, the anomaly detection model 310 is configured using an anomaly detection technique called PaDiM (a Patch Distribution Modeling Framework for Anomaly Detection and Localization). PaDiM is disclosed, for example, in the paper "T. Defard, A. Setkov, A. Loesch, and R. Audigier, "Padim: a patch distribution modeling framework for anomaly detection and localization," arXiv:2011.08785(2020), https: / / arxiv.org / abs / 2011.08785, posted on 17 Nov 2020."
[0052] In the PaDiM technology, a feature vector is generated from image data using a feature extraction model. A plurality of feature vectors are generated in advance from a plurality of image data representing normal objects, and a mean vector and a covariance matrix indicating the distribution of the plurality of feature vectors are calculated (the set of the mean vector and the covariance matrix is also called Gaussian parameters). The Gaussian parameters are calculated through a training process. In an inspection process, a feature vector is generated from image data representing the object to be inspected. The distance (here, the Mahalanobis distance) obtained from the feature vector and the Gaussian parameters is then calculated as an anomaly score.
[0053] In this embodiment, a portion of an image classification model 310t for classifying images into multiple classes is used as the feature extraction model 310a. Input image data Id is input to the image classification model 310t. The input image data Id is bitmap data representing a rectangular image having two sides parallel to a first direction Dx and two sides parallel to a second direction Dy perpendicular to the first direction Dx. The image classification model 310t uses the input image data Id to generate output data Od indicating the type of the input image data Id (e.g., the confidence level for each class).
[0054] Various models can be used as the image classification model 310t. In this embodiment, a known model called ResNet18 is used as the image classification model 310t. ResNet18 is disclosed in, for example, the paper "K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in ICML, 2016."
[0055] In this embodiment, the image classification model 310t includes multiple processing blocks BC1-BC4 and BF. Each of the processing blocks BC1-BC4 is a convolutional neural network (CNN). Each of the processing blocks BC1-BC4 includes one or more convolutional layers. The processing block BF includes one or more fully connected layers. The processing block BF generates output data Od by reducing the dimensionality of the feature map from the previous processing block.
[0056] The image classification model 310t has multiple calculation parameters. For example, each convolution layer has filter weights and biases used in the convolution calculation. Each fully connected layer has weights and biases used in the calculation. The multiple calculation parameters are adjusted by training the image classification model 310t. Note that data of various trained image classification models is publicly available on the Internet. In this embodiment, data of a publicly available trained image classification model is used as the image classification model 310t. The trained image classification model has been trained using a large amount of image data (e.g., a dataset called ImageNet, which is classified into 1,000 classes). Such a trained model can appropriately extract image features of the textile 700.
[0057] To generate the feature vector, multiple feature maps obtained from multiple convolutional layers of the image classification model 310t are used. In this embodiment, the processor 210 uses a first feature map fm1 from the convolutional layer included in the first processing block BC1, a second feature map fm2 from the convolutional layer included in the second processing block BC2, and a third feature map fm3 from the convolutional layer included in the third processing block BC3. The feature extraction model 310a included in the anomaly detection model 310 is the first part of the image classification model 310t, and the part including the processing blocks BC1-BC3 is used. Note that the feature extraction model 310a may include the entire image classification model 310t.
[0058] The feature maps fm1, fm2, and fm3 each represent the feature values of a plurality of pixels aligned in a first direction Dx and a second direction Dy. The width W indicates the number of pixels in the first direction Dx of the first feature map fm1, and the height H indicates the number of pixels in the second direction Dy of the first feature map fm1. The feature maps fm1, fm2, and fm3 have different resolutions. The order of increasing resolution is fm1, fm2, and fm3.
[0059] The numbers n1, n2, and n3 in the figure indicate the number of channels of the feature maps fm1, fm2, and fm3, respectively, where n1<n2<n3.
[0060] The processor 210 converts the feature maps fm2 and fm3 into feature maps fm2r and fm3r having the same resolution as the largest feature map fm1 by a known enlargement process (i.e., a resolution conversion process, e.g., nearest neighbor). The resolution conversion is performed for each channel. These feature maps fm1, fm2r, and fm3r collectively represent W*H*(n1+n2+n3) feature quantities.
[0061] The processor 210 forms a feature matrix FM using R channels out of the n1+n2+n3 channels of the feature maps fm1, fm2r, and fm3r (R<n1+n2+n3). The R channels may be selected in advance. Alternatively, the processor 210 may select the R channels randomly (for example, using random numbers). The feature matrix FM is composed of H*W*R elements. The feature matrix FM represents the feature vectors of each of the H*W pixels. The feature vector xij in the figure represents the feature vector at position (i, j) (i represents the pixel position in the first direction Dx, and j represents the pixel position in the second direction Dy). The feature vector xij is an R-dimensional vector having R elements.
[0062] 6 shows N training block images TI1-TIN. The processor 210 uses the N training block images TI1-TIN to calculate N feature matrices FM, resulting in N feature vectors for each of the H*W pixel locations.
[0063] The bottom of FIG. 6 shows N feature vectors xij,1-xij,N calculated from N training block images TI1-TIN. These N feature vectors correspond to positions (i, j). The processor 210 calculates a mean vector μij and a covariance matrix Σij using the N feature vectors xij,1-xij,N. The mean vector μij indicates the average feature vector xij obtained from images of the normal textile 700t. The covariance matrix Σij is a matrix of covariances between elements of multiple feature vectors xij obtained from images of the normal textile 700t. The covariance matrix Σij is calculated according to the calculation formula described in the above-mentioned paper by PaDiM. The set of the mean vector μij and the covariance matrix Σij is also called Gaussian parameters.
[0064] The processor 210 calculates Gaussian parameters for each of the H*W pixels. As a result, the processor 210 determines the value of each element of the Gaussian matrix GM composed of the H*W Gaussian parameters. The processor 210 stores data of the anomaly detection model 310, which includes the feature extraction model 310a and the Gaussian matrix GM, in the storage device 215 (here, the non-volatile storage device 230). Then, the processor 210 ends the training process of FIG. 3. As described above, in this embodiment, the value of each element of the Gaussian matrix GM is determined by the training process.
[0065] A3. Inspection Process: Figure 7 is a flowchart showing an example of the inspection process. An operator attaches the fabric 700 to be inspected (also referred to as the target fabric 700) to the conveying device 900 (Figure 2). The operator then operates the operation unit 250 of the data processing device 200 (Figure 1) to input an instruction to start the inspection process. In response to the instruction, the processor 210 starts the inspection process in accordance with the inspection program 233.
[0066] In S710, the processor 210 controls the drive motor 922 (FIG. 2) to transport the target fabric 700 so as to capture images of uninspected portions of the target fabric 700. Note that in the first S710 after the start of the inspection process, transportation of the target fabric 700 may be omitted.
[0067] In S720, the processor 210 provides a photographing instruction to the digital camera 110. The digital camera 110 photographs the target fabric 700. The processor 210 acquires photographed image data from the digital camera 110.
[0068] In S730, the processor 210 divides the captured image into a plurality of block images in the same manner as in S130 (FIG. 3).
[0069] In S740, the processor 210 estimates the first angle AA1 of each of the plurality of block images. The method of estimating the first angle AA1 is the same as the method of estimating the first angle AA1 in S140 (FIG. 3).
[0070] In S750, the processor 210 corrects the angle of each block image so that the first angle AA1 becomes the first specific angle AA1Z. The correction method is the same as the correction method in S150 (FIG. 3).
[0071] In S890, the processor 210 inputs the corrected block image into the anomaly detection model 310 to determine whether the target textile 700 has an anomaly (i.e., a defect). The determination method may be any method suitable for the anomaly detection model 310. In this embodiment, the processor 210 calculates an anomaly score for each of the H*W positions using the method described in the above-mentioned paper by PaDiM. Specifically, the processor 210 calculates a feature matrix F by inputting the corrected block image into the anomaly detection model 310 ( FIG. 6 ). Here, the R channels used during training are used. The feature matrix F represents the feature vector for each of the H*W positions. The processor 210 calculates the anomaly score using the feature vector xij and the Gaussian parameters (μij, Σij). The anomaly score is calculated as the Mahalanobis distance between the feature vector xij and the Gaussian parameters (μij, Σij). (The formula for calculating the anomaly score is the formula described in the above-mentioned paper by PaDiM.) The anomaly score is calculated for each of the H*W positions. A large anomaly score indicates a high probability of an anomaly.
[0072] The anomaly scores for each of the H*W positions form a so-called anomaly map (also called a heat map). Figures 8(A)-8(D) show examples of block images and anomaly maps. Note that the resolution of the anomaly map is lower than that of the block images. In the figures, each image has been resized to the same size for ease of comparison.
[0073] 8A shows a block image BIa of a normal target fabric 700a and an anomaly map BHa. As shown in the figure, the anomaly map BHa does not show any abnormal portions with large anomaly scores.
[0074] 8(B) shows a block image BIb of a target fabric 700b having a linear defect 701b and an anomaly map BHb. Such a defect 701b can be formed due to various causes. For example, a defect in a yarn (e.g., a weft yarn Wf or a warp yarn Wp (FIG. 2)) can cause the defect 701b. Also, the defect 701b can be formed due to contact between the target fabric 700b and another component (e.g., a device for transporting the fabric 700). The anomaly map BHb shows a linear anomaly portion ANb representing the defect 701b.
[0075] 8(C) shows a block image BIc of a target fabric 700c having a point-like defect 701c and an anomaly map BHc. Such a defect 701c can be formed due to various causes. For example, fraying of a yarn (e.g., a weft yarn Wf or a warp yarn Wp (FIG. 2)) can cause the defect 701c. Also, the defect 701c can be formed due to contact between the target fabric 700b and another component (e.g., a device for transporting the fabric 700). The anomaly map BHc shows a point-like abnormal portion ANc indicating the defect 701c.
[0076] 8D shows a block image BId of a target fabric 700d having a stain 701d and an abnormality map BHd. The abnormality map BHd shows an abnormal portion ANd indicating the stain 701d.
[0077] The processor 210 calculates an anomaly score for each of the multiple positions in the multiple block images. The processor 210 uses the multiple anomaly scores to determine whether an anomaly condition for determining that the target textile 700 has an anomaly is met. The anomaly condition may be various conditions. For example, the anomaly condition may be that the maximum anomaly score is equal to or greater than a first threshold. The first threshold may be determined experimentally in advance so as to appropriately detect an anomaly in the target textile 700. The anomaly condition may also be that the total number of pixels having an anomaly score equal to or greater than a second threshold is equal to or greater than a third threshold. The second and third thresholds may be determined experimentally in advance so as to appropriately detect an anomaly in the target textile 700.
[0078] In S894 (FIG. 7), the processor 210 determines whether the abnormal condition is satisfied. If the abnormal condition is not satisfied (S894: No), the processor 210 determines in S896 whether the conveyance is completed. If an uninspected portion of the target fabric 700 remains, the determination result in S896 is No. In this case, the processor 210 proceeds to S710 and inspects the uninspected portion of the target fabric 700. If the conveyance is completed (S896: Yes), the processor 210 ends the inspection process.
[0079] If the abnormality condition is met in S894 (S894: Yes), the processor 210 executes a first process in S898. The first process may be any process related to the abnormality. In this embodiment, the processor 210 stops the conveyance of the target fabric 700 and notifies the operator of the abnormality. Various methods may be used to notify the operator of the abnormality. For example, the processor 210 may display an error message or an abnormality map on the display unit 240. Alternatively, the processor 210 may output a warning sound from a speaker (not shown). After S898, the processor 210 ends the inspection process.
[0080] As described above, in this embodiment, the processor 210 of the data processing device 200 executes the following process to determine whether or not the target fabric 700 has a defect. In steps S720 and S730 ( FIG. 7 ), the processor 210 acquires block image data representing a captured image of the target fabric 700. The block image data is an example of scanned image data, which is an optically scanned image of the target fabric 700. In step S750, the processor 210 performs a correction process for the block image. As shown in FIG. 4(F) , the correction process includes adjusting the first angle AA1 of the first direction Da ( FIG. 4(F) ) relative to the reference direction Dx to approach the first specific angle AA1Z. As described with reference to FIGS. 4(A) and 4(F) , the first angle AA1 indicates the first direction Da, which is the direction in which multiple patterns PTa are regularly arranged in the image representing the target fabric 700. The patterns PTa are formed by the yarns Wf and Wp. In S890, the processor 210 inputs the processed block image obtained by the correction process into the anomaly detection model 310 to determine whether or not there is a defect in the target fabric 700. As described in S150 and S290 of FIG. 3 and in FIG. 5, the anomaly detection model 310 (specifically, the Gaussian matrix G M ) is trained using an image of the training fabric 700t in which the first angle AA1 is the first specific angle AA1Z (e.g., the corrected block image Ba4 ( FIG. 4(F))).
[0081] With this configuration, even if the first angle AA1 indicated by the target fabric 700 represented by the block image deviates from the first specific angle AA1Z, the block image is corrected so that the first angle AA1 becomes the first specific angle AA1Z (the block image is an example of a photographed image). Furthermore, the multiple training block images include a training block image in which the first angle AA1 is the first specific angle AA1Z. Therefore, by using the anomaly detection model 310, the processor 210 can appropriately determine whether or not the target fabric 700 has a defect.
[0082] In this embodiment, as described in FIG. 3 , the anomaly detection model 310 is trained using a plurality of training block images representing a training fabric 700t. As described in S152 and S154, the plurality of training block images are configured such that the distribution range of the plurality of first angles AA1 in the plurality of training block images is a specific range AA1S including a first specific angle AA1Z. As described in S152 ( FIG. 3 ) and FIG. 4 (G), the specific range AA1S is determined using the distribution of the first angles AA1 estimated using the captured images of the training fabric 700t. Here, the training fabric 700t is supported by a conveying device 900 that conveys the target fabric 700 for photographing the target fabric 700. The first angle AA1 indicates an angle based on the rotation of the fabric 700 (e.g., the training fabric 700t).
[0083] According to this configuration, even if a tilt in the image of the target fabric 700 remains in the corrected block image, the anomaly detection model 310 has been trained using a training block image that shows such a tilt, and therefore the processor 210 can appropriately determine whether or not there is a defect in the target fabric 700 by using the anomaly detection model 310.
[0084] The specific range AA1S is determined using the distribution of the first angle AA1 estimated using the photographed images of the training fabric 700t. Therefore, it is easy to avoid using a specific range AA1S that is wider than the width of the actual distribution of the first angle AA1. As a result, the burden of training the anomaly detection model 310 can be reduced.
[0085] Note that if the effect of the tilt of the image of the target fabric 700 in the corrected block image (i.e., the effect on the inspection results) is small, steps S152 and S154 may be omitted.
[0086] B. Second Example: B1. Training Process: Figure 9 is a flowchart showing an example of the training process of the second example. In this example, the processor 210 trains the anomaly detection model 310 using a plurality of training block images showing different first angles AA1.
[0087] S110, S130, and S140 are the same as S110, S130, and S140, respectively, in Fig. 3. The processor 210 estimates the first angle AA1 of each of the plurality of block images representing the normal fabric 700t.
[0088] In S142b, the processor 210 determines an angle range AA1R using the distribution of the multiple first angles AA1. The angle range AA1R is used for data augmentation, which will be described later. FIG. 10 is a histogram showing an example of the distribution of the first angles AA1. The horizontal axis represents the first angle AA1, and the vertical axis represents the frequency F. The processor 210 determines the angle range AA1R (specifically, a lower limit AA1L and an upper limit AA1H) so as to include most of the multiple first angles AA1. The method for determining the angle range AA1R may be the same as the method for determining the reference range AA1A (FIG. 4(G)) described in S152 (FIG. 3).
[0089] In S144b, the processor 210 determines a first specific angle AA1Z. In this embodiment, the processor 210 uses the center value of the angle range AA1R (i.e., the average value of the lower limit AA1L and the upper limit AA1H) as the first specific angle AA1Z.
[0090] S150 is the same as S150 in Fig. 3. The processor 210 corrects the angle of each block image so that the first angle AA1 becomes the first specific angle AA1Z.
[0091] In S154b, the processor 210 generates data for a plurality of training block images by data augmentation. In this embodiment, the processor 210 generates the plurality of training block images by rotating the corrected block images generated in S150 by various angles.
[0092] Here, the processor 210 generates data for the plurality of training block images so that the plurality of first angles AA1 of the plurality of training block images finally obtained are approximately uniformly distributed within the angle range AA1R. For example, the processor 210 randomly determines the angle of rotation for each of the plurality of training block images so that the plurality of angles (corresponding to the plurality of first angles AA1) obtained from the first specific angle AA1Z by rotation are approximately uniformly distributed within the angle range AA1R.
[0093] As described above, in S150, data for a plurality of corrected block images is generated. The processor 210 generates a plurality of training block images using the plurality of corrected block images. If a missing portion (e.g., portion P4a in FIG. 4(F)) occurs due to image rotation, the processor 210 may fill in the missing portion using the original captured image (S110), as described in S150 (FIG. 3).
[0094] Furthermore, the processor 210 may acquire data of a plurality of captured images in S110 in order to generate N training block images, similar to the embodiment of FIG.
[0095] S290 is the same as S290 in Fig. 3. The processor 210 trains the anomaly detection model 310 (here, the Gaussian matrix G M ) using the N training block images. Then, the processor 210 ends the processing in Fig. 9.
[0096] B2. Inspection Process: Fig. 11 is a flowchart showing an example of the inspection process of Example 2. The main difference from the inspection process of Fig. 7 is that S740 and S750 for correcting the first angle AA1 are omitted.
[0097] S710, S720, and S730 are the same as S710, S720, and S730, respectively, in Fig. 7. The processor 210 divides the captured image into a plurality of block images.
[0098] In S890b, the processor 210 inputs each of the multiple block images into the anomaly detection model 310 to determine whether or not the target textile 700 has an anomaly. The only difference from S890 in Fig. 7 is that uncorrected block images are used instead of corrected block images.
[0099] S894, S896, and S898 after S890b are the same as S894, S896, and S898, respectively, in Fig. 7. If the determination result in S896 is No, the processor 210 proceeds to S710.
[0100] As described above, in this embodiment, the processor 210 executes the following process to determine whether or not the target fabric 700 has a defect. In S720 and S730 ( FIG. 11 ), the processor 210 acquires block image data representing a captured image of the target fabric 700. In S890b, the processor 210 inputs the block image data into the anomaly detection model 310 to determine whether or not the target fabric has a defect. As described in FIG. 9 , the anomaly detection model 310 is trained using multiple training block images representing the training fabric 700t. The data for the multiple training block images is generated by data augmentation in S154b. In this embodiment, the first angle AA1 of the training block image changes due to data augmentation. As described in FIGS. 4(A) and 4(F) , the first angle AA1 indicates the direction in which the multiple patterns PTa are regularly arranged in the image of the fabric 700 (i.e., the first direction Da in which the line PTL formed by the multiple patterns PTa extends). The pattern PTa is formed by the yarns Wf and Wp. As described in S154b, the training block images are configured such that the distribution range of the first angles AA1 of the first directions Da of the training block images relative to the reference direction Daz is an angle range AA1R. As described in S154b and FIG. 10, the angle range AA1R is determined using the distribution of the first angles AA1 estimated using the captured images of the training fabric 700t. Here, the training fabric 700t is supported by the conveying device 900, which conveys the target fabric 700 for photographing the target fabric 700. Furthermore, the first angle AA1 indicates the angle of the first direction Da based on the rotation of the fabric 700 (e.g., the training fabric 700t).
[0101] According to this configuration, the plurality of training block images are configured so that the distribution range of the plurality of first angles AA1 in the plurality of training block images is the angle range AA1R, and the angle range AA1R is determined using the distribution of the first angles AA1 estimated using the photographed images. Therefore, even if the first angle AA1 indicated by the target textile 700 represented by the photographed images changes, the processor 210 can appropriately determine the presence or absence of a defect in the target textile 700 by using the anomaly detection model 310. Furthermore, it is easy to avoid using an angle range AA1R wider than the width of the actual distribution of the first angles AA1. As a result, the burden of training the anomaly detection model 310 can be reduced.
[0102] C. Third Embodiment: C1. Training Process: FIG. 12 is a flowchart illustrating an example of the training process of the third embodiment. This differs from the embodiment of FIG. 3 in that the skew of the fabric 700 is corrected in addition to the tilt caused by the rotation of the image of the fabric 700. Skew of the fabric 700 can occur due to various causes. Skew of the fabric 700 can be caused by, for example, uneven application of force to the fabric 700 during the manufacturing process of the fabric 700, an error in the attachment position of the fabric 700 relative to the conveying device 900, deformation of the soft fabric 700, and the like. Note that data augmentation of angles (S152, S154 (FIG. 3)) is omitted.
[0103] Steps S110 and S130 are the same as steps S110 and S130 in Fig. 3. The processor 210 divides the captured image into a plurality of block images.
[0104] In S140c, the processor 210 estimates a first angle AC1 for each block image. In this embodiment, the first angle AC1 is different from the first angle AA1 in Fig. 4(F) and indicates a direction that is approximately parallel to the first direction Dx (details will be described later).
[0105] Fig. 13 is a flowchart showing an example of the process of estimating the first angle AC1. Step S410 is the same as step S410 in Fig. 5. The processor 210 binarizes the block image.
[0106] 14(A) to 14(E) are diagrams showing information processed in relation to the first angle AC1. FIG. 14(A) shows an example of a portion of a binarized block image. The binarized image Bc2 represents a plurality of first portions Pc1, which are bright island-like portions, and a second portion Pc2, which is the remaining dark portion. The binarized image Bc2 represents a pattern represented by the plurality of first portions Pc1 and second portions Pc2. This pattern is formed by the yarns Wf and Wp (FIG. 2).
[0107] FIG. 14A shows a first pattern PTc1 composed of two first portions Pc1 (e.g., first portions Pc11 and Pc12) and a second portion Pc2 surrounding the first portions Pc1. This first pattern PTc1 is an example of a pattern formed by threads Wf and Wp. The multiple first patterns PTc1 are regularly arranged in a direction approximately parallel to the first direction Dx. The multiple first patterns PTc1 are arranged linearly in a direction approximately parallel to the first direction Dx to form lines PTLc1. The first direction Dc1 is the direction in which the lines PTLc1 extend. Although not shown, the multiple lines PTLc1 are arranged in the second direction Dy on the binarized image Bc2. The multiple lines PTLc1 are also arranged in the second direction Dy on the captured image.
[0108] S420 (FIG. 13) is the same as S420 in FIG. 5. In this embodiment, the island-shaped first portion Pc1 is used as the feature point. The processor 210 selects the unprocessed first portion Pc1 as the target feature point.
[0109] In S430c1, the processor 210 selects a feature point from a search area located in the first direction Dx of the target feature point. FIG. 14B shows the target feature point Pc11 and the selection area SA11. In this embodiment, the selection area SA11 is a circular area. The center Pc11s of the selection area SA11 is located at a distance Ds1 in the first direction Dx from the center of gravity position Pc11c of the target feature point Pc11. The value Rc1 indicates the radius of the selection area SA11. As shown in FIG. 14A, the selection area SA11 associated with the target feature point Pc11 includes a first portion Pc12. The processor 210 selects the first portion Pc12 from this selection area SA11.
[0110] The distance Ds1 and the radius Rc1 are experimentally determined in advance to match the fabric 700 so that the selection area SA11 can include one entire first portion Pc1 but not two entire first portions Pc1. One or more first portions Pc1 may contact the selection area SA11. The processor 210 selects the first portion Pc1 that is included in the selection area SA11 with the largest size (e.g., the largest number of pixels) from the one or more first portions Pc1 that contact the selection area SA11.
[0111] In S440c1, the processor 210 calculates the angle AC1v of the vector Vc1 extending from the starting point to the end point. FIG. 14C shows an example of the vector Vc1. The method for determining the vector Vc1 is the same as the method for determining the vector Va in FIG. 4C. The starting point Pc1s of the vector Vc1 is the center of gravity of the target feature point Pc1 (e.g., the first portion Pc11) selected in S420, and the end point Pc1e of the vector Vc1 is the center of gravity of the feature point Pc1 (e.g., the first portion Pc12) selected in S430c1. The method for calculating the angle AC1v is the same as the method for calculating the angle AA1v in FIG. 4C. In this embodiment, the reference direction of the angle AC1v is the first direction Dx.
[0112] S450 (Fig. 13) is the same as S450 in Fig. 5. If unprocessed feature points remain (S450: No), the processor 210 proceeds to S420 to process new feature points.
[0113] If processing of all feature points is completed (S450: Yes), in S460c1, the processor 210 estimates a first angle AC1 of the block image using the multiple angles AC1v of the multiple vectors Vc1. The method of estimating the first angle AC1 is the same as the method of estimating the first angle AA1 in S460 of FIG. 5. FIG. 14(D) shows an example of a histogram of the angles AC1v. The processor 210 adopts the angle associated with the maximum frequency F as the first angle AC1. The method of estimating the first angle AC1 may be various other methods (for example, a method using an approximation curve Ccv1, a method using various summary statistics (average, median, mode, etc.) of the multiple angles AC1v, etc.).
[0114] 14A , vector Vc1 indicates the direction in which two first portions Pc1 (e.g., feature points Pc11 and Pc12) that form first pattern PTc1 are arranged. The multiple first patterns PTc1 are arranged regularly in approximately the same direction as vector Vc1. First angle AC1 indicates the direction in which the multiple first patterns PTc1 are arranged regularly (i.e., first direction Dc1, which is the direction in which line PTLc1 formed by the multiple first patterns PTc1 extends).
[0115] After estimating the first angle AC1 (FIG. 13: S460c1), the processor 210 ends the processing in FIG. 13. Note that in S140c (FIG. 12), the processor 210 estimates the first angle AC1 for each of the multiple block images.
[0116] In S150c, the processor 210 corrects the angle of each block image so that the first angle AC1 becomes the first specific angle AC1Z. In this embodiment, the direction of the first specific angle AC1Z is predetermined to be the direction associated with the typical first angle AA1 (in this embodiment, the +Dx direction). Hereinafter, the +Dx direction will also be referred to as the first reference direction. The angle correction method is the same as the correction method in S150 ( FIG. 5 ) and FIG. 4(F). FIG. 14(E) shows an example of correction of the first angle AC1. The processor 210 generates data for a corrected block image Bc4 by rotating the block image Bc3 so that the first angle AC1 becomes the same as the first specific angle AC1Z.
[0117] In steps S140c and S150c (FIG. 12), the processor 210 corrects the angle based on the rotation of the block image. In steps S160c and S170c (FIG. 12), the processor 210 corrects the angle based on the shear (also called skew) of the corrected block image.
[0118] In S160c, the processor 210 estimates a second angle AC2 of each corrected block image. The second angle AC2 indicates a direction approximately parallel to the second direction Dy. The block image to be processed is the block image corrected in S150c.
[0119] 15 is a flowchart showing an example of a process for estimating the second angle AC2. In S410c2, the processor 210 acquires a binarized image of the corrected block image of S150c ( FIG. 12 ). The processor 210 may acquire data of the binarized image by binarizing the corrected block image. Alternatively, the processor 210 may acquire data of the binarized image of the corrected block image by rotating the binarized image generated in S410 ( FIG. 13 ) by the same angle as in S150c ( FIG. 12 ).
[0120] 14(F) to 14(J) are diagrams showing information processed in relation to the second angle AC2. FIG. 14(F) shows an example of a portion of a binarized block image. The binarized image Bc2r shows a plurality of first portions Pc1, which are bright island-like portions, and second portions Pc2, which are the remaining dark portions.
[0121] FIG. 14(F) shows a second pattern PTc2 composed of two first portions Pc1 (e.g., first portions Pc21 and Pc22) and a second portion Pc2 surrounding the first portions Pc1. This second pattern PTc2 is an example of a pattern formed by threads Wf and Wp. Although not shown, the multiple second patterns PTc2 are regularly arranged in a direction roughly parallel to the -Dy direction. The multiple second patterns PTc2 are linearly arranged in a direction roughly parallel to the -Dy direction to form lines PTLc2. The second direction Dc2 is the direction in which the lines PTLc2 extend. Although not shown, the multiple lines PTLc2 are arranged in the first direction Dx on the binarized image Bc2r. The multiple lines PTLc2 are also arranged in the first direction Dx on the captured image.
[0122] S420 (FIG. 15) is the same as S420 in FIG. 5. In this embodiment, the island-shaped first portion Pc1 is used as the feature point. The processor 210 selects the unprocessed first portion Pc1 as the target feature point.
[0123] In S430c2, the processor 210 selects a feature point from a search area located in the -Dy direction of the target feature point. FIG. 14(G) shows the target feature point Pc21 and the selection area SA21. In this embodiment, the selection area SA21 is a circular area. The center Pc21s of the selection area SA21 is located at a distance Ds2 in the -Dy direction from the center of gravity position Pc21c of the target feature point Pc21. The value Rc2 indicates the radius of the selection area SA21. As shown in FIG. 14(F), the selection area SA21 associated with the target feature point Pc21 includes a first portion Pc22. The processor 210 selects the first portion Pc22 from this selection area SA21.
[0124] The method of determining the distance Ds2 and the radius Rc2 is the same as the method of determining the distance Ds1 and the radius Rc1 in Fig. 14(B) . In addition, the processor 210 selects the first portion Pc1 that is included in the selection area SA21 and has the largest size (e.g., the largest number of pixels) from one or more first portions Pc1 that contact the selection area SA21.
[0125] In S440c2, the processor 210 calculates the angle AC2v of the vector Vc2 extending from the starting point to the end point. FIG. 14(H) shows an example of the vector Vc2. The method for determining the vector Vc2 is the same as the method for determining the vector Va in FIG. 4(C). The starting point Pc2s of the vector Vc2 is the center of gravity of the target feature point Pc1 (e.g., the first portion Pc21) selected in S420 (FIG. 15), and the end point Pc2e of the vector Vc2 is the center of gravity of the feature point Pc2 (e.g., the first portion Pc22) selected in S430c2. The method for calculating the angle AC2v is the same as the method for calculating the angle AA1v in FIG. 4(C). In this embodiment, the reference direction of the angle AC2v is the -Dy direction.
[0126] S450 (Fig. 15) is the same as S450 in Fig. 5. If unprocessed feature points remain (S450: No), the processor 210 proceeds to S420 to process new feature points.
[0127] If processing of all feature points is complete (S450: Yes), in S460c2, the processor 210 estimates a second angle AC2 of the block image using the multiple angles AC2v of the multiple vectors Vc2. The method of estimating the second angle AC2 is the same as the method of estimating the first angle AA1 in S460 of FIG. 5. FIG. 14I shows an example of a histogram of the angles AC2v. The processor 210 adopts the angle associated with the maximum frequency F as the second angle AC2. The method of estimating the second angle AC2 may be various other methods (for example, a method using an approximation curve Ccv2, a method using various summary statistics (average, median, mode, etc.) of the multiple angles AC2v, etc.).
[0128] 14(F), vector Vc2 indicates the direction in which two first portions Pc1 (e.g., feature points Pc21 and Pc22) that form second pattern PTc2 are arranged. The multiple second patterns PTc2 are arranged regularly in approximately the same direction as vector Vc2. Second angle AC2 indicates the direction in which the multiple second patterns PTc2 are arranged regularly (i.e., second direction Dc2, which is the direction in which line PTLc2 formed by the multiple second patterns PTc2 extends).
[0129] After estimating the second angle AC2 (FIG. 15: S460c2), the processor 210 ends the processing in FIG. 15. Note that in S160c (FIG. 12), the processor 210 estimates the second angle AC2 for each of the multiple block images.
[0130] In S170c, the processor 210 corrects the angle of each corrected block image so that the second angle AC2 becomes a second specific angle AC2Z. In this embodiment, the direction of the second specific angle AC2Z is predetermined to be a direction associated with a typical second angle AA2 (in this embodiment, the -Dy direction). Hereinafter, the -Dy direction will also be referred to as the second reference direction. The processor 210 also corrects the second angle AC2 by shearing (also called skew) the block image.
[0131] FIG. 14(J) illustrates an example of correcting the second angle AC2. The processor 210 generates data for a new corrected block image Bc5 by shearing the corrected block image Bc4 generated in S150c ( FIG. 12 ) so that the second angle AC2 is equal to the second specific angle AC2Z (shearing transformation is a type of affine transformation). Various processes can be employed for the shearing transformation. For example, the processor 210 may move a pixel line (e.g., pixel line PLc) parallel to the first direction Dx in the corrected block image Bc4 parallel to the first direction Dx. The amount of movement may be set to a value proportional to the distance from a predetermined reference pixel line (e.g., pixel line PLcs at the end of the +Dy direction). The proportionality coefficient indicates the amount of change in the second angle AC2 due to the shearing transformation and is associated with the difference between the second angle AC2 and the second specific angle AC2Z. If the shear transformation of the image results in missing portions (e.g., portion P5c), the processor 210 may fill in the missing portions using the original captured image (S110 (FIG. 12)), as described in S150 (FIG. 3).
[0132] As described above, angle correction based on rotation (S140c, S150c) and angle correction based on shear (S160c, S170c) are performed. Hereinafter, the corrected block images obtained by these processes are referred to as training block images. The processor 210 generates data for N training block images. As in the embodiment of FIG. 3, the processor 210 may acquire data for multiple captured images in S110 to generate N training block images. The processor 210 may also increase the total number of training block images through data augmentation. The data augmentation process may include various processes such as enlargement, reduction, and rotation.
[0133] S290 is the same as S290 in Fig. 3. The processor 210 trains the anomaly detection model 310 (here, the Gaussian matrix G M ) using the N training block images. Then, the processor 210 ends the processing in Fig. 12.
[0134] C2. Inspection Process: Fig. 16 is a flowchart showing an example of the inspection process of the third embodiment. The main difference from the inspection process of Fig. 7 is that in addition to correcting the first angle AC1 by rotation, correcting the second angle AC2 by shear transformation is also performed.
[0135] S710, S720, and S730 are the same as S710, S720, and S730, respectively, in Fig. 7. The processor 210 divides the captured image into a plurality of block images.
[0136] In S740c, the processor 210 estimates the first angle AC1 of each of the plurality of block images. The method of estimating the first angle AC1 is the same as the method of estimating the first angle AC1 in S140c (FIG. 12).
[0137] In S750c, the processor 210 corrects the angle of each block image so that the first angle AC1 becomes the first specific angle AC1Z. The correction method is the same as the correction method in S150c (FIG. 12) (here, image rotation).
[0138] In S760c, the processor 210 estimates the second angle AC2 for each of the plurality of corrected block images obtained in S750c. The method for estimating the second angle AC2 is the same as the method for estimating the second angle AC2 in S160c (FIG. 12).
[0139] In S770c, the processor 210 corrects the angle of each corrected block image so that the second angle AC2 becomes the second specific angle AC2Z. The correction method is the same as the correction method in S170c (FIG. 12) (here, shear transformation of the image).
[0140] S890 is the same as S890 in Fig. 7. The processor 210 inputs each of the multiple corrected block images generated by S740c-S770c to the anomaly detection model 310, and determines whether the target fabric 700 has an anomaly.
[0141] S894, S896, and S898 after S890 are the same as S894, S896, and S898, respectively, in Fig. 7. If the determination result in S896 is No, the processor 210 proceeds to S710.
[0142] As described above, in this embodiment, the processor 210 executes a correction process to determine whether or not the target fabric 700 has a defect ( FIG. 16 ). The correction process includes a correction of S750c (referred to as the first correction) and a correction of S770c (referred to as the second correction). The first correction (S750c) is a correction of the first angle AC1 by rotating the image, as in the embodiment of FIG. 7 . The second correction (S770c) is a correction of the second angle AC2 by shearing the image. Specifically, as shown in FIG. 14(J), the second correction adjusts the second angle AC2 of the second direction Dc2 relative to the second reference direction (here, the −Dy direction) to approach the second specific angle AC2Z by shearing the affine transformation. As described in FIG. 14(F), the second angle AC2 indicates the second direction Dc2, which is the direction in which the multiple second patterns PTc2 in the image representing the target fabric 700 are regularly arranged. The second pattern PTc2 is formed by the yarns Wf and Wp. In S890, the processor 210 inputs the block images processed by the correction process into the anomaly detection model 310 to determine whether or not there is a defect in the target fabric 700. As described in S170c and S290 of FIG. 12, the anomaly detection model 310 (specifically, the Gaussian matrix GM) is trained using multiple training images (here, training block images) representing the training fabric 700t. The multiple training images include multiple training images corrected by S170c ( FIG. 12 ). The correction by S170c, as shown in FIG. 14(J), approximates the second angle AC2 of the second direction Dc2 of the training image relative to the second reference direction (here, the −Dy direction) to a second specific angle AC2Z by shearing an affine transformation.
[0143] According to this configuration, even if the second angle AC2 indicated by the target fabric 700 represented by the block image deviates from the second specific angle AC2Z, the block image is corrected so that the second angle AC2 becomes the second specific angle AC2Z (the block image is an example of a captured image). The training images also include training images in which the second angle AC2 of the training images has been corrected to approach the second specific angle AC2Z. Therefore, by using the anomaly detection model 310, the processor 210 can appropriately determine whether or not the target fabric 700 has a defect.
[0144] D. Fourth Embodiment: D1. Training Process: Figure 17 is a flowchart showing an example of the training process of the fourth embodiment. The difference from the embodiment of Figure 9 is that in addition to the data augmentation that changes the first angle AC1, data augmentation that changes the second angle AC2 is performed.
[0145] S110 and S130 are the same as S110 and S130, respectively, in Fig. 9. The processor 210 divides the captured image into a plurality of block images.
[0146] S140c is the same as S140c in Fig. 12. The processor 210 estimates the first angle AC1 of each block image.
[0147] In S142d, the processor 210 determines a first angle range AC1R using the distribution of the multiple first angles AC1. FIG. 18A is a histogram showing an example of the distribution of the first angles AC1. The processor 210 determines the first angle range AC1R (specifically, a lower limit AC1L and an upper limit AC1H) so as to include most of the multiple first angles AC1. The method for determining the first angle range AC1R is the same as the method for determining the angle range AA1R in S142b of FIG. 9. The first angle range AC1R may be determined using an approximation curve Cd1 that approximates the histogram of the first angles AC1.
[0148] In S144d (FIG. 17), the processor 210 determines a first specific angle AC1Z. In this embodiment, the processor 210 uses the center value of the first angle range AC1R (i.e., the average value of the lower limit AC1L and the upper limit AC1H) as the first specific angle AC1Z.
[0149] S150c is the same as S150c in Fig. 12. The processor 210 corrects the angle of each block image so that the first angle AC1 becomes the first specific angle AC1Z.
[0150] S160c is the same as S160c in Fig. 12. The processor 210 estimates the second angle AC2 of each corrected block image.
[0151] In S162d, the processor 210 determines a second angle range AC2R using the distribution of the multiple second angles AC2. FIG. 18(B) is a histogram showing an example of the distribution of the second angles AC2. The processor 210 determines the second angle range AC2R (specifically, a lower limit AC2L and an upper limit AC2H) so as to include most of the multiple second angles AC2. The method for determining the second angle range AC2R is the same as the method for determining the first angle range AC1R in S142d. The second angle range AC2R may be determined using an approximation curve Cd2 that approximates the histogram of the second angles AC2.
[0152] In S164d (FIG. 17), the processor 210 determines the second specific angle AC2Z. In this embodiment, the processor 210 uses the center value of the second angle range AC2R (i.e., the average value of the lower limit AC2L and the upper limit AC2H) as the second specific angle AC2Z.
[0153] S170c is the same as S170c in Fig. 12. The processor 210 corrects the angle of each corrected block image so that the second angle AC2 becomes the second specific angle AC2Z.
[0154] In S172d, the processor 210 generates data for a plurality of training block images by data augmentation. In this embodiment, the processor 210 generates data for a plurality of training block images by performing one or both of a rotation and a shear transformation on the corrected block images generated in S170c.
[0155] Here, processor 210 generates data for the multiple training block images so that the multiple first angles AC1 of the multiple training block images ultimately obtained are approximately uniformly distributed within a first angle range AC1R, and the multiple second angles AC2 are approximately uniformly distributed within a second angle range AC2R. For example, processor 210 randomly determines the angle of rotation for each of the multiple training block images so that the multiple angles (corresponding to the multiple first angles AC1) obtained from the first specific angle AC1Z by the rotation are approximately uniformly distributed within the first angle range AC1R. Processor 210 also randomly determines the amount of change in angle due to the shear transformation for each of the multiple training block images so that the multiple angles (corresponding to the multiple second angles AC2) obtained from the second specific angle AC2Z by the shear transformation are approximately uniformly distributed within the second angle range AC2R.
[0156] As described above, in S170c, data for a plurality of corrected block images is generated. The processor 210 uses the plurality of corrected block images to generate data for a plurality of training block images. If a missing portion (e.g., portion P5c in FIG. 14(J)) occurs due to the rotation or shear transformation, the processor 210 may fill in the missing portion using the original captured image (S110).
[0157] Furthermore, the processor 210 may acquire data of a plurality of captured images in S110 in order to generate N training block images, similar to the embodiment of FIG.
[0158] S290 is the same as S290 in Fig. 3. The processor 210 trains the anomaly detection model 310 (here, the Gaussian matrix G M ) using the N training block images. Then, the processor 210 ends the processing in Fig. 17.
[0159] The inspection process of this embodiment is the same as the inspection process of the embodiment of FIG. 11 . Thus, in this embodiment, the processor 210 performs the following process to determine whether or not a defect exists in the target fabric 700. In S720 and S730 ( FIG. 11 ), the processor 210 acquires data of block images representing photographed images of the target fabric 700. In S890b, the processor 210 inputs the block images into the anomaly detection model 310 to determine whether or not a defect exists in the target fabric. As described in FIG. 17 , the anomaly detection model 310 is trained using a plurality of training block images representing the training fabric 700t. The data of the plurality of training block images is generated by data augmentation in S172d.
[0160] In this embodiment, the first angle AC1 of the training block image is changed by data augmentation. As described in FIGS. 14(A) and 14(E), the first angle AC1 indicates the direction in which the multiple first patterns PTc1 are regularly arranged in the image of the textile 700 (i.e., the first direction Dc1 in which the lines PTLc1 formed by the multiple first patterns PTc1 extend). The first pattern PTc1 is formed by the yarns Wf and Wp. As described in S172d, the multiple training block images are configured such that the distribution range of the first angles AC1 of the multiple first directions Dc1 of the multiple training block images relative to the reference direction (here, the +Dx direction) is a first angle range AC1R. As described in S172d and FIG. 18(A), the first angle range AC1R is determined using the distribution of the first angles AC1 estimated using the captured images of the training textile 700t. Here, the training fabric 700t is supported by a conveying device 900 that conveys the target fabric 700 for photographing the target fabric 700. Also, a first angle AC1 indicates the angle of a first direction Dc1 based on the rotation of the fabric 700 (e.g., the training fabric 700t).
[0161] According to this configuration, the training block images are configured so that the distribution range of the first angles AC1 in the training block images is the first angle range AC1R, and the first angle range AC1R is determined using the distribution of the first angles AC1 estimated using the captured images. Therefore, even if the first angle AC1 indicated by the target textile 700 represented by the captured images changes, the processor 210 can appropriately determine the presence or absence of a defect in the target textile 700 by using the anomaly detection model 310. Furthermore, it is easy to avoid using a first angle range AC1R that is wider than the width of the actual distribution of the first angles AC1. As a result, the burden of training the anomaly detection model 310 can be reduced.
[0162] Furthermore, in this embodiment, the second angle AC2 of the training block image changes through data augmentation. As described in FIGS. 14(F) and 14(J), the second angle AC2 indicates the direction in which the multiple second patterns PTc2 are regularly arranged in the image of the textile 700 (i.e., the second direction Dc2 in which the line PTLc2 formed by the multiple second patterns PTc2 extends). The second pattern PTc2 is formed by the yarns Wf and Wp. As described in S172d, the multiple training block images are configured such that the distribution range of the second angles AC2 of the multiple second directions Dc2 of the multiple training block images relative to the reference direction (here, the -Dy direction) is the second angle range AC2R. As described in S172d and FIG. 18(B), the second angle range AC2R is determined using the distribution of the second angles AC2 estimated using the captured images of the training textile 700t. Here, the training fabric 700t is supported by a conveying device 900 that conveys the target fabric 700 for photographing the target fabric 700. Also, a second angle AC2 indicates the angle of a second direction Dc2 based on the skew of the fabric 700 (e.g., the training fabric 700t).
[0163] According to this configuration, the training block images are configured so that the distribution range of the second angles AC2 in the training block images is the second angle range AC2R, and the second angle range AC2R is determined using the distribution of the second angles AC2 estimated using the captured images. Therefore, even if the second angles AC2 indicated by the target textile 700 represented by the captured images change, the processor 210 can appropriately determine the presence or absence of a defect in the target textile 700 by using the anomaly detection model 310. Furthermore, it is easy to avoid using a second angle range AC2R that is wider than the width of the actual distribution of the second angles AC2. As a result, the burden of training the anomaly detection model 310 can be reduced.
[0164] E. Fifth Example: E1. Training Process: FIG. 19 is a flowchart showing an example of the training process of the fifth example. In this example, the processor 210 trains the distortion correction model 320 in addition to the anomaly detection model 310. Because the textile 700 is soft, it may be locally deformed. If a captured image shows the textile 700 with locally deformed features, the deformation may reduce the accuracy of the inspection using the anomaly detection model 310. Therefore, in this example, the processor 210 uses the distortion correction model 320 in the inspection process to correct the local deformation (i.e., distortion) of the textile 700 in the captured image. In the training process, the distortion correction model 320 is trained in addition to the anomaly detection model 310.
[0165] The operator attaches the training fabric 700t, which is free of defects and has little distortion, to the conveying device 900. The operator may straighten out any distortions in the training fabric 700t (also called straightening) before attaching the training fabric 700t to the conveying device 900. Alternatively, the operator may straighten out any distortions in the training fabric 700t after attaching the training fabric 700t to the conveying device 900. In either case, any distortions that are small enough to have an acceptable effect on the inspection may remain. Then, the operator operates the operation unit 250 of the data processing device 200 (FIG. 1) to input an instruction to start the training process. In response to the instruction, the processor 210 starts the training process in accordance with the training program 232.
[0166] In S110e, the processor 210 provides a photographing instruction to the digital camera 110. The digital camera 110 photographs the training fabric 700t. The processor 210 acquires photographed image data from the digital camera 110.
[0167] Steps S130 and S290 are the same as steps S130 and S290 in Fig. 3, respectively. This trains the anomaly detection model 310. In this embodiment, distortions including rotation and skew can be corrected by the distortion correction model 320. Therefore, corrections for rotation and skew are omitted.
[0168] In S310e, the processor 210 pastes a defect image into a portion of the block image obtained in S130 (the block image generated by pasting a defect image is referred to as a first-type image). FIGS. 20A-20E are diagrams showing information processed in the training process. FIG. 20A shows an example of a block image. This block image Beo represents a fabric 700t that is free of abnormalities and has little distortion. FIG. 20B shows an example of a first-type image. Data for the first-type image Be1 is generated by pasting an image of the defect De into the block image Beo. A grid is shown by dotted lines on the first-type image Be1. The multiple grid points CP are used in the distortion imparting process, which will be described later.
[0169] Although not shown in the drawings, in this embodiment, data on a plurality of defect images representing defects different from each other is stored in advance in the non-volatile storage device 230. In S310e, the processor 210 executes the following processing. The processor 210 randomly selects a defect image from a plurality of available defect images. The processor 210 randomly selects a block image to be processed from a plurality of block images. The processor 210 randomly determines a position within the selected block image. Then, the processor 210 generates data of a first-type image by pasting the selected defect image at the determined position.
[0170] In S320e (FIG. 19), the processor 210 randomly generates a displacement vector field (e.g., using random numbers). FIG. 20(C) shows an example of the displacement vector field. In this embodiment, the displacement vector field VFe represents the displacement vectors Ve of each of a plurality of grid points CP (also referred to as control points CP) on the first-type image Be1. In this embodiment, the control points CP are arranged in a grid pattern along the first direction Dx and the second direction Dy. The processor 210 randomly determines the displacement vectors Ve of the control points CP. The positions of each of the control points CP on the first-type image Be1 (and thus the block image) are predetermined. In addition, the density of the control points CP (e.g., the number of control points CP per unit area) is experimentally determined in advance so that the displacement vector field VFe appropriately represents the distortion that may actually occur in the target textile 700.
[0171] If multiple displacement vectors V e are determined independently and randomly, the displacement vector field V Fe may exhibit unrealistic distortion for the fabric 700. For example, the difference between the two displacement vectors V e of two adjacent control points CP may be excessively large. Therefore, in this embodiment, the processor 210 adjusts the displacement vectors V e so that the difference between the two displacement vectors V e of two adjacent control points CP is equal to or smaller than a reference value. The control points CP adjacent to the control point CP of interest may be the so-called four-neighbor control points CP.
[0172] 20(D) shows the difference in displacement vector Ve. The figure shows two adjacent control points CPa and CPb, and the displacement vectors Vea and Veb of the control points CPa and CPb. The difference vector dVe is calculated by Vea-Veb. The processor 210 determines multiple displacement vectors Ve so that the following displacement condition is satisfied. (Displacement condition) The length of each difference vector dVe for all combinations of two adjacent control points CP is equal to or less than the reference THe.
[0173] Various methods may be used to determine the multiple displacement vectors Ve. For example, the processor 210 first randomly determines each of the multiple displacement vectors Ve of the multiple control points CP. The length of each displacement vector Ve is limited to an upper limit value or less. The upper limit value is determined experimentally in advance so that the upper limit value represents the displacement (particularly the maximum displacement) that can be formed by the actual distortion of the target fabric 700. Next, the processor 210 selects a pair of two adjacent control points CP to be processed (referred to as a target pair). If the length of the difference vector dVe of the target pair exceeds a reference THe, the processor 210 adjusts one displacement vector Ve of one control point CP of the target pair so that the length of the difference vector dVe is equal to or less than the reference THe. The adjustment of the displacement vector Ve may include changing the direction of the displacement vector Ve and / or the length of the displacement vector Ve. The processor 210 adjusts the displacement vector Ve while changing the target pair until the displacement condition is satisfied.
[0174] Alternatively, the processor 210 may perform the following process: The processor 210 selects a plurality of control points CP from the plurality of control points CP at a low density (i.e., roughly) (referred to as target control points CP). For example, the processor 210 may select one control point CP at intervals of 10 control points CP along the first direction Dx, and one control point CP at intervals of 10 control points CP along the second direction Dy. The processor 210 randomly determines a provisional displacement vector for each of the selected target control points CP. The length of the provisional displacement vector is limited to an upper limit value or less. The processor 210 determines a plurality of displacement vectors Ve of the plurality of control points CP by interpolation (e.g., cubic B-spline interpolation) using the provisional displacement vectors. Here, the components in the first direction Dx and the components in the second direction Dy are interpolated, respectively. Interpolation using multiple tentative displacement vectors of multiple coarsely selected target control points CP reduces the possibility of generating a difference vector dVe with an excessively large length. Processor 210 may repeat the process of adjusting the upper limit on the length of the tentative displacement vector, determining the tentative displacement vector, and interpolating until the displacement condition is satisfied.
[0175] In S330e (FIG. 19), the processor 210 applies distortion to the first-type image using the displacement vector field VFe (the image generated by applying distortion is referred to as the second-type image). FIG. 20(E) shows an example of the second-type image. Data for the second-type image Be2 is generated by applying distortion to the first-type image Be1 (FIG. 20(B)). The second-type image Be2 generated in this manner represents the same defect De as the defect De in the first-type image Be1 (FIG. 20(B)) (however, the defect De may be distorted on the second-type image Be2). Control points CP that have moved in accordance with the displacement vector field VFe are shown on the second-type image Be2.
[0176] Various methods may be used to apply distortion using the displacement vector field VFe. The processor 210 may determine the color value of each pixel of the second-type image Be2 by interpolation (e.g., interpolation using a cubic B-spline) using multiple displacement vectors Ve represented by the displacement vector field VFe. For example, the processor 210 may displace each of multiple pixels of the first-type image Be1 according to the displacement vector field VFe. Then, the processor 210 may calculate the color value of a pixel of interest in the second-type image Be2 by interpolation using multiple displaced pixels located in the vicinity of the pixel of interest.
[0177] In this embodiment, the processor 210 ultimately generates data of M pairs (M is an integer equal to or greater than 2) of first-type images and second-type images (referred to as training pairs). The number M of training pairs may be any number suitable for training the distortion correction model 320 (e.g., approximately several hundred or several thousand). To generate M training pairs, the processor 210 may perform the processes of S310e to S330e on M block images. Furthermore, to obtain M block images, the processor 210 may obtain data of multiple captured images in S110e. Then, in S130, the processor 210 may obtain multiple block images from each of the multiple captured images.
[0178] In S380e (FIG. 19), the processor 210 trains the distortion correction model 320 using the M training pairs. FIG. 21 is a block diagram illustrating an example of the distortion correction model 320. In this embodiment, the distortion correction model 320 is configured using a distortion correction technique called DIRNet (End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network). DIRNet is disclosed, for example, in the paper "Bob D. de Vos, Floris F. Berendsen, Max A. Viergever, Marius Staring, Ivana Isgum, "End-to-end unsupervised deformable image registration with a convolutional neural network", arXiv:1704.06065(2017), https: / / arxiv.org / abs / 1704.06065."
[0179] The DIRNet technology generates a displacement vector field DVF to warp (i.e., deform) a moving image E2 onto a fixed image E1. This displacement vector field DVF corrects the distortion of the moving image E2 relative to the fixed image E1. In the training process of this embodiment, a second-type image (e.g., a second-type image Be2) is used as the moving image E2, and a first-type image (e.g., a first-type image Be1) is used as the fixed image E1.
[0180] The distortion correction model 320 includes a convolutional net regressor 320a, a spatial transformer 320b, and a resampler 320c.
[0181] The convolutional net regressor 320a includes multiple convolutional networks (not shown). Data obtained by concatenating data from fixed image E1 and data from moved image E2 is input to the convolutional net regressor 320a. The convolutional net regressor 320a analyzes pairs of spatially corresponding (i.e., co-located) patches EP1, EP2, each pair consisting of a patch EP1 from fixed image E1 and a patch EP2 from moved image E2. The convolutional net regressor 320a is trained to output two-dimensional displacements of control points at positions corresponding to the pair of patches EP1, EP2. The convolutional net regressor 320a outputs the displacements of each of the control points for the pairs of patches EP1, EP2. The set 320ao of displacements of the control points is used by the spatial transformer 320b.
[0182] The spatial transformer 320b uses the displacement set 320ao to generate a displacement vector field DVF that represents dense displacements (e.g., displacements of all pixels) that enable the resampler 320c to warp the image. The displacement vector field DVF can be generated in various ways. In this embodiment, the spatial transformer 320b generates the displacement vector field DVF by interpolating with cubic B-splines.
[0183] The resampler 320c generates data for the corrected image E3 by warping the moved image E2 according to the displacement vector field DVF. Various methods for warping an image using the displacement vector field DVF may be used. For example, the processor 210 may displace each of multiple pixels in the moved image E2 according to the displacement vector field DVF. The processor 210 may then calculate the color value of a pixel of interest in the corrected image E3 by interpolation using multiple displaced pixels located in the vicinity of the pixel of interest.
[0184] The training method for the distortion correction model 320 may be any method suitable for the distortion correction model 320, such as the method described in the above-mentioned DIRNet paper. The processor 210 may train the distortion correction model 320, for example, as follows: The processor 210 obtains a subset consisting of multiple training pairs from M sets of training pairs of a first type of image and a second type of image. The processor 210 generates data for the corrected image E3 by performing calculations on the distortion correction model 320 using data for the training pairs included in the subset. The processor 210 calculates the image similarity SM using data for the first type of image (i.e., the fixed image E1) and data for the corrected image E3. The image similarity SM may be various values indicating the similarity between the two images E1 and E3. For example, the image similarity SM may be normalized cross correlation.
[0185] The processor 210 calculates a plurality of image similarities SM using a plurality of training pairs of the subset. The processor 210 calculates a loss using the plurality of image similarities SM. The loss may be various values that are larger as the image similarities SM are smaller. For example, the loss may be the sum of the differences between the maximum value of the image similarities SM (here, 1) and the image similarities SM.
[0186] The processor 210 adjusts a plurality of operation parameters (convolution filter, bias, etc.) of the convolution net regressor 320a so as to reduce the loss. For the adjustment, the backpropagation method and the gradient descent method may be used, for example.
[0187] After the adjustment, the processor 210 determines whether a training termination condition is met. The training termination condition may be any condition indicating that the distortion correction model 320 has been properly trained. For example, the termination condition may be that the loss calculated using a predetermined number of training pairs not used in training is equal to or less than a predetermined loss threshold. Alternatively, the training termination condition may be that an operator inputs an instruction to terminate training. The operator may input an instruction to terminate training or an instruction to continue training to the operation unit 250 by comparing the corrected image E3 obtained from the training pairs not used in training with the fixed image E1.
[0188] The processor 210 repeats the adjustment using the new subset until the training termination condition is met. When the training termination condition is met, the processor 210 stores the data of the trained distortion correction model 320 in the storage device 215 (here, the non-volatile storage device 230). This completes the training of the distortion correction model 320, i.e., S380e (FIG. 19). Then, the processing of FIG. 19 ends.
[0189] E2. Inspection Process: Fig. 22 is a flowchart showing an example of the inspection process of Example 5. The only difference from the inspection process of Fig. 7 is that distortion correction (S810e) is performed instead of the correction of the first angle AA1 (S740, S750).
[0190] S710, S720, and S730 are the same as S710, S720, and S730, respectively, in Fig. 7. The processor 210 divides the captured image into a plurality of block images.
[0191] In S810e, the processor 210 corrects the distortion of each of the multiple block images using the distortion correction model 320 ( FIG. 21 ). In this embodiment, data of a reference block image representing the undistorted textile 700 is stored in advance in the non-volatile storage device 230 (not shown). The processor 210 generates data of a corrected image E3 using a pair of a block image and a reference block image. The block image is used as a moved image E2, and the reference block image is used as a fixed image E1. The distortion correction model 320 generates data of the corrected image E3 by correcting the distortion of the block image relative to the reference block image. If the moved image E2 (block image) represents a defect, the corrected image E3 represents the same defect. If the moved image E2 (block image) does not represent a defect, the corrected image E3 does not represent a defect.
[0192] S890 is the same as S890 in Fig. 7. The processor 210 inputs each of the multiple corrected images obtained in S810e into the anomaly detection model 310 and determines whether the target textile 700 has an abnormality. S894, S896, and S898 after S890 are the same as S894, S896, and S898 in Fig. 7, respectively. If the determination result in S896 is No, the processor 210 proceeds to S710.
[0193] As described above, in this embodiment, the processor 210 executes the following process to determine whether or not the target fabric 700 has a defect. In S720 and S730 (FIGS. 22 and 11), the processor 210 acquires block image data representing a captured image of the target fabric 700. In S810e, the processor 210 performs a correction process for the block image. In S890, the processor 210 inputs the image processed by the correction process into the anomaly detection model 310 to determine whether or not the target fabric 700 has a defect. Here, the correction process (S810e) includes distortion correction, which corrects distortion of the target fabric 700 in the image representing the target fabric 700, as described with reference to FIG. 21. Therefore, even if the target fabric 700 in the captured image has distortion, the processor 210 can appropriately determine whether or not the target fabric 700 has a defect by using the anomaly detection model 310.
[0194] In this embodiment, distortion correction (S810e) includes distortion correction using the distortion correction model 320 ( FIG. 21 ), a machine learning model that corrects distortion. As described in S330e and S380e of FIG. 19 and FIG. 21 , the distortion correction model 320 is trained to generate a corrected image E3 having a high image similarity SM with the first-type image Be1 from a second-type image Be2 obtained by applying distortion to a first-type image Be1 representing the training fabric 700t. In other words, the distortion correction model 320 is trained to generate the first-type image Be1 from the second-type image Be2. Therefore, the distortion correction model 320 can appropriately correct image distortion.
[0195] In this embodiment, as described in S310e of Fig. 19 and Fig. 20(B), the first type image Be1 is obtained by adding an image of a defect (e.g., an image of the defect De) to a portion of the block image representing the training fabric 700t. As described in Fig. 20(E), the second type image Be2 represents the same defect De as the defect De represented by the first type image Be1. Therefore, the distortion correction model 320 can appropriately correct the distortion of the captured image for inspection, even if the captured image represents the target fabric 700 having a defect.
[0196] In this embodiment, the distortion correction model 320 does not directly generate the corrected image E3, but generates the corrected image E3 by deforming the moved image E2. Therefore, if the moved image E2 does not represent a defect, a corrected image E3 that does not represent the defect is generated. If the moved image E2 represents a defect, a corrected image E3 that represents the same defect is generated. Therefore, adding an image of the defect to the first type of training image Be1 (S310e) may be omitted.
[0197] In this embodiment, as described in S320e and S330e of FIG. 19, distortion is applied according to the respective displacement vectors Ve of the multiple control points CP arranged in the first-type image Be1 (FIG. 20C). As described in FIG. 20D, the multiple displacement vectors Ve of the multiple target control points CP are configured so that the difference between the displacement vectors Ve of adjacent control points CP is equal to or less than a reference value. In this embodiment, the multiple displacement vectors Ve are configured so that the length of the difference vector dVe between two adjacent control points CP is equal to or less than a reference value THe. Therefore, the processor 210 can apply distortion that may actually occur in the textile 700 to the first-type image Be1. As a result, the distortion correction model 320 is trained to correct distortion that may actually occur.
[0198] F. Sixth Example: F1. Training Process: FIG. 23 is a flowchart showing an example of the training process of the sixth example. In this example, the distortion correction model 320 is omitted. The processor 210 corrects distortion of the fabric 700 in the captured image in the inspection process described below without using the distortion correction model 320. The training process of this example is the same as the training process of FIG. 19 except that S310e-S380e for the distortion correction model 320 are omitted. The processor 210 trains the anomaly detection model 310 through S110e, S130, and S290.
[0199] F2. Inspection Process: Fig. 24 is a flowchart showing an example of the inspection process of Example 6. The only difference from the inspection process of Fig. 22 is that the distortion correction process using the distortion correction model 320 (S810e) is replaced with distortion correction process using keypoint matching (S810f).
[0200] S710, S720, and S730 are the same as S710, S720, and S730, respectively, in Fig. 22. The processor 210 divides the captured image into a plurality of block images.
[0201] In step S810f, the processor 210 corrects distortion in each of the plurality of block images using keypoint matching. The distortion correction using keypoint matching may be various processes. In this embodiment, step S810f includes steps S812f, S814f, and S816f.
[0202] In S812f, the processor 210 forms multiple matching pairs between the block image and the reference block image through keypoint matching. FIG. 25 illustrates an example of distortion correction using keypoint matching. The figure shows a block image Bf1 and a reference block image Bfs. The block image Bf1 is a block image generated in S730 (FIG. 24) and represents the target textile 700. In the block image Bf1, the target textile 700 may have distortion. Furthermore, the block image Bf1 may represent a defect (e.g., defect Df). The reference block image Bfs is an image representing the textile 700 that is free of defects and has minimal distortion (e.g., a photographed image of the textile 700). In this embodiment, data for the reference block image Bfs is stored in advance in the non-volatile storage device 230 (not illustrated). The reference block image Bfs may include a small distortion that has an acceptable impact on inspection.
[0203] The processor 210 detects key points from each of the block image Bf1 and the reference block image Bfs. Key points are points expected to be highly unique within the image. Multiple lines RL and RLz are shown in the figure. The ends of these lines RL and RLz indicate key points. Hereinafter, the key point K1 detected from the block image Bf1 will be referred to as the captured key point K1. Furthermore, the key point Ks detected from the reference block image Bfs will be referred to as the reference key point Ks. In this embodiment, points representing characteristic portions, such as corners and edges, of the pattern formed by the yarns Wf and Wp of the fabric 700 ( FIG. 2 ) are detected as the key points K1 and Ks. Although not shown, in reality, many more key points K1 and Ks are detected (e.g., several tens or several hundreds).
[0204] Various methods may be used for detecting keypoints. For example, the detection method may be selected in advance from among a search for extrema (maximum and minimum) using Difference-of-Gaussian (DoG), Harris corner detection, and Features from Accelerated Segment Test (FAST) corner detection. Furthermore, an algorithm for detecting keypoints and calculating feature descriptors may be employed for detecting keypoints. The feature descriptor is information describing the features of the keypoints. The algorithm for detecting keypoints and calculating feature descriptors may be selected in advance from among Scale Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF), and Oriented Fast and Rotated Brief (ORB). In this embodiment, the processor 210 detects keypoints according to the ORB algorithm. To detect keypoints using the ORB algorithm, for example, the function "detect" or the function "detectAndCompute" of OpenCV (Open Source Computer Vision Library) can be used.
[0205] The processor 210 performs matching between the captured keypoint K1 and the reference keypoint Ks. Keypoint matching associates keypoints K1 and Ks that indicate the same part of the same subject (for example, the same corner of a pattern formed by the threads Wf and Wp). A pair of associated keypoints K1 and Ks is also called a matching pair.
[0206] The multiple lines RL in the figure indicate suitable matching pairs K1 and Ks. For the suitable matching pairs K1 and Ks, the position of the captured keypoint K1 in the block image Bf1 is close to the position of the reference keypoint Ks in the reference block image Bfs. The multiple lines RLz indicate unsuitable matching pairs K1 and Ks. For the unsuitable matching pairs K1 and Ks, the position of the captured keypoint K1 in the block image Bf1 is far from the position of the reference keypoint Ks in the reference block image Bfs.
[0207] Various methods may be used for keypoint matching. For example, the processor 210 may perform keypoint matching using feature descriptors of keypoints. The feature descriptors may be various information describing the features of the keypoints. For example, the feature descriptors are calculated so that they change depending on the distribution of color values of multiple pixels surrounding the keypoints. Various methods may be used for calculating the feature descriptors. For example, the algorithm for calculating the feature descriptors may be selected in advance from BRIEF (Binary Robust Independent Elementary Features), BRISK (Binary Robust Invariant Scalable Keypoints), SIFT, SURF, and ORB. In this embodiment, the processor 210 uses feature descriptors based on the ORB algorithm.
[0208] The processor 210 calculates a feature descriptor for each captured keypoint K1 by analyzing the data of the block image Bf1. The feature descriptor can be calculated based on the ORB algorithm using, for example, the OpenCV function "compute." The processor 210 may detect multiple keypoints K1 and calculate the feature descriptor for each keypoint K1 by executing the OpenCV function "detectAndCompute."
[0209] In addition, the feature descriptors of each reference keypoint Ks are calculated in advance. In this embodiment, data representing the multiple reference keypoints Ks and the feature descriptors of each reference keypoint Ks are stored in advance in the non-volatile storage device 230 (not shown).
[0210] Various methods of keypoint matching using feature descriptors may be used. The matching method may be selected in advance from, for example, FLANN (Fast Library for Approximate Nearest Neighbor)-based matching and brute-force matching. For FLANN-based matching, for example, matching using "cv2.FlannBasedMatcher" in OpenCV may be used. For brute-force matching, for example, matching using "cv2.BFMatcher" in OpenCV may be used. In this embodiment, the processor 210 performs brute-force matching.
[0211] In this embodiment, as described in FIG. 2 , the woven fabric 700 is configured so that the same pattern formed by the yarns Wf and Wp is regularly arranged. Therefore, keypoint pairs K1 and Ks that are significantly different in position within the images Bf1 and Bfs but represent similar portions of the patterns may be associated (e.g., a matching pair represented by the line RLz). Various methods can be employed to reduce the likelihood of forming such inappropriate matching pairs. For example, the processor 210 may calculate a homography matrix using multiple matching pairs and remove matching pairs classified as outliers based on the homography matrix. For example, the OpenCV function "cv2.findHomography" can be used for this process. This function can use an algorithm called RANSAC or LEAST_MEDIAN. Both algorithms classify multiple matching pairs into good pairs (inliers) and outliers. In FIG. 25 , a good matching pair K1 and Ks, as represented by the line RL, is likely to be classified as a good pair. Poor matching pairs K1, Ks, as represented by the line RLz, are likely to be classified as outliers. The processor 210 may adopt multiple keypoint pairs classified as good pairs as final matching pairs.
[0212] The keypoint matching method may be as follows: In this embodiment, distortion of the textile 700 is caused by local deformation of the textile 700. Therefore, there is a limit to the displacement (e.g., the displacement of the photographed keypoint K1) that can be formed by the actual distortion of the target textile 700. Therefore, the processor 210 may perform keypoint matching (e.g., keypoint matching based on feature descriptors) using the target photographed keypoint K1 and reference keypoints Ks included in a search area that includes the target photographed keypoint K1. The search area may be, for example, a circular area centered on the position of the target photographed keypoint K1. The radius is experimentally determined in advance so that the radius represents the displacement (particularly the maximum displacement) that can be formed by the actual distortion of the target textile 700. The processor 210 can form appropriate matching pairs by performing such keypoint matching on multiple photographed keypoints K1. Note that after this keypoint matching, the processor 210 may remove matching pairs that are classified as outliers based on the homography matrix.
[0213] In S814f (FIG. 24), the processor 210 generates a displacement vector field using multiple matching pairs. FIG. 25 shows an example of the displacement vector field VFf. In this embodiment, the processor 210 uses the photographic key point K1 of the matching pair K1, Ks as the control point CPf. The processor 210 uses the vector extending from the coordinate of the photographic key point K1 of the matching pair K1, Ks to the coordinate of the reference key point Ks as the displacement vector Vf. As shown in the figure, the displacement vector field VFf represents the displacement vectors Vf of each of the multiple control points CPf arranged on the block image Bf1.
[0214] In S816f (FIG. 24), the processor 210 corrects distortion in the block image Bf1 using the displacement vector field VFf. The correction method is the same as the distortion applying method using the displacement vector field VFe in S330e (FIG. 19). Applying distortion based on the displacement vector field VFf reduces the distortion contained in the block image Bf1. FIG. 25 shows an example of a corrected block image Bf2. In the corrected block image Bf2, distortion is reduced compared to that in the block image Bf1. If the block image Bf1 represents a defect (e.g., defect Df), the corrected block image Bf2 represents the same defect (however, the defect may be deformed from the defect in the block image Bf1).
[0215] S890 (FIG. 24) is the same as S890 in FIG. 22. The processor 210 inputs each of the multiple corrected block images obtained in S810f into the anomaly detection model 310 and determines whether the target textile 700 has an abnormality. S894, S896, and S898 after S890 are the same as S894, S896, and S898 in FIG. 7, respectively. If the determination result in S896 is No, the processor 210 proceeds to S710.
[0216] As described above, in this embodiment, distortion correction includes correction of distortion using the results of keypoint matching using a reference image (here, the reference block image Bfs). Therefore, even if the target fabric 700 in the captured image has distortion, the processor 210 can appropriately determine whether or not there is a defect in the target fabric 700 by using the anomaly detection model 310.
[0217] G. Seventh Example: G1. Training Process: Fig. 26 is a flowchart showing an example of the training process of the seventh example. The difference from the training process of Fig. 19 is that distortion is imparted to the training block image for training the anomaly detection model 310. In this example, the distortion correction model 320 is omitted.
[0218] S110e and S130 are the same as S110e and S130, respectively, in Fig. 19. The processor 210 acquires data of a plurality of block images representing a normal and less distorted training fabric 700t.
[0219] S310g is the same as S310e in Fig. 19. The processor 210 pastes the defect image into a portion of the block image obtained in S130. S310g is executed when training images with defects are used to train the anomaly detection model 310. In this embodiment, S310g is omitted.
[0220] S320g and S330g are the same as S320e and S330e in FIG. 19 . The processor 210 randomly generates a displacement vector field and applies distortion to the block images using the displacement vector field. As a result, the processor 210 generates data of training block images having distortion. The processor 210 may execute the processes of S320g and S330g on N block images to generate N training block images. Furthermore, the processor 210 may acquire data of multiple captured images in S110e to acquire N block images. Then, in S130, the processor 210 may acquire multiple block images from each of the multiple captured images.
[0221] In S340g, the processor 210 uses the N training block images to train the anomaly detection model 310. The training method is the same as the training method in S290 of Fig. 19 etc. After S340g, the processor 210 ends the processing of Fig. 26.
[0222] The inspection process of this embodiment is the same as the inspection process of Fig. 11. As described above, in this embodiment, the anomaly detection model 310 is trained using a plurality of training block images having distortion. Therefore, even if the target textile 700 represented by the captured image has distortion, the processor 210 can appropriately determine the presence or absence of a defect in the target textile 700 by using the anomaly detection model 310.
[0223] H. Eighth Example: H1. Training Process: FIG. 27 is a flowchart showing an example of the training process of the eighth example. The training process of this example includes S110e and S130 of FIG. 19, process SC1 of S140c-S290 of FIG. 12, and process SE1 of S310e-S380e of FIG. 19. In this way, the training method for the anomaly detection model 310 is the same as the training method of the example of FIG. 12. The training method for the distortion correction model 320 is the same as the training method of the example of FIG. 19.
[0224] H2. Inspection Process: FIG. 28 is a flowchart illustrating an example of the inspection process of the eighth embodiment. The inspection process of this embodiment includes steps S710, S720, and S730 of FIG. 16, step SC2 from S740c to S770c, and step SE2 from S810e to S890 of FIG. 22. In step SE2, the block image processed by step SC2 is processed. In this manner, the processor 210 corrects the first angle AC1 (S750c), the second angle AC2 (S770c), and the distortion (S810e) for the block image representing the captured image of the target fabric 700. In step S890, the processor 210 inputs each of the multiple corrected block images generated by steps SC2 and SE2 into the anomaly detection model 310 and determines whether the target fabric 700 has an abnormality.
[0225] As described above, in this embodiment, the processor 210 executes the following process to determine whether or not the target fabric 700 has a defect. In S720 and S730 ( FIG. 28 ), the processor 210 acquires block image data representing a captured image of the target fabric 700. In S750c, S770c, and S810e, the processor 210 performs a correction process on the block image. In S890, the processor 210 inputs the image processed by the correction process into the anomaly detection model 310 to determine whether or not the target fabric 700 has a defect. Here, the correction process (S750c, S770c, S810e) includes a distortion correction (S810e) that corrects distortion of the target fabric 700 in the image representing the target fabric 700. Therefore, even if the target fabric 700 in the captured image has distortion, the processor 210 can appropriately determine whether or not the target fabric 700 has a defect by using the anomaly detection model 310.
[0226] In this embodiment, the correction process (S750c, S770c, S810e) further includes angle correction (S750c, S770c). As shown in FIG. 14(E), the correction in S750c is a correction of the first angle AC1 by image rotation (a type of affine transformation) (referred to as the first correction). Specifically, the first correction is a rotation of the affine transformation that brings the first angle AC1 of the first direction Dc1 relative to the reference direction Dx closer to the first specific angle AC1Z. As described in FIG. 14(A), the first angle AC1 indicates the first direction Dc1, which is the direction in which multiple patterns PTc1 in the image representing the target fabric 700 are regularly arranged. The first pattern PTc1 is formed by the yarns Wf and Wp. As shown in FIG. 14(J), the correction in S770c is a correction of the second angle AC2 by image shear transformation. Specifically, the second correction adjusts the second angle AC2 of the second direction Dc2 relative to the second reference direction (here, the -Dy direction) to approach the second specific angle AC2Z by shearing the affine transformation. As described in FIG. 14(F), the second angle AC2 indicates the second direction Dc2, which is the direction in which the multiple second patterns PTc2 in the image representing the target fabric 700 are regularly arranged. The second patterns PTc2 are formed by the yarns Wf and Wp. The distortion correction performed in S810e then corrects the distortion of the target fabric 700 in the image processed by the angle correction (S750c, S770c).
[0227] According to this configuration, among the distortions of the target textile 700 in the block image (i.e., the captured image), the angular deviation is appropriately corrected by angle correction (here, S750c, S770c). The distortion correction in S810e is performed on the angle-corrected block image. Therefore, the effect of the angular deviation on the distortion correction in S810e is mitigated. As described above, the processor 210 can appropriately determine the presence or absence of a defect in the target textile 700 by using the anomaly detection model 310.
[0228] Furthermore, when a machine learning model (e.g., distortion correction model 320) is used for distortion correction in S810e, the machine learning model can be trained without considering angular misalignment, thereby reducing the burden of training the machine learning model (e.g., multiple training images corresponding to different angles can be omitted).
[0229] Note that either the first correction (S750c) or the second correction (S770c) may be omitted from the inspection process (FIG. 28). The correction (S150c or S170c) corresponding to the omitted angle correction may be omitted from the training process (FIG. 27). Also, instead of the distortion correction of S810e, the distortion correction of S810f (FIG. 24) may be performed. In this case, the distortion correction model 320 may be omitted. Process SE1 may be omitted from the training process (FIG. 27).
[0230] I. Ninth Example: I1. Training Process: FIG. 29 is a flowchart illustrating an example of the training process of the ninth example. The training process of this example includes steps S110e and S130 of FIG. 26, steps SD1 of S140c-S172d of FIG. 17, and step SG1 of S310g-S340g of FIG. 26. In step SG1, the block image processed by step SD1 is processed. In this manner, the processor 210 trains the anomaly detection model 310 using a plurality of training block images having various first angles AC1, various second angles AC2, and various distortions. Note that in this example, the distortion correction model 320 is omitted.
[0231] The inspection process of this embodiment is the same as the inspection process of FIG. 11 . Thus, in this embodiment, the processor 210 executes the following process to determine whether or not a defect exists in the target fabric 700. In S720 and S730 ( FIG. 11 ), the processor 210 acquires block image data representing a photographed image of the target fabric 700. In S890b, the processor 210 inputs the block image into the anomaly detection model 310 to determine whether or not a defect exists in the target fabric. As shown in S320g-S340g of FIG. 29 , the anomaly detection model 310 is trained using second-type images (here, training block images) obtained by performing image processing SD1 and SG1, which includes applying distortion (S330g), on first-type images representing the training fabric 700t (here, block images obtained in S130). Therefore, even if the target fabric 700 in the captured image has distortion, the processor 210 can appropriately determine whether or not there is a defect in the target fabric 700 by using the anomaly detection model 310.
[0232] In this embodiment, the image processing SD1 and SG1 for generating training block images includes an angle change process (S172d). The angle change process (S172d) includes a first change process for changing the first angle AC1 and a second change process for changing the second angle AC2. The first change process changes the first angle AC1 of the first direction Dc1 (FIG. 14E) of the image to be processed relative to the first reference direction (here, the +Dx direction) through an affine transformation (here, rotation). The second change process changes the second angle AC2 of the second direction Dc2 (FIG. 14J) of the image to be processed relative to the second reference direction (here, the -Dy direction) through shearing of the affine transformation. The distortion application process (S330g) applies distortion to the image processed by the angle change process (S172d). The training block images obtained through such image processing can appropriately represent the actual textile 700. Therefore, the processor 210 can appropriately determine whether or not there is a defect in the target fabric 700 by using the anomaly detection model 310 trained using such training block images.
[0233] The first change process of the first angle AC1 or the second change process of the second angle AC2 may be omitted from the angle change process (S172d).
[0234] J. Tenth Example: The fabric to be inspected may be a knitted fabric instead of a woven fabric (e.g., the woven fabric 700 (FIG. 2)). FIG. 30 is a diagram showing an example of a knitted fabric pattern. The knitted fabric 700J is knitted using the yarn Wj. The knitted fabric 700J may be knitted using various knitting methods. In either case, the knitted fabric 700J is configured so that patterns associated with the knitting method are regularly arranged. The captured image of the knitted fabric 700J generated by the digital camera 110 shows the detailed pattern of the knitted fabric 700J. Various methods may be used to identify the direction in which the patterns are arranged. For example, the gap PJ1 surrounded by the yarn Wj forms an island-like portion, similar to the first portions Pa1 and Pc1 (FIGS. 4(B) and 14(A)). Therefore, as in the above examples, the processor 210 can estimate the direction in which a pattern composed of multiple gaps PJ1 is arranged (e.g., the direction in which multiple patterns PTj1 are arranged) by using the binarized image. Furthermore, the pattern may be the shape of the yarn Wj (e.g., a loop, a knot, etc.), as in the pattern PTj2. In this case, the processor 210 may detect multiple patterns PTj2 from the captured image by template matching using a template image of the pattern PTj2. The processor 210 may then estimate the direction in which the patterns PTj2 are arranged using the detected multiple patterns PTj2. In this way, the direction of the fabric of the knitted fabric 700J, woven fabric 700, etc. represented by the captured image (i.e., the direction in which the patterns formed by the yarn are regularly arranged) may be determined by template matching.
[0235] K. Modifications: (1) In the examples of FIGS. 3 and 7 , multiple training block images showing various second angles AC2 may be further used to train the anomaly detection model 310. For example, the processor 210 may execute S160c, S162d, S164d, and S170c of FIG. 17 and modified S172d using the multiple training block images generated in S150 of FIG. 3 . In modified S172, the change in the first angle AC1 is omitted, and the processor 210 changes the second angle AC2. In S290, the multiple training block images generated in modified S172d are used. In this modification, the multiple training block images are configured such that the distribution range of the second angles AC2 relative to the reference direction (here, the −Dy direction) of the multiple second directions Dc2 ( FIG. 14(J) ) of the multiple training block images is a second angle range AC2R. 18(B), the second angle range AC2R is determined using the distribution of the second angle AC2 estimated using the captured image of the training fabric 700t. Here, the training fabric 700t is supported by a conveying device 900 that conveys the target fabric 700 for photographing the target fabric 700. The second angle AC2 indicates the angle of the second direction Dc2 based on the skew of the fabric 700 (e.g., the training fabric 700t). With this configuration, similar to the embodiments of FIGS. 17 and 18, even if the second angle AC2 indicated by the target fabric 700 represented by the captured image changes, the processor 210 can appropriately determine the presence or absence of a defect in the target fabric 700 by using the anomaly detection model 310.
[0236] 12 and 16, a plurality of training block images showing various second angles AC2 may be further used to train the anomaly detection model 310. For example, the processor 210 may execute S160c, S162d, S164d, and S170c of FIG. 17, as well as a modified S172d, using the plurality of training block images generated in S170c of FIG. 12. In the modified S172d, the change in the first angle AC1 is omitted, and the processor 210 changes the second angle AC2. In S290, the plurality of training block images generated in the modified S172d are used. In the inspection process of FIG. 16, S760c and S770c are omitted. In this case, as in the examples of Figures 17 and 18, even if the second angle AC2 indicated by the target fabric 700 represented by the captured image changes, the processor 210 can appropriately determine whether or not there is a defect in the target fabric 700 by using the anomaly detection model 310.
[0237] (2) In the examples of FIGS. 3 and 7 , the processor 210 may determine the first specific angle AA1Z using multiple captured images of the textile 700. For example, the processor 210 may execute S142b and S144b of FIG. 9 between S140 and S150. In this variation, the first specific angle AA1Z is determined using a distribution of first angles AA1 ( FIG. 10 ) estimated using block images (i.e., captured images) of the training textile 700t. Here, the training textile 700t is supported by a conveying device 900 that conveys the target textile 700 for imaging. Furthermore, the first angle AA1 indicates an angle based on the rotation of the textile 700 (e.g., the training textile 700t), as described with reference to FIG. 4(F) and other figures. This configuration reduces the discrepancy between the first specific angle AA1Z and the first angle AA1 indicated by the block images used in the inspection process. Therefore, the correction amount (e.g., rotation angle) by correcting the first angle AA1 (S150 (FIG. 3), S750 (FIG. 7)) is reduced, and as a result, the negative impact of correcting the first angle AA1 on determining whether or not there is a defect is mitigated.
[0238] 12 and 16, the processor 210 may determine one or both of the first specific angle AC1Z and the second specific angle AC2Z using multiple captured images of the fabric 700. For example, the processor 210 may execute steps S142d and S144d of FIG. 17 between steps S140c and S150c of FIG. 12. The processor 210 may also execute steps S162d and S164d of FIG. 17 between steps S160c and S170c of FIG. 12. When the first specific angle AC1Z is determined as described above, the amount of correction (e.g., rotation angle) required to correct the first angle AC1 (steps S150c ( FIG. 12) and S750c ( FIG. 16)) is reduced. Therefore, the possibility of false detection of a defect due to the correction of the first angle AC1 can be reduced. When the second specific angle AC2Z is determined as described above, the correction amount (e.g., the amount of change in angle due to shear transformation) of the second angle AC2 (S170c (FIG. 12), S770c (FIG. 16)) is reduced, thereby mitigating the negative impact of the correction of the second angle AC2 on the determination of the presence or absence of a defect.
[0239] (3) In the above embodiment and modified examples, the first specific angles AA1Z, AC1Z and their angle ranges AA1A, AA1R, AC1R are determined (e.g., S152 (FIG. 3), S142b, S144b (FIG. 9), S142d, S144d (FIG. 17)) using a distribution of multiple first angles AA1, AC1 obtained from multiple block images. As described with reference to FIGS. 4(D), 4(C), 14(A), and 14(C), multiple vectors Va, Vc1 indicating angles AA1v, AC1v corresponding to the first angles AA1, AC1 are obtained from one block image. These angles AA1v, AC1v also indicate the directions Da, Dc1 of the fabric 700. The processor 210 may determine the first specific angles AA1Z, AC1Z and ranges AA1A, AA1R, AC1R using the distribution of the multiple angles AA1v, AC1v of the multiple vectors Va, Vc1 obtained from one image.
[0240] In the above embodiment and modified examples, the second specific angle AC2Z and the second angle range AC2R are determined (e.g., S162d, S164d ( FIG. 17 )) using the distribution of multiple second angles AC2 obtained from multiple block images. As described with reference to FIGS. 14(F) and 14(H), multiple vectors Vc2 indicating angles AC2v corresponding to the second angle AC2 are obtained from one block image. Such angles AC2v also indicate the direction Dc2 of the textile 700. The processor 210 may determine the second specific angle AC2Z and the second angle range AC2R using the distribution of multiple angles AC2v of multiple vectors Vc2 obtained from one image.
[0241] (4) In the above embodiment and modified examples, the first angles AA1 and AC1 may represent angles of the first directions Da and Dc1 based on the skew of the fabric (e.g., woven fabric 700 or knitted fabric 700J). In this case, correction of the first angles AA1 and AC1 (e.g., S150 (FIG. 3), S750 (FIG. 7), S150c (FIGS. 12, 17, 27), and S750c (FIGS. 16, 28)) may be performed by shear transformation in a direction intersecting the first directions Da and Dc1 (e.g., second direction Dy). Changes in the first angles AA1 and AC1 due to data augmentation (e.g., S154 (FIG. 3), S154b (FIG. 9), and S172d (FIGS. 17, 29)) may be performed by shear transformation.
[0242] In the above embodiment and modified examples, the first angles AA1 and AC1 may be angles based on both the rotation and the skew of the fabric. For example, the first angles AA1 and AC1 may be determined by the sum of the rotation and the shear transformation.
[0243] In the above embodiment and modified examples, the second angle AC2 may represent the angle of the second direction Dc2 based on the rotation of the fabric (e.g., the woven fabric 700 or the knitted fabric 700J). In this case, correction of the second angle AC2 (e.g., S170c (FIGS. 12, 17, 27), S770c (FIGS. 16, 28)) may be performed based on the rotation. Changes in the second angle AC2 based on data augmentation (e.g., S172d (FIGS. 17, 29)) may be performed based on the rotation.
[0244] In the above embodiment and modified examples, the second angle AC2 may be an angle based on both the rotation and the skew of the fabric. For example, the second angle AC2 may be determined by the addition of the rotation and the shear transformation.
[0245] (5) In the above embodiment and modified examples, the anomaly detection model may be various models for detecting anomalies, instead of the anomaly detection model 310 in FIG. 6 . For example, in addition to the model 310 in FIG. 6 , the anomaly detection model may include an image generation model (e.g., an autoencoder) trained to generate an image of an object (textile, in this case) without defects from an image of the same object with defects. In the inspection process, the processor 210 inputs a block image representing a captured image of the textile into the image generation model to generate an intermediate block image, which is a block image without defects. The processor 210 generates a difference image between the block image and the intermediate block image. The processor 210 inputs the difference image into the model 310 to determine whether the textile has an anomaly (i.e., a defect).
[0246] The anomaly detection model may be configured using a technique called FastFlow (FastFlow: Unsupervised Anomaly Detection and Localization via 2D Normalizing Flows). FastFlow is disclosed in, for example, the following paper: Jiawei Yu, Ye Zheng, Xiang Wang, Wei Li, Yushuang Wu, Rui Zhao, Liwei Wu, "Fastflow: Unsupervised Anomaly Detection and Localization via 2d Normalizing Flows." arXiv:2111.07677, http: / / arxiv.org / abs / 2111.07677
[0247] The anomaly detection model may also be an image generation model (e.g., an autoencoder) trained to generate defect-free images from images with defects. The difference image between the captured image and the generated image obtained from the captured image can be used as an image indicating the defect.
[0248] The method for training the anomaly detection model may be any method suitable for the anomaly detection model. In any case, the various training block images described in the above embodiment and modifications may be used to train the anomaly detection model.
[0249] (6) In the above embodiment and modified examples, the distortion correction model may be various models for correcting distortion instead of the distortion correction model 320 in FIG. 21 . For example, the set of displacements of a plurality of control points 320ao ( FIG. 21 ) indicates the respective displacement vectors of a plurality of control points arranged in a matrix, similar to the displacement vector field VFe in FIG. 20(C). Here, the processor 210 may transform the image of the minimum rectangle by a homography transformation using the four displacement vectors of the four control points that form the minimum rectangle. The processor 210 performs such a homography transformation on each of the plurality of minimum rectangles that form the block image.
[0250] The distortion correction model may also be an image generation model (e.g., an autoencoder) trained to generate, from an image of an object (textile) with distortion, an image of the same object with reduced distortion. In this case, the distortion correction model is preferably trained to generate, from an image of the object with a defect, an image of the object with the same defect. The training process for such a distortion correction model preferably includes a process of adding defects to the training images, as in S310e of FIG. 19 .
[0251] (6) In the training process of the above embodiment and modified example, data augmentation in a range narrower than the angle distribution range may be applied to one or both of the first angle AC1 and the second angle AC2, as in S152 and S154 of FIG. 3 .
[0252] (7) In the above embodiment and modified examples, the device that optically reads the fabric may be a stationary reading device (e.g., a scanner) instead of a portable reading device such as the digital camera 110. In this way, the optically read image of the fabric is not limited to a photographed image taken by a camera, and may be various read images such as a scanned image read by a scanner.
[0253] (8) In the above embodiment and modifications, the processor 210 may cause the GPU 260 to execute various operations. For example, the processor 210 may cause the GPU 260 to execute some or all of the operations of a machine learning model (e.g., the anomaly detection model 310 or the distortion correction model 320).
[0254] (9) In the above embodiment and modified examples, the training process may be performed by a device (e.g., a computer) different from the determination device (e.g., the data processing device 200) that determines whether or not there is a defect in the fabric. Furthermore, the determination device may be a type of device (e.g., a digital camera, a scanner, a smartphone, or a tablet computer) different from a personal computer. Furthermore, multiple devices (e.g., computers) that can communicate with each other via a network may share some of the functions of the inspection process performed by the determination device and collectively provide the inspection process functions (a system including these devices corresponds to the determination device).
[0255] In each of the above embodiments, a part of the configuration realized by hardware may be replaced by software, and conversely, a part or all of the configuration realized by software may be replaced by hardware. For example, the function of the anomaly detection model 310 ( FIG. 6 ) may be realized by a dedicated hardware circuit.
[0256] Furthermore, when some or all of the functions of the present disclosure are realized by a computer program, the program can be provided in a form stored on a computer-readable recording medium (e.g., a non-transitory recording medium). The program can be used while stored on the same or a different recording medium (computer-readable recording medium) from when it was provided. The "computer-readable recording medium" is not limited to portable recording media such as memory cards and CD-ROMs, but can also include internal storage devices within a computer, such as various ROMs, and external storage devices connected to a computer, such as a hard disk drive.
[0257] The above-described examples and modifications can be combined as appropriate. The above-described examples and modifications are provided to facilitate understanding of the present disclosure and are not intended to limit the present invention. The present invention may be modified or improved without departing from the spirit thereof, and the present invention includes equivalents thereof.
[0258] 110...digital camera, 120...light source, 200...data processing device, 210...processor, 215...storage device, 220...volatile storage device, 230...non-volatile storage device, 232...training program, 233...inspection program, 240...display unit, 250...operation unit, 260...graphics processing unit, 270...communication interface, 310...anomaly detection model, 310a...feature extraction model, 310t...image Classification model, 320... distortion correction model, 320a... convolutional net regressor, 320b... spatial transformer, 320c... resampler, 320ao... displacement set, 700... fabric, 700a... target fabric, 700b... target fabric, 700c... target fabric, 700d... target fabric, 700t... training fabric, 700J... knitted fabric, 900... conveying device, 910... first roller, 912... rotary encoder, 920... second roller, 922... drive motor
Claims
1. A program for a computer that judges whether or not there is a defect in a woven or knitted fabric, comprising: a function for acquiring data of a read image, which is an optically read image of the target fabric; a correction function for performing a correction process on the read image, the correction process including a first correction that brings an angle of a first direction, which is a direction in which a first pattern formed by yarns is regularly arranged in an image representing the target fabric, relative to a first reference direction closer to a first specific angle; and a judgment function for inputting the image processed by the correction process into a machine learning model to judge whether or not there is a defect in the target fabric, the machine learning model being trained using an image of fabric in which the angle of the first direction is the first specific angle.
2. The program according to claim 1, wherein the machine learning model is trained using a plurality of training images representing a fabric, the plurality of training images are configured such that a distribution range of angles of a plurality of first directions of the plurality of training images is a specific range including the first specific angle, and the specific range is determined using a distribution of angles of the first direction based on one or both of the rotation and skew of the fabric estimated using a read image of the fabric supported by a conveying device that conveys the target fabric for reading the target fabric.
3. The program as described in claim 1, wherein the correction process includes a second correction for approximating an angle of a second direction, in which a second pattern formed by threads is regularly arranged in an image representing the target fabric, with respect to a second reference direction to a second specific angle by shearing of an affine transformation, and the machine learning model is trained using a plurality of training images representing fabric, and the plurality of training images include a plurality of training images corrected by a correction for approximating an angle of the second direction of the training images with respect to the second reference direction to the second specific angle by shearing of the affine transformation.
4. The program of claim 1, wherein the machine learning model is trained using a plurality of training images representing a fabric, the plurality of training images are configured such that a distribution range of angles of a plurality of second directions of the plurality of training images is an angle range, the second directions being directions in which a second pattern formed by threads in the fabric images is regularly arranged, and the angle range is determined using a distribution of angles of the second directions based on one or both of the rotation and skew of the fabric estimated using a read image of the fabric supported by a conveying device that conveys the target fabric for reading the target fabric.
5. A program as described in claim 1, wherein the first specific angle is determined using a distribution of angles of the first direction based on one or both of the rotation and skew of the fabric estimated using a read image of the fabric supported by a conveying device that conveys the target fabric for reading the target fabric.
6. A program for a computer that judges the presence or absence of defects in a woven or knitted fabric, the program causing a computer to realize a function of acquiring data of a read image, which is an optically read image of a target fabric, and a judgment function of inputting the read image into a machine learning model to judge the presence or absence of defects in the target fabric, the machine learning model being trained using a plurality of training images representing fabric, the plurality of training images being configured such that a distribution range of angles of a plurality of first directions of the plurality of training images relative to a first reference direction is a first angle range, the first direction being a direction in which a first pattern formed by yarns in the image of the fabric is regularly arranged, and the first angle range being determined using a distribution of angles of the first directions based on one or both of the rotation and skew of the fabric estimated using a read image of a fabric supported by a conveying device that conveys the target fabric for reading the target fabric.
7. A program for a computer that judges whether or not there is a defect in a woven or knitted fabric, the program causing a computer to realize the following functions: a function for acquiring data of a read image, which is an optically read image of the target fabric; a correction function for performing a correction process on the read image, the correction process including distortion correction for correcting distortion of the target fabric in an image representing the target fabric; and a judgment function for inputting the image processed by the correction process into a machine learning model to judge whether or not there is a defect in the target fabric.
8. A program as described in claim 7, wherein the distortion correction includes correcting distortion using a distortion correction model that is a machine learning model that corrects distortion, and the distortion correction model is trained to generate the first type of image from a second type of image obtained by applying distortion to a first type of image representing a fabric.
9. A program according to claim 8, wherein the first type of image is obtained by adding an image of a defect to a portion of an image representing a texture, and the second type of image represents the same defect as the defect represented by the first type of image.
10. A program as described in claim 8 or 9, wherein the distortion is applied according to the respective displacement vectors of a plurality of control points arranged in the first type image, and the plurality of displacement vectors of the plurality of control points are configured so that the difference between the displacement vectors of adjacent control points is equal to or smaller than a reference value.
11. The program according to claim 7, wherein the distortion correction includes correcting the distortion using a result of keypoint matching using a reference image.
12. A program as described in claim 7, wherein the correction process further includes angle correction, which includes one or both of: a first correction for making an angle of a first direction, in which a first pattern formed by threads is regularly arranged in an image representing the target fabric, with respect to a first reference direction by an affine transformation, approaching a first specific angle; and a second correction for making an angle of a second direction, in which a second pattern formed by threads is regularly arranged in an image representing the target fabric, with respect to a second reference direction by shearing an affine transformation, approaching a second specific angle; and wherein the distortion correction corrects distortion of the target fabric in the processed image by the angle correction.
13. A program for a computer that judges whether or not there is a defect in a woven or knitted fabric, the program causing the computer to realize a function of acquiring data of a read image, which is an optically read image of the target fabric, and a judgment function of inputting the read image into a machine learning model and judging whether or not there is a defect in the target fabric, the machine learning model being trained using a second type of image obtained by performing image processing, including the addition of distortion, on a first type of image representing the fabric.
14. A program as described in claim 13, wherein the image processing further includes an angle change process, the angle change process including one or both of: a first change process for changing an angle of a first direction of the image to be processed relative to a first reference direction by an affine transformation, the first direction being a direction in which a first pattern formed by threads in the image of the fabric is regularly arranged, and a second change process for changing an angle of a second direction of the image to be processed relative to a second reference direction by shearing an affine transformation, the second direction being a direction in which a second pattern formed by threads in the image of the fabric is regularly arranged, and wherein the imparting of distortion imparts distortion to an image already processed by the angle change process.
15. A judgment device for judging the presence or absence of defects in a woven or knitted fabric, comprising: an acquisition unit for acquiring data of a read image, which is an optically read image of the target fabric; a correction unit for performing a correction process on the read image, the correction process including a first correction for bringing an angle of a first direction, which is a direction in which a first pattern formed by threads is regularly arranged in the image representing the target fabric, relative to a first reference direction closer to a first specific angle; and a judgment unit for inputting the image processed by the correction process into a machine learning model to judge the presence or absence of defects in the target fabric, the machine learning model being trained using an image of fabric in which the angle of the first direction is the first specific angle.
16. A judgment device for judging the presence or absence of defects in a woven or knitted fabric, comprising: an acquisition unit for acquiring data of a read image, which is an optically read image of a target fabric; and a judgment unit for inputting the read image into a machine learning model to judge the presence or absence of defects in the target fabric, wherein the machine learning model is trained using a plurality of training images representing fabric, the plurality of training images are configured so that a distribution range of angles of a plurality of first directions of the plurality of training images relative to a first reference direction is a first angle range, the first direction being a direction in which a first pattern formed by yarns in the image of the fabric is regularly arranged, and the first angle range is determined using a distribution of angles of the first directions based on one or both of the rotation and skew of the fabric estimated using a read image of a fabric supported by a conveying device that conveys the target fabric for reading the target fabric.
17. A judgment device for judging the presence or absence of defects in a woven or knitted fabric, comprising: an acquisition unit for acquiring data of a read image, which is an optically read image of the target fabric; a correction unit for performing a correction process on the read image, the correction process including distortion correction for correcting distortion of the target fabric in an image representing the target fabric; and a judgment unit for inputting the image processed by the correction process into a machine learning model to judge the presence or absence of defects in the target fabric.
18. A judgment device for determining whether or not there is a defect in a woven or knitted fabric, comprising: an acquisition unit for acquiring data of a read image, which is an optically read image of a target fabric; and a judgment unit for inputting the read image into a machine learning model and determining whether or not there is a defect in the target fabric, wherein the machine learning model is trained using a second type of image obtained by performing image processing, including the addition of distortion, on a first type of image representing the fabric.
Citation Information
Patent Citations
Image distortion correction method, image distortion correction device and image distortion correction program based on representative point measurement
JP2004038884A
Correction program, correction method, and information processor
JP2021156629A
