Program and data processing apparatus

Abstract

The color of the yarn is determined.SOLUTION: A plurality of read thread images, which are a plurality of images read under mutually different reading conditions of a specific thread included in a fabric including a plurality of threads, are acquired. Using each of the plurality of read thread images, a candidate color that is a candidate of a color of a specific thread is determined for each read thread image. The color of the specific yarn is determined using the plurality of candidate colors of the plurality of read yarn images.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present specification relates to techniques for determining the color of a thread. [Background technology]

[0002] Conventionally, fabrics for sewing, such as woven fabrics and knitted fabrics, are made using threads of various colors. The threads are dyed, for example, using dyes. Patent Document 1 discloses a technique for re-dyeing threads so that the threads remaining after the job is completed can be used for a new purpose. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2022-547996 Summary of the Invention [Problem to be solved by the invention]

[0004] The color of a thread can be referenced by various processes. For example, the color of a thread can be referenced to determine whether the color of each of multiple threads in a fabric is consistent with the design specifications. The color of a thread can be determined using a scanned image, which is an image of the thread optically read by a reading device such as a digital camera or scanner. However, the color of a thread can vary due to various causes. For example, the color of the thread can change due to variations in dyeing. Furthermore, the color of the thread in the scanned image can change depending on how light from a light source hits the thread during reading. There is room for ingenuity in determining the color of a thread that can vary in this way.

[0005] This specification discloses techniques for determining the color of a thread. [Means for solving the problem]

[0006] The techniques disclosed in this specification can be implemented in the following application examples.

[0007] [Application Example 1] A program that causes a computer to realize the following functions: a function for acquiring multiple read yarn images, which are multiple images of a specific yarn contained in a fabric containing multiple yarns, read under different reading conditions; a candidate color determination function for using each of the multiple read yarn images to determine a candidate color that is a candidate for the color of the specific yarn for each read yarn image; and a color determination function for determining the color of the specific yarn using the multiple candidate colors of the multiple read yarn images.

[0008] According to this configuration, a candidate color that is a candidate for the color of a specific thread is determined for each read thread image using each of the multiple read thread images, and the color of the specific thread is determined using multiple candidate colors of the multiple read thread images, so that the color of the specific thread can be determined appropriately.

[0009] The technology disclosed in this specification can be realized in various forms, such as a data processing method and a data processing device, a computer program for realizing the functions of the method or device, a recording medium (e.g., a non-temporary recording medium) on which the computer program is recorded, and the like. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is an explanatory diagram illustrating a data processing device according to an embodiment; [Figure 2] FIG. 1 is a perspective view showing a schematic configuration of an inspection device 10. [Figure 3] 10A-10D are diagrams showing examples of multiple threads in a fabric as represented by a scanned image. [Figure 4] 10 is a flowchart illustrating an example of an inspection process. [Figure 5] (A)-(C) are diagrams showing examples of images to be processed. [Figure 6] 10 is a flowchart illustrating an example of a process for determining a representative color of a thread portion. [Figure 7] 1A is a block diagram showing an outline of the candidate color determination process, and FIG. 1B is a block diagram showing an outline of the representative color determination process. [Figure 8] 10 is a diagram illustrating an example of frequency distribution of red R, green G, and blue B gradation values. [Figure 9] FIG. 10 is a diagram illustrating an example of a color determination model. [Figure 10] 10 is a flowchart illustrating a second embodiment of the inspection process. [Figure 11] 10 is a flowchart illustrating an example of a process for determining a warp color. [Figure 12] 10A is a diagram showing an example of a plurality of warp portion images IMpp and a plurality of warp portions Pp, and FIG. 10B is a diagram showing an example of a determination model. DETAILED DESCRIPTION OF THE INVENTION

[0011] A. First Example: A1.Device configuration: 1 is an explanatory diagram showing a data processing device according to an embodiment. The data processing device 200 is, for example, a personal computer. The data processing device 200 executes a process for inspecting the color of threads in a fabric containing multiple threads (for example, a woven fabric, a knitted fabric, or a fabric for sewing such as denim).

[0012] The data processing device 200 includes a processor 210, a storage device 215, a display unit 240, an operation unit 250, a graphics processing unit (GPU) 260, and a communication interface 270. These elements are connected to each other via a bus. The storage device 215 includes a volatile storage device 220 and a non-volatile storage device 230.

[0013] The processor 210 is a device configured to process data, such as a central processing unit (CPU) or a system on a chip (SoC). The volatile storage device 220 is, for example, a dynamic random access memory (DRAM), and the nonvolatile storage device 230 is, for example, a flash memory. The nonvolatile storage device 230 stores data for a program 231, an object detection model M1, color classification models M2r, M2g, and M2b, a reference pattern D1, and color determination models CMr, CMg, and CMb. The object detection model M1 and the color classification models M2r, M2g, and M2b are each program modules that form trained machine learning models.

[0014] The display unit 240 is a device configured to display images, such as a liquid crystal display or an organic EL display. The operation unit 250 is a device configured to receive operations by a user, such as a button, a lever, or a touch panel overlaid on the display unit 240. The display unit 240 and the operation unit 250 may form a so-called touch screen. The user can input various requests and instructions to the data processing device 200 by operating the operation unit 250. The display unit 240 may display operation elements (e.g., buttons, sliders, etc.), and the displayed elements may be operated through operation of the operation unit 250.

[0015] The GPU 260 is a computing device configured to perform various numerical operations such as image processing and machine learning. The GPU 260 performs various operations in accordance with instructions from the processor 210. In this embodiment, the GPU 260 performs operations for the object detection model M1 and the color classification models M2r, M2g, and M2b in accordance with instructions from the processor 210.

[0016] The communication interface 270 is an interface for communicating with other devices (for example, it includes one or more of a USB interface, a wired LAN interface, an IEEE802.11 wireless interface, and an industrial camera interface (for example, CameraLink, CoaXPress, etc.)). In this embodiment, the inspection device 10 is connected to the communication interface 270.

[0017] The inspection device 10 includes a conveying device 900, digital cameras 111-114, a rotary encoder 120 (simply referred to as the encoder 120), and light sources 131, 132, and 133.

[0018] The digital cameras 111-114 use image sensors such as CCDs and CMOSs ​​to capture images of the objects, thereby generating captured image data representing the objects. In this embodiment, the digital cameras 111-114 are used to capture images of the fabric. The encoder 120 is used to calculate the relative position of the fabric with respect to the conveying device 900. The light sources 131, 132, and 133 irradiate the fabric with light in order to capture a clear image of the fabric.

[0019] The conveying device 900 is a device that conveys fabric to be inspected, and includes a conveying mechanism 950, an operation unit 980 that receives operations by a user, and a control unit 990.

[0020] FIG. 2 is a perspective view showing a schematic configuration of the inspection device 10. The conveying mechanism 950 is a device that conveys the fabric 700 for inspection (such a device is also called a fabric inspection machine). The conveying mechanism 950 is equipped with a plurality of rollers (including two rollers 910, 920) and a conveying motor (not shown) that drives one or more rollers to convey the fabric 700. In FIG. 2, the operation unit 980 and the control unit 990 of the conveying device 900 are not shown.

[0021] The partial path Pth in the figure indicates the portion of the transport path of the fabric 700 between the rollers 910 and 920. In this embodiment, the fabric 700, which is longer than the partial path Pth, is wound around a roller (not shown). The fabric 700 is pulled out from this roller, transported from the first roller 910 along the partial path Pth to the second roller 920, and then wound around another roller (not shown).

[0022] Between the rollers 910 and 920 (i.e., on the partial path Pth), the fabric 700 forms a flat portion, that is, a flat portion 700F. The light sources 131, 132, and 133 irradiate the flat portion 700F with light.

[0023] In the figure, the forward direction Df indicates the conveying direction on the partial path Pth (the forward direction Df is also referred to as the conveying direction Df). The reverse direction Db indicates the opposite direction of the forward direction Df, i.e., the conveying direction when rewinding the fabric 700. The vertical direction Dt indicates a direction parallel to the flat portion 700F and perpendicular to the partial path Pth.

[0024] Hereinafter, the vertical direction Dt will also be referred to as the +Dt direction, and the direction opposite to the vertical direction Dt will also be referred to as the -Dt direction. Similarly, for other directions, the same direction and the opposite direction are expressed by a positive sign or a negative sign before the sign.

[0025] The ends 700e1 and 700e2 in the drawing are ends in a direction perpendicular to the partial path Pth of the fabric 700. Hereinafter, the left end 700e1 will be referred to as the left end 700e1, and the right end 700e2 will be referred to as the right end 700e2.

[0026] In the drawing, the reading area Ar, which is the area read by the digital cameras 111-114, is hatched. The reading area Ar is a rectangular area extending in a direction perpendicular to the partial path Pth and is included in the flat portion 700F (i.e., the reading area Ar is illuminated by light from the light sources 131, 132, and 133). A first range R1 is the range of the reading area Ar in the conveying direction Df, and a second range R2 is the range of the reading area Ar in the vertical direction Dt. The size of the second range R2 is larger than the size of the fabric 700 in the vertical direction Dt.

[0027] In the figure, partial areas R11-R14 indicate areas read by digital cameras 111-114, respectively. In this embodiment, digital cameras 111-114 (and thus partial areas R11-R14) are arranged side by side in the vertical direction Dt. The partial areas R11-R14 together represent the entire reading area Ar.

[0028] The light sources 131, 132, and 133 are located in different directions relative to the fabric 700. In the example of FIG. 2, the second light source 132 is located in front of the fabric 700. The second light source 132 irradiates light onto the fabric 700 from the front of the fabric 700. The first light source 131 is located to the left of the second light source 132. The first light source 131 irradiates light onto the fabric 700 from a diagonal left direction of the fabric 700 (i.e., a direction between the front direction and the left direction). The third light source 133 is located to the right of the second light source 132. The third light source 133 irradiates light onto the fabric 700 from a diagonal right direction of the fabric 700 (i.e., a direction between the front direction and the right direction). In this way, the lighting directions, which are the directions of the light sources as seen from the fabric 700, are different between the light sources 131, 132, and 133. As will be described below, the apparent color of the fabric 700 (and therefore the yarn) can change depending on the lighting direction.

[0029] The operation unit 980 (FIG. 1) of the conveying device 900 is configured to receive operations from a user (for example, an operator who inspects the fabric 700). In this embodiment, the operation unit 980 includes a plurality of switches (not shown) (such as a push switch and a foot switch). The plurality of switches includes a conveying start switch and a stop switch.

[0030] The control unit 990 (FIG. 1) of the conveying device 900 is connected to the operation unit 980 and a conveying motor and power supply (not shown). The control unit 990 is an electric circuit configured to control the conveying motor in response to the operation of the operation unit 980 and signals from the data processing device 200. The control unit 990 may be configured using a computer or dedicated hardware (such as an Application Specific Integrated Circuit (ASIC)). When the conveyance start switch is pressed while the fabric 700 is not being conveyed, the control unit 990 starts conveying the fabric 700. When the stop switch is pressed while the fabric 700 is being conveyed, the control unit 990 stops conveying the fabric 700. Furthermore, the control unit 990 starts conveying the fabric 700 in response to a start command from the data processing device 200. The control unit 990 stops conveying the fabric 700 in response to a stop command from the data processing device 200.

[0031] The conveying device 900 is connected to an encoder 120 that detects the direction and amount of change in position due to conveyance. In this embodiment, the encoder 120 is connected to a roller (e.g., the first roller 910 (FIG. 2)). The encoder 120 may have various configurations that detect the direction and amount of change in position due to conveyance. For example, the encoder 120 may be an incremental encoder. The data processing device 200 (FIG. 1) can use information from the encoder 120 to obtain the current conveyance position of the fabric 700 conveyed by the conveying device 900 (i.e., the relative position of the fabric 700 with respect to the conveying device 900).

[0032] A2.Thread color: 3(A)-3(D) are diagrams showing examples of multiple threads of fabric 700 represented by scanned images generated by digital cameras 111-114 (FIG. 2). FIGS. 3(A)-3(D) show partial images IMp1-IMp4, which are portions of the scanned images. As will be described later, the scanned images are bitmap data that represent the color values ​​of each of multiple pixels arranged in a matrix along a first direction Dx and a second direction Dy perpendicular to the first direction Dx.

[0033] As shown in the figure, the fabric 700 is configured such that multiple warp threads Wp and multiple weft threads Wf intersect with each other. The warp threads Wp are threads extending in the vertical direction (in this embodiment, the warp threads Wp are approximately parallel to the second direction Dy). Multiple thread portions extending in the vertical direction at multiple positions in the horizontal direction of the fabric 700 correspond to the multiple warp threads Wp. The weft threads Wf are threads extending in the horizontal direction (in this embodiment, the weft threads Wf are approximately parallel to the first direction Dx). Multiple thread portions extending in the horizontal direction at multiple positions in the vertical direction of the fabric 700 correspond to the multiple weft threads Wf. The multiple weft threads Wf may be formed by folding back a single long thread multiple times. In this way, a single long thread may form multiple weft threads Wf extending in the horizontal direction at multiple positions in the vertical direction. The same applies to the multiple warp threads Wp.

[0034] Partial images IMp1-IMp3 in Figures 3(A)-3(C) show the same portion of fabric 700. The light sources that are lit are different between Figures 3(A)-3(C). Figure 3(A) shows the case where the first light source 131 is lit, Figure 3(B) shows the case where the second light source 132 is lit, and Figure 3(C) shows the case where the third light source 133 is lit. Each figure shows an enlarged view of portions IMq1-IMq3 of partial images IMp1-IMp3. Portions IMq1-IMq3 show the same portion of fabric 700. Portions IMq1-IMq3 show multiple warp yarns Wp and multiple weft yarns Wf.

[0035] The visible parts of each thread that are not hidden by other threads are more likely to receive light from the light source and are therefore displayed in bright colors in the partial images IMp1-IMp3 (i.e., the scanned image). Shadows are more likely to appear at the boundary between two intersecting threads. The boundary is displayed in dark colors in the scanned image.

[0036] The apparent color of each thread may change depending on the lighting direction. In the example of FIG. 3(A), light is irradiated onto the fabric 700 from diagonally to the left of the fabric 700. The right side of the warp threads Wp tends to be darker than the left side. In the example of FIG. 3(B), light is irradiated onto the fabric 700 from the front of the fabric 700. The brightness of the right and left sides of the warp threads Wp may be approximately the same. In the example of FIG. 3(C), light is irradiated onto the fabric 700 from diagonally to the right of the fabric 700. The left side of the warp threads Wp tends to be darker than the right side. As a result, the apparent color of the warp threads Wp in the scanned image may change depending on the lighting direction. The apparent color of the weft threads Wf may also change depending on the lighting direction.

[0037] The apparent color of each yarn can vary due to various other factors. For example, color is imparted to the yarn by dyeing. Variations in dyeing can result in variations in the apparent color of the yarn. For example, cheese dyeing can be performed using a cylindrically wound yarn (also called cheese). The dye passes from the inner periphery of the cheese to the outer periphery, or from the outer periphery to the inner periphery, of the cheese, thereby imparting color to the yarn forming the cheese. Here, the color of the dyed yarn can differ between the inner and outer periphery of the cheese. Furthermore, the ease of dyeing can vary depending on the type (e.g., country of origin) of the raw yarn material (e.g., cotton, silk, etc.) used for dyeing. As a result, the color of the dyed yarn can vary depending on the type of raw yarn material. Furthermore, the color of the dyed yarn can vary depending on the time of production of the dye.

[0038] Partial image IMp4 in FIG. 3(D) is a portion of the scanned image. On fabric 700 in FIG. 3(D), first portions Aa exhibiting a light color and second portions Ab exhibiting a dark color are alternately arranged in the second direction Dy. The color of the first portions Aa is, for example, yellow, and the color of the second portions Ab is, for example, the color of Osmanthus flowers. The second portion Ab located at the top in the figure includes an even darker portion Abx. Such unintended colored portions Abx can be formed due to variations in the color of the yarn.

[0039] Large deviations in thread color are not tolerated, but small deviations in thread color are tolerated. In this embodiment, the processor 210 determines the thread color using multiple images representing the thread during the inspection process described below, thereby reducing the likelihood that the tolerated color deviation will be determined to be incorrect.

[0040] A3. Inspection process: FIG. 4 is a flowchart illustrating an example of an inspection process. In this embodiment, the data processing device 200 inspects the color of the thread contained in the fabric 700. For inspection, the fabric 700 (FIG. 2) is attached to the conveying mechanism 950. In this embodiment, an operator attaches the fabric 700 to the conveying mechanism 950. Alternatively, a machine (e.g., a robot arm) may attach the fabric 700 to the conveying mechanism 950. After the fabric 700 is attached, an instruction to start the inspection is input to the data processing device 200 (FIG. 1). In this embodiment, the operator inputs the instruction to start the inspection by operating the operation unit 250. The processor 210 starts the inspection process in response to the start instruction. In this embodiment, the processor 210 proceeds with the inspection process in accordance with the program 231.

[0041] In this embodiment, the processor 210 repeatedly reads the portion of the fabric 700 within the reading area Ar using the light sources 131, 132, and 133 one by one. In S110, the processor 210 selects an unused light source from the light sources 131, 132, and 133 as the target light source. In S120, the processor 210 turns on the target light source and turns off the other light sources. In S130, the fabric 700 is photographed. In this embodiment, the processor 210 supplies a reading instruction to each of the digital cameras 111-114 (FIG. 2). The digital cameras 111-114 read the fabric 700 in accordance with the reading instruction. The processor 210 obtains read image data representing the read image from each of the digital cameras 111-114.

[0042] 5(A)-5(C) are diagrams showing examples of images to be processed. FIG. 5(A) shows examples of scanned images IMr1-IMr4 obtained from digital cameras 111-114 (FIG. 2), respectively. Each of scanned images IMr1-IMr4 is a rectangular image having two sides parallel to a first direction Dx and two sides parallel to a second direction Dy perpendicular to the first direction Dx. The second direction Dy is approximately the same as the conveying direction Df (FIG. 2) of the fabric 700 in scanned images IMr1-IMr4. The first direction Dx is approximately the same as the vertical direction Dt of the fabric 700 in scanned images IMr1-IMr4. The data of each of scanned images IMr1-IMr4 is bitmap data representing the color values ​​of a plurality of pixels arranged in a matrix along the first direction Dx and the second direction Dy. The color value is represented by, for example, the gradation value of each of red R, green G, and blue B (for example, a value greater than or equal to zero and less than or equal to 255).

[0043] As described in FIG. 2, partial regions R11-R14 corresponding to scanned images IMr1-IMr4 (FIG. 5(A)) are arranged side by side in the vertical direction Dt. The first scanned image IMr1 represents a portion of the fabric 700 including the left edge 700e1 and the background BG. The second scanned image IMr2 and the third scanned image IMr3 represent portions of the fabric 700 that are inside the edges 700e1 and 700e2, respectively. The fourth scanned image IMr4 represents a portion of the fabric 700 including the right edge 700e2 and the background BG. These scanned images IMr1-IMr4 as a whole represent the entire scanning area Ar. Although not shown, the background BG may represent various objects located outside the fabric 700, such as parts of the conveying mechanism 950.

[0044] The processor 210 generates data of a combined image by combining the scanned images IMr1-IMr4. Fig. 5(B) is a diagram showing an example of the combined image. The combined image IMrc represents a portion of the scanned area Ar of the fabric 700.

[0045] Various methods may be used to generate the combined image IMrc. For example, the partial regions R11-R14 (FIG. 2) may be arranged side by side in the vertical direction Dt so as not to overlap one another within the reading area Ar with any gaps. In this case, the processor 210 may generate data for the combined image IMrc by connecting the respective edges of the read images IMr1-IMr4 (FIG. 5(A)) arranged side by side in the first direction Dx. Alternatively, the partial regions R11-R14 may be arranged so that two adjacent partial regions partially overlap. In this case, the processor 210 may generate data for the combined image IMrc by combining the read images IMr1-IMr4 in the same arrangement as the partial regions R11-R14. The corresponding portion of one of the read images may be used as the image for the overlapping portion of the two read images.

[0046] In S140 (FIG. 4), the processor 210 determines whether all of the light sources 131-133 have been used. If an unused light source remains (S140: No), the processor 210 proceeds to S110 and reads the portion of the fabric 700 within the reading area Ar using the unused light source.

[0047] If all light sources have been used (S140: Yes), in S150, the processor 210 determines the color of each thread using multiple scanned images associated with multiple illumination directions (i.e., multiple light sources 131, 132, 133). In this embodiment, the processor 210 uses the combined image IMrc (FIG. 5(B)) as the scanned image. Also, in this embodiment, S150 includes S153 and S156. In S153, the processor 210 determines the representative color of each of the multiple thread portions represented by the scanned images.

[0048] Fig. 6 is a flowchart showing an example of a process for determining a representative color of a thread portion. In steps S210 to S240, the processor 210 determines a candidate color for each illumination direction of the thread portion in the scanned image. Fig. 7(A) is a block diagram showing an overview of the candidate color determination process. The diagram shows the configuration corresponding to steps S210 to S235.

[0049] In S210, the processor 210 detects thread portions using the scanned image (here, the combined image IMrc (FIG. 5(B))). FIG. 5(C) shows an example of thread portion processing. The upper left of the figure shows a part of a partial image IMt, which is a part of the combined image IMrc. As described with reference to FIGS. 3(A) to 3(D), the partial image IMt shows multiple warp threads Wp and multiple weft threads Wf. The upper right of the figure shows a partial image IMt that is the same as the upper left partial image IMt. In the upper right partial image IMt, multiple thread portions Pp, Pf are each hatched. The warp portion Pp is a visible portion of the warp thread Wp that is not hidden by other threads. In this embodiment, a continuous area that represents a portion of the warp thread Wp that overlaps with one or more weft threads Wf is used as the warp portion Pp. Similarly, the weft portion Pf is a visible portion of the weft thread Wf that is not hidden by other threads. In this embodiment, a continuous region representing a portion of the weft thread Wf that overlaps with one or more warp threads Wp is adopted as the weft thread portion Pf.

[0050] The method for detecting the thread portions Pp and Pf may be various methods. In this embodiment, the processor 210 detects the thread portions Pp and Pf using a trained object detection model M1 ( FIG. 1 ). The object detection model M1 may be various models capable of detecting the thread portions Pp and Pf. In this embodiment, the object detection model M1 is a model called "RTMDet" disclosed in the following paper: Chengqi Lyu, Wenwei Zhang, Haian Huang, Yue Zhou, Yudong Wang, Yanyi Liu, Shilong Zhang and Kai Chen. "Rtmdet: An Empirical Study of Designing Real-Time Object Detectors", arXiv.2212.07784, December 16, 2022, https: / / doi.org / 10.48550 / arXiv.2212.07784.

[0051] RTMDet is a model that detects the bounding box and category (i.e., object type) of an object and performs pixel-level region division called instance segmentation. In this embodiment, the object detection model M1 is pre-trained to detect the warp portion Pp and the weft portion Pf. The object detection model M1 may be trained using various methods, such as the training method described in the above-mentioned paper by RTMDet.

[0052] The detection result for one thread portion includes a bounding box (rectangular area) surrounding the thread portion, a reliability (also called certainty) for each type of thread portion (here, warp portion Pp or weft portion Pf), and a mask area indicating the pixel-level area where the thread portion is located. The processor 210 uses the mask area with a reliability equal to or greater than a predetermined threshold as the area of ​​the warp portion Pp or weft portion Pf. Note that the warp threads Wp and weft threads Wf are soft and can bend. The outline of the area of ​​the thread portions Pp and Pf may include a curved portion instead of a rectangle as shown in FIG. 5(C).

[0053] A first input size (specifically, the number of pixels in the first direction Dx and the number of pixels in the second direction Dy), which is the image size accepted by the object detection model M1, is determined in advance. The processor 210 divides the combined image IMrc into multiple partial images each having the first input size, and detects thread portions Pp and Pf from each partial image. The object detection model M1 detects multiple warp portions Pp and multiple weft portions Pf from one partial image. In this embodiment, as described in S110-S140 of FIG. 4, multiple combined images associated with different illumination directions are generated. The processor 210 detects multiple thread portions Pp and Pf from each combined image.

[0054] In S215 (FIG. 6), the processor 210 selects an unprocessed thread portion from the multiple thread portions Pp, Pf as the target thread portion. In S220, the processor 210 selects an unused lighting direction from the multiple lighting directions by processing the target thread portion as the target lighting direction. In S225, the processor 210 acquires an image of the thread portion associated with the target thread portion and the target lighting direction from the images of the multiple thread portions detected in S210. In this embodiment, the processor 210 acquires, as the thread portion image, an image included in a bounding box associated with the target thread portion from the combined image associated with the target lighting direction. The image IMw in FIG. 7(A) is an example of a thread portion image. This thread portion image IMw represents the warp thread portion Pp. The mask area Ma represents the warp thread portion Pp.

[0055] In S230 and S235 (FIG. 6), the processor 210 determines candidate colors that are candidates for the color of the thread portion. Various methods may be used to determine candidate colors. In this embodiment, the processor 210 determines candidate colors for the thread portion using trained color classification models M2r, M2g, and M2b (FIGS. 1 and 7(A)). The color classification models M2r, M2g, and M2b may be various models that classify the color of an input image into one of multiple colors. In this embodiment, each of the color classification models M2r, M2g, and M2b is a so-called convolutional neural network. Although not shown, each of the color classification models M2r, M2g, and M2b includes, for example, one or more sets of sets each including a convolutional layer and a pooling layer following the convolutional layer, connected in series, and one or more fully connected layers following the final pooling layer.

[0056] The thread may be a variety of colors. In thread color inspection, two similar colors (e.g., blue and navy blue) may be distinguished. In this embodiment, three color classification models M2r, M2g, and M2b are used to accurately classify the colors of thread portions. The color classification models M2r, M2g, and M2b are associated with red (R), green (G), and blue (B), respectively. The first color classification model M2r classifies the color of an input image into one of multiple colors close to red (e.g., red, yellow, and the color of Osmanthus flowers). The second color classification model M2g classifies the color of an input image into one of multiple colors close to green (e.g., green, yellow-green, and olive). The third color classification model M2b classifies the color of an input image into one of multiple colors close to blue (e.g., blue, navy blue, and light blue). The color classification models M2r, M2g, and M2b are each configured to classify the color of an input image into one of multiple colors of multiple threads that may be used. Hereinafter, colors that can be classified by the color classification models M2r, M2g, and M2b will be referred to as class colors.

[0057] The training method for each of the color classification models M2r, M2g, and M2b may be any method suitable for the color classification models M2r, M2g, and M2b. For example, to train the third color classification model M2b, yarn portion images for each of multiple class colors (e.g., blue, navy blue, and light blue) are prepared. The training yarn portion images may be acquired using the object detection model M1, as in S210 and S225 (FIG. 6). The training yarn portion images are input to the third color classification model M2b. The size of the yarn portion images (specifically, the number of pixels in the first direction Dx and the number of pixels in the second direction Dy) is adjusted by resolution conversion to a second input size, which is the image size accepted by the third color classification model M2b. Classification result data is generated by calculation of the third color classification model M2b using the training yarn portion image data. Multiple parameters of the third color classification model M2b are adjusted so as to reduce the loss indicating the difference between the classification result data and classification data indicating the color of the thread portion of the input thread portion image. The multiple parameters are adjusted, for example, by a method using backpropagation and gradient descent. The loss may be, for example, so-called cross entropy. By using images of blue, navy blue, and light blue thread portions for training, the trained third color classification model M2b can appropriately classify blue, navy blue, and light blue. The first color classification model M2r and the second color classification model M2g are also trained using images of thread portions of each of the multiple class colors.

[0058] In S230, the processor 210 determines the dominant color component of the thread portion image. The dominant color component is a representative color component among RGB that represents the thread portion. Various methods may be used to determine the dominant color component. In this embodiment, the processor 210 determines the dominant color component using the frequency of gradation values ​​equal to or greater than a threshold. FIG. 8 is a diagram showing an example of the frequency distribution of red (R), green (G), and blue (B) gradation values ​​of the portion of the thread portion image included in the mask area (i.e., the thread portion). The horizontal axis of each frequency distribution represents the gradation value V, and the vertical axis represents the frequency F. The processor 210 calculates the total numbers Cr, Cg, and Cb of pixels having a gradation value V equal to or greater than a predetermined threshold Vth. The total numbers Cr, Cg, and Cb correspond to RGB, respectively. The processor 210 determines the color component associated with the largest total number among the total numbers Cr, Cg, and Cb as the dominant color component. In the example of FIG. 8, the total number of blues Cb is greater than the total number of reds Cr and greens Cg, so the dominant color component is blue B.

[0059] In S235, the processor 210 determines a candidate color by inputting the thread portion image into a color classification model associated with the dominant color component. For example, the thread portion image IMw (FIG. 7A) is input into the third color classification model M2b, thereby determining the candidate color CC. Here, the processor 210 adjusts the size of the thread portion image to a second input size accepted by the color classification model by resolution conversion.

[0060] In this way, the processor 210 determines the dominant color component of the thread portion from RGB (S230), and determines candidate colors using a color classification model associated with the dominant color component (S235). This improves the accuracy of color classification compared to when a single color classification model is used to classify all colors.

[0061] In S240 (FIG. 6), the processor 210 determines whether processing of all lighting directions related to the target yarn portion has been completed. If unused lighting directions remain (S240: No), the processor 210 proceeds to S220 and performs processing for the unused lighting directions.

[0062] If processing for all lighting directions is complete (S240: Yes), in S245-S255, the processor 210 determines a representative color of the target thread portion using multiple candidate colors associated with multiple lighting directions. FIG. 7B is a block diagram showing an overview of the representative color determination process. The thread color determination subsystem SS shows the processing of S210-S235 described above (FIGS. 6 and 7A). Three partial images IMt1, IMt2, and IMt3 associated with the three lighting directions are processed by the thread color determination subsystem SS. As a result, three candidate colors CC1, CC2, and CC3 of the target thread portion are determined. The three candidate colors CC1, CC2, and CC3 are associated with the three lighting directions, respectively. In S245-S255, the processor 210 determines a representative color Cw using the candidate colors CC1, CC2, and CC3. In this embodiment, the processor 210 counts the number of votes, which is the total number of candidate colors, for each class color of the color classification model associated with the dominant color component. The processor 210 determines one representative color Cw by comparing the number of votes for each class color with a threshold value for each class color. The threshold value is determined using a variation parameter.

[0063] In S245 (FIG. 6), the processor 210 determines variation parameters. The variation parameters are parameters that indicate variation factors related to the acquisition of a yarn portion image or the yarn. In this example, the yarn dyeing method is cheese dyeing. The variation parameters include a parameter P1 that indicates whether the yarn represented by the target yarn portion is included in the inner portion or the outer portion of the cheese.

[0064] Such a variation parameter P1 is determined in advance for each yarn. That is, the variation parameter P1 for each of the multiple warp yarns Wp and the multiple weft yarns Wf included in the fabric 700 is determined in advance. The operator stores data of a reference pattern D1 (FIG. 1) representing an appropriate variation parameter P1 for each of the multiple yarns in the non-volatile storage device 230 of the data processing device 200 in advance. The reference pattern D1 also represents a design color, which is the design color of each yarn.

[0065] The processor 210 determines the variation parameter P1 of the target yarn portion by referencing the data of the reference pattern D1. Here, the processor 210 determines the correspondence between the target yarn portion and the yarn represented by the reference pattern (i.e., the yarn in the fabric 700). Various methods may be used to determine this correspondence. For example, the processor 210 determines the correspondence between the target yarn portion and the yarn according to the following method. First, the processor 210 classifies multiple warp portions Pp into groups for each warp thread Wp. A partial image IMt identical to the partial image IMt in the upper right corner of FIG. 5(C) is shown in the lower right corner. The processor 210 calculates the center of gravity Ppc of each warp portion Pp. The processor 210 projects the multiple centers of gravity Ppc onto a projection line PLp perpendicular to the second direction Dy to obtain a distribution of the frequency FQ of the positions of the centers of gravity Ppc in the first direction Dx. In this embodiment, the warp threads Wp are approximately parallel to the second direction Dy. Therefore, the distribution of the frequencies FQ of the centers of gravity Ppc forms multiple clusters CLp that are spaced apart from one another. Multiple centers of gravity Ppc (i.e., multiple warp portions Pp) belonging to one cluster CLp represent one warp thread Wp. The processor 210 obtains multiple clusters CLp corresponding to all of the multiple warp threads Wp of the fabric 700 by using the multiple warp portions Pp included in the combined image IMrc (FIG. 5(B)). The order of the multiple clusters CLp in the first direction Dx is the same as the order of the multiple warp threads Wp in the vertical direction Dt in the fabric 700 (FIG. 2). The processor 210 associates the target yarn portion with the warp thread Wp in the same order as the order of the cluster CLp that includes the target yarn portion.

[0066] Similarly, when the target yarn portion is a weft portion Pf, the processor 210 can determine the correspondence between the target yarn portion and the weft Wf. The processor 210 calculates the center of gravity Pfc of each weft portion Pf. The processor 210 obtains the distribution of the frequency FQ of the positions of the centers of gravity Pfc in the second direction Dy by projecting the centers of gravity Pfc onto a projection line PLf perpendicular to the first direction Dx. In this embodiment, the weft Wf is approximately parallel to the first direction Dx. Therefore, the distribution of the frequency FQ of the centers of gravity Pfc forms multiple clusters CLf that are spaced apart from each other. Multiple centers of gravity Pfc (i.e., multiple weft portions Pf) belonging to one cluster CLf represent one weft Wf. The processor 210 can obtain multiple clusters CLf corresponding to the multiple wefts Wf of the fabric 700 by using the multiple weft portions Pf included in the combined image IMrc. The order of the clusters CLf in the second direction Dy indicates the order of the weft yarns Wf in the fabric 700 (FIG. 2) in the conveyance direction Df.

[0067] One combined image IMrc represents only a portion of multiple weft threads Wf among all the weft threads Wf of the fabric 700. Therefore, it is not easy to determine the correspondence between the cluster CLf and the weft threads Wf in the fabric 700 using only one combined image IMrc. The processor 210 may acquire a user instruction specifying a reference correspondence, which is a correspondence between one weft thread portion Pf on the combined image IMrc or one cluster CLf and one weft thread Wf in the fabric 700. The processor 210 can determine the correspondence between multiple clusters CLf and multiple weft threads Wf in the fabric 700 according to the reference correspondence. Note that the user instruction specifying the reference correspondence may be acquired in S245 ( FIG. 6 ) or may be acquired before S245.

[0068] In S250, the processor 210 determines a threshold value for each class color. As will be described later, the processor 210 determines, in principle, the representative color Cw as the class color having a number of votes equal to or greater than the threshold value. In this embodiment, the processor 210 determines the threshold value using a color determination model associated with the dominant color component among the color determination models CMr, CMg, and CMb (FIG. 1) associated with red R, green G, and blue B, respectively.

[0069] FIG. 9 is a diagram showing an example of a color determination model (also simply referred to as a determination model). FIG. 9 shows an example of a determination model CMb used when the dominant color component is blue B. In this embodiment, the determination model CMb is a table that defines a reference threshold and an adjustment amount associated with the variation parameter P1 for each class color (here, blue CL1, navy blue CL2, and light blue CL3). The determination model CMb also defines a color priority order, which is the priority order of the class colors (here, blue, navy blue, and light blue).

[0070] A reference threshold is associated with each class color CL1, CL2, and CL3. The reference threshold is experimentally determined in advance so that an appropriate representative color Cw is determined from the three candidate colors CC1, CC2, and CC3 (FIG. 7(B)). For example, the reference threshold may be larger the easier it is to obtain a number of votes. The ease of obtaining a number of votes may vary depending on various factors, such as the performance of the color classification model and the appearance of the color of the yarn. As shown in the figure, the reference threshold may differ among multiple class colors. Alternatively, the reference threshold may be common to all class colors.

[0071] Each class color CL1, CL2, and CL3 is further associated with an adjustment amount for each variation parameter P1. The adjustment amount indicates the amount of adjustment relative to a reference threshold. For example, if the variation parameter P1 indicates the inner periphery of the cheese, the adjustment amount for blue CL1 is zero, the adjustment amount for navy blue CL2 is -1, and the adjustment amount for light blue CL3 is +1. The numbers in parentheses indicate the adjusted thresholds.

[0072] The likelihood of obtaining a number of votes may vary depending on the variable parameter P1. The adjustment amount may be experimentally determined in advance to mitigate the effect of changes in the likelihood of obtaining a number of votes on the determination of the representative color Cw. For example, the yarn may be more difficult to dye in the inner portion of the cheese than in the outer portion of the cheese. If the yarn is dyed blue, the color of the yarn in the inner portion of the cheese may be lighter than the intended blue. Such a color change is within the acceptable range. However, the candidate color for the yarn portion image of such yarn may be erroneously determined to be light blue CL3 instead of blue CL1. The number of votes for light blue CL3 may then be erroneously increased. In such a case, the adjustment amount for light blue CL3 may be +1 or more. This reduces the possibility that the representative color Cw of the blue target yarn portion is erroneously determined to be light blue CL3. Furthermore, if the yarn is dyed navy blue, the color of the yarn in the inner portion of the cheese may be lighter than the intended navy blue. Such a color change is within the acceptable range. However, the candidate color of the thread portion image of such a thread may be erroneously determined to be blue CL1 instead of navy blue CL2. As a result, the number of votes for navy blue CL2 may be erroneously reduced. In such a case, the adjustment amount for navy blue CL2 may be -1 or less. This makes it easier for the representative color Cw to be determined to be navy blue CL2. This reduces the possibility that the representative color Cw of the navy blue target thread portion will be erroneously determined to be a color other than navy blue CL2.

[0073] The same applies when the yarn is susceptible to dyeing due to the variable parameter P1. For example, the yarn may be more susceptible to dyeing on the outer periphery of the cheese than on the inner periphery of the cheese. If the yarn is dyed blue, the color of the yarn on the outer periphery of the cheese may be darker than the intended blue. This color change is within the acceptable range. However, the candidate color for the yarn portion image of such a yarn may be erroneously determined to be navy blue CL2 instead of blue CL1. The number of votes for navy blue CL2 may then be erroneously increased. In such a case, the adjustment amount for navy blue CL2 may be +1 or more. This reduces the possibility that the representative color Cw of the blue target yarn portion is erroneously determined to be navy blue CL2. Also, if the yarn is dyed light blue, the color of the yarn on the outer periphery of the cheese may be darker than the intended light blue. This color change is within the acceptable range. However, the candidate color for the yarn portion image of such a yarn may be erroneously determined to be blue CL1 instead of light blue CL3. The number of votes for light blue CL3 may then be erroneously decreased. In such a case, the adjustment amount of the light blue CL3 may be -1 or less. This makes it easier for the representative color Cw to be determined to be the light blue CL3. This reduces the possibility that the representative color Cw of the light blue target yarn portion will be erroneously determined to be a color different from the light blue CL3.

[0074] The processor 210 determines the threshold values ​​of the class colors CL1, CL2, and CL3 by referring to the determination model CMb and adjusting the reference threshold values ​​by the adjustment amount associated with the variation parameter P1.

[0075] Although not shown in the figure, the configurations of the determination models CMr and CMg associated with red R and green G are similar to the configuration of the determination model CMb for blue B.

[0076] In S255 (FIG. 6), the processor 210 determines a representative color Cw using multiple candidate colors CC1, CC2, and CC3 (FIG. 7(B)) associated with the target yarn portion and a threshold value for each class color. The processor 210 counts the number of votes, which is the total number of candidate colors, for each class color of the color classification model associated with the dominant color component. The processor 210 then identifies class colors that have a number of votes equal to or greater than the threshold value, and determines the representative color Cw as that class color.

[0077] For example, if the candidate colors CC1, CC2, and CC3 are blue, light blue, and light blue, the vote counts for blue CL1, navy blue CL2, and light blue CL3 (FIG. 9) are 1, 0, and 2. When the variation parameter P1 indicates the inner periphery of the cheese, the vote count (1) for blue CL1 is equal to or greater than the threshold value (1), and the vote counts (0, 2) for navy blue CL2 and light blue CL3 are less than the threshold value (1, 3). Therefore, the representative color Cw is determined to be blue CL1.

[0078] When the variable parameter P1 indicates the outer periphery of the cheese, the number of votes (1) for blue CL1 is equal to or greater than the threshold (1), and the number of votes (2) for light blue CL3 is equal to or greater than the threshold (1). In this way, multiple class colors may have a number of votes equal to or greater than the threshold associated with the class color. In this case, the processor 210 determines the representative color Cw according to a predetermined color priority order. For example, the determination model CMb in FIG. 9 specifies the priority order as blue, navy blue, and light blue. Between blue CL1 and light blue CL3, the processor 210 determines blue CL1, which has the highest priority, as the representative color Cw.

[0079] The color priority order may be various. In this embodiment, the color priority order is the order of decreasing likelihood of misclassification by the color classification models M2r, M2g, and M2b. The likelihood of misclassification by the third color classification model M2b is determined as follows: 100 thread portion images representing thread portions of blue thread, 100 thread portion images representing thread portions of navy blue thread, and 100 partial images representing thread portions of light blue thread are prepared. The color of each thread portion image is determined by inputting each of the 300 thread portion images into the third color classification model M2b. The total number of images incorrectly classified as blue CL1, the total number of images incorrectly classified as navy blue CL2, and the total number of images incorrectly classified as light blue CL3 are calculated. The total number of incorrectly classified images indicates the likelihood of misclassification. In the example of FIG. 9, the order of decreasing likelihood of misclassification is blue, navy blue, and light blue. In this case, the color priority order may be determined as blue, navy blue, and light blue.

[0080] The possibility of misclassification by other color classification models M2r and M2g is calculated in a similar manner. Generally, the possibility of misclassification by a color classification model can be calculated as follows: For each class color, K (e.g., K=100) thread portion images representing thread portions of the thread of the class color are prepared. The color of each image is determined by inputting multiple thread portion images into the color classification model. For each class color, the total number of thread portion images that are incorrectly classified into the class color is calculated. The calculated total number indicates the likelihood of misclassification. Note that multiple illumination directions may be used, as in the example of FIG. 2. In this case, it is preferable that the K thread portion images include approximately the same number of thread portion images for each illumination direction.

[0081] In S260 (FIG. 6), processor 210 determines whether processing of all thread portions is complete. If unprocessed thread portions remain (S260: No), processor 210 proceeds to S215 and processes the unprocessed thread portions. If processing of all thread portions is complete (S260: Yes), processor 210 ends the processing of FIG. 6, i.e., the processing of S153 in FIG. 4.

[0082] In S156, the processor 210 determines the color Cs of each of the multiple threads using the multiple representative colors Cw of the multiple thread portions. As described in FIG. 5(C), one warp thread Wp includes multiple warp portions Pp. The processor 210 determines the color Cs of the warp thread Wp by majority voting using the multiple representative colors Cw of the multiple warp portions Pp included in the single warp thread Wp. Similarly, one weft thread Wf includes multiple weft portions Pf. The processor 210 determines the color Cs of the weft thread Wf by majority voting using the multiple representative colors Cw of the multiple weft portions Pf included in the single weft thread Wf. The correspondence between the warp portions Pp and the warp thread Wp and the correspondence between the weft portions Pf and the weft thread Wf can be determined by the method described in S245 (FIG. 6).

[0083] In S160, the processor 210 inspects the color Cs of each of the multiple threads. In this embodiment, the processor 210 obtains the design color of each thread by referring to the reference pattern D1 ( FIG. 1 ). The processor 210 compares the thread color Cs with the design color for each thread. The processor 210 stores result data representing the comparison results for each of the multiple threads (e.g., whether the thread color Cs is the same as the design color) in the storage device 215 (e.g., the non-volatile storage device 230).

[0084] In S170, the processor 210 determines whether the inspection is complete. The condition for completing the inspection may be various. In this embodiment, the condition for completing the inspection is that the inspection of a predetermined portion of the fabric 700 (FIG. 2) is completed. If the inspection is not completed (S170: No), in S180, the processor 210 transports the fabric 700 a predetermined transport distance in the transport direction Df. This transport distance may be determined in advance so that no gap occurs between the inspected portion of the fabric 700 and the newly inspected portion. After transporting the predetermined transport distance (S180), the processor 210 proceeds to S110 and inspects a new portion of the fabric 700. If the inspection is completed (S170: Yes), the processor 210 ends the inspection process of FIG. 4.

[0085] As described above, in this embodiment, the processor 210 (FIG. 1) executes the following processes in accordance with the program 231. In S210-S225 (FIG. 6), the processor 210 acquires multiple thread portion images representing thread portions (for example, thread portion image IMw (FIG. 7(A))). The thread portion image is an example of a read thread image, which is a read image of a specific thread contained in the fabric 700. Among the multiple thread portion images representing a specific thread, one or both of the illumination direction and the thread portion represented by the thread portion image are different. The illumination direction is an example of a reading condition. Furthermore, if the thread portion represented by the thread portion image is different among the multiple thread portion images, the reading position on the fabric 700 is different among the multiple thread portion images. In this way, the thread portion represented by the thread portion image is also an example of a reading condition. In S235, the processor 210 uses each of the multiple thread portion images to determine a candidate color for each thread portion image (for example, candidate colors CC1-CC3 (FIG. 7(B))). The candidate colors CC1-CC3 are each candidates for the color of the specific thread. In S255 (FIG. 6) and S156 (FIG. 4), the processor 210 determines the color Cs of the specific thread using the multiple candidate colors of the multiple thread portion images.

[0086] In this way, the processor 210 uses each of the multiple thread portion images to determine candidate colors CC1-CC3, which are candidates for the color of a specific thread, for each thread portion image, and determines the color Cs of the specific thread using the multiple candidate colors of the multiple thread portion images. The multiple thread portion images include multiple thread portion images scanned under different scanning conditions. Therefore, the processor 210 can appropriately determine the color of the specific thread. For example, as described with reference to FIGS. 3(A)-3(C), the apparent color of the thread in the scanned image may change depending on the illumination direction. In general, the apparent color of the thread in the scanned image may change depending on the scanning conditions. The candidate color of the thread portion image may be determined to be an incorrect color due to the scanning conditions. In this embodiment, the processor 210 determines the color Cs of the specific thread using multiple candidate colors CC1-CC3 of the multiple thread portion images associated with different scanning conditions. Even if one candidate color shows an incorrect color different from the actual color due to reading conditions, the processor 210 can appropriately determine the color Cs of a particular thread by using multiple candidate colors CC1-CC3.

[0087] In this embodiment, as shown in FIGS. 3A-3C, each thread Wp (Wf) includes multiple thread portions Pp (Pf). As described with reference to FIGS. 6 and 7B, the processor 210 determines multiple candidate colors CC1, CC2, and CC3 associated with a target thread portion, which is a single thread portion. As described in S150 (FIG. 4), the processor 210 determines the color Cs of a specific thread using multiple candidate colors of multiple thread portions. As described in S255 (FIG. 6) and 7B, the processor 210 determines a representative color Cw of the target thread portion using multiple candidate colors CC1, CC2, and CC3 associated with the target thread portion. The group of multiple candidate colors CC1, CC2, and CC3 associated with the target thread portion (i.e., one thread portion) is an example of a processing group used to determine the representative color Cw.

[0088] In S255 (FIG. 6), processor 210 uses the processing group (candidate colors CC1, CC2, CC3) to calculate the number of votes for each class color CL1, CL2, CL3. That is, each of candidate colors CC1, CC2, CC3 of the processing group is classified into one of class colors CL1, CL2, CL3. One or more candidate colors belonging to one class color are an example of a same-color group, which is a group of candidate colors that indicate the same color. If the total number of candidate colors included in the same-color group (i.e., the number of votes) is equal to or greater than a threshold, processor 210 determines the representative color Cw of the target yarn portion (i.e., the processing group) as a color of the same-color group. In S156 (FIG. 4), processor 210 determines the color Cs of the specific yarn using the representative color Cw. Furthermore, in S245 and S250 (FIG. 6), processor 210 determines the threshold using variable parameter P1. In this example, the variation parameter P1 indicates whether the yarn represented by the target yarn portion is included in the inner or outer portion of the cheese, and is an example of a variation factor associated with a particular yarn.

[0089] When the variable factors fluctuate, the possibility of misclassifying colors by the color classification models M2r, M2g, and M2b can increase. The possibility of misclassifying colors can be reduced by retraining the color classification models M2r, M2g, and M2b. However, retraining the color classification models M2r, M2g, and M2b every time the variable factors fluctuate requires preparing new training images and adjusting the parameters of the models M2r, M2g, and M2b using a computer, which is a significant burden. In this embodiment, the processor 210 determines a threshold value using the variable factors. Then, if the same-color group contains a number of candidate colors equal to or greater than the threshold value, the processor 210 determines the representative color Cw of the processing group as the color of the same-color group. Therefore, even when the variable factors fluctuate, the processor 210 can appropriately determine the representative color Cw of the processing group. Retraining the color classification models M2r, M2g, and M2b can be omitted.

[0090] Furthermore, in this embodiment, the processing group (candidate colors CC1, CC2, and CC3) may be classified into multiple class colors. That is, the processing group may include multiple same-color groups that represent different colors. Here, multiple class colors may have a number of votes equal to or greater than the threshold associated with the class color. That is, each of the multiple same-color groups may include a number of candidate colors equal to or greater than the threshold associated with the same-color group. In this case, the processor 210 determines the representative color Cw of the target yarn portion (i.e., the processing group) to be the color with the highest priority based on a predetermined color priority order among the multiple class colors (i.e., the multiple colors of the multiple same-color groups). Therefore, the processor 210 can determine the representative color Cw to be an appropriate color based on the color priority order.

[0091] In this embodiment, as described in S235 (FIG. 6), the processor 210 inputs the thread portion image into one of the trained color classification models M2r, M2g, or M2b to determine candidate colors for the thread portion image. Each of the color classification models M2r, M2g, or M2b is trained to classify the thread color represented by the image input to the color classification models M2r, M2g, or M2b into one of multiple colors (e.g., class colors CL1-CL3). As described in FIG. 9, the color priority order is configured so that a color more likely to be misclassified by the color classification models M2r, M2g, or M2b has a lower priority. This reduces the likelihood that a misclassified color will be selected as the representative color Cw.

[0092] In this embodiment, the yarn dyeing method is cheese dyeing. As described in FIG. 9, the variable parameter P1 indicates a portion of the inner and outer peripheral portions of the raw yarn (i.e., cheese) wound for cheese dyeing that corresponds to a specific yarn (the portion corresponding to the specific yarn is referred to as a specific cheese portion). Therefore, the processor 210 can determine a representative color Cw suitable for the specific cheese portion.

[0093] Furthermore, in this embodiment, as described in S210-S225 (FIG. 6), a plurality of thread portion images corresponding to different illumination directions are associated with the target thread portion. That is, the plurality of reading conditions associated with the plurality of thread portion images include a plurality of conditions indicating different illumination directions when reading a specific thread to generate the thread portion image (S130 (FIG. 4)). The apparent color of the thread may change depending on the illumination direction. In such a case, the processor 210 can appropriately determine the color Cs of the specific thread by using the plurality of thread portion images corresponding to different illumination directions.

[0094] Furthermore, in this embodiment, as described in FIG. 5(C), the threads Wp and Wf form multiple thread portions Pp and Pf that are visible on the fabric 700 and are not hidden by other threads. The multiple thread portion images representing a specific thread include multiple thread portion images representing different thread portions. As described above, the thread portions (i.e., reading positions) represented by the thread portion images are examples of reading conditions. Thus, the multiple reading conditions associated with the multiple thread portion images include multiple conditions representing different thread portions represented by the thread portion images. Furthermore, the multiple reading conditions associated with the multiple thread portion images representing a specific thread differ in either or both of the thread portion and the illumination direction. In S255 (FIG. 6), the processor 210 determines the representative color Cw of the common thread portion using multiple candidate colors CC1, CC2, and CC3 of the multiple thread portion images representing a common thread portion and associated with different illumination directions. In S156 (FIG. 4), the processor 210 determines the color of the specific thread using the respective representative colors Cw of the multiple thread portions. Therefore, when the apparent color of the thread changes depending on the lighting direction, the processor 210 can appropriately determine the representative color Cw of the common thread portion. Then, the processor 210 can appropriately determine the color Cs of a specific thread by using the representative colors Cw of each of the multiple thread portions.

[0095] Furthermore, in this embodiment, as described in S235 (FIG. 6), the processor 210 determines candidate colors for the thread portion image by inputting the thread portion image into one of the trained color classification models M2r, M2g, and M2b. The color classification models M2r, M2g, and M2b are each trained to classify the thread color represented by the image input to the color classification models M2r, M2g, and M2b into one of multiple colors. Therefore, the processor 210 can appropriately determine candidate colors for the thread portion image.

[0096] In S255 (FIG. 6), if multiple class colors have a number of votes equal to or greater than the threshold value associated with the class color, the processor 210 determines the representative color Cw in accordance with color priority. Alternatively, the processor 210 may determine the representative color Cw to be the color in the same color group that includes the most candidate colors (i.e., the class color with the most votes). This configuration allows the processor 210 to reduce the possibility of determining an erroneously determined candidate color as the representative color Cw.

[0097] B. Second Example: Fig. 10 is a flowchart showing a second embodiment of the inspection process. There are two differences from the embodiment of Fig. 4. The first difference is that a single light source (here, the second light source 132) is used instead of the three light sources 131, 132, and 133. That is, the illumination direction is fixed. The second difference is that the color of a specific yarn is determined using multiple candidate colors of multiple yarn portion images representing the specific yarn and a threshold value for each class color.

[0098] In S120b, the processor 210 turns on the light source 132 (the other light sources 131 and 133 are turned off). In S130, the fabric 700 is photographed. The processing of S130 is the same as the processing of S130 in FIG. 4. The processor 210 acquires data of read images IMr1-IMr4 (FIG. 5(A)) from each of the digital cameras 111-114. The processor 210 combines the read images IMr1-IMr4 to generate data of a combined image IMrc (FIG. 5(B)).

[0099] In S150b, the processor 210 determines the color of each yarn using the combined image IMrc, which is an example of a scanned image. In this embodiment, S150b includes S153b and S156b. In S153b, the processor 210 determines the color of each of the multiple warp yarns Wp.

[0100] 11 is a flowchart showing an example of a process for determining the color of the warp thread Wp. In S210b, the processor 210 detects the thread portion of the warp thread Wp using the combined image IMrc (FIG. 5(B)). The method for detecting the thread portion is the same as the method for detecting the thread portion in S210 (FIG. 6). The processor 210 detects the warp thread portion Pp (FIG. 5(C)) using the trained object detection model M1 (FIG. 1).

[0101] In S225b, the processor 210 selects an unprocessed warp thread portion Pp from the multiple warp thread portions Pp detected in S210b (the selected warp thread portion Pp is referred to as the target warp thread portion Pp). The processor 210 acquires a warp thread portion image, which is an image of the target warp thread portion Pp, from the combined image IMrc. The thread portion image IMw in FIG. 7(A) is an example of a warp thread portion image. The method for acquiring the warp thread portion image is the same as the method for acquiring the thread portion image in S225 (FIG. 6).

[0102] S230b and S235b are the same as S230 and S235 in Fig. 6, respectively, except that a warp portion image is used instead of a thread portion image. In S230b, the processor 210 determines a dominant color component of the warp portion image. In S235b, the processor 210 determines candidate colors by inputting the warp portion image into a color classification model associated with the dominant color component.

[0103] In S238b, the processor 210 determines whether or not the processing of all warp thread portions Pp is complete. If an unprocessed warp thread portion Pp remains (S238b: No), the processor 210 proceeds to S225b and processes the unprocessed warp thread portion Pp. If the processing of all warp thread portions Pp is complete (S238b: Yes), the processor 210 proceeds to S242b.

[0104] In S242b, the processor 210 selects an unprocessed warp thread Wp from among the multiple warp threads Wp represented by the combined image IMrc (the selected warp thread Wp is referred to as the target warp thread Wp). The processor 210 determines the correspondence between the multiple warp thread portions Pp and the multiple warp threads Wp according to the method described with reference to FIG. 5(C). FIG. 12(A) is a diagram showing an example of multiple warp thread portion images IMpp and multiple warp thread portions Pp associated with one warp thread Wp. The projection line PLp and cluster CLp in the figure are the same as the projection line PLp and cluster CLp in FIG. 5(C). The multiple warp thread portions Pp forming one cluster CLp are associated with one warp thread Wp. A candidate color CCp is associated with each warp thread portion image IMpp. The processor 210 selects the target warp thread Wp from among the multiple warp threads Wp represented by the multiple clusters CLp.

[0105] In S245b-S255b (Fig. 11), the processor 210 determines the color Cp of the target warp thread Wp using a plurality of candidate colors CCp of a plurality of warp portions Pp associated with the target warp thread Wp. As in S245-S255 of Fig. 6, the processor 210 determines variation parameters (S245b), determines threshold values ​​for each class color according to the variation parameters (S250b), and determines the color Cp of the target warp thread Wp using the plurality of candidate colors CCp and the threshold values ​​for each class color (S255b). Details will be described below.

[0106] In S245b, the processor 210 determines the fluctuation parameters of the target warp yarn Wp. In this embodiment, the fluctuation parameters include the following parameters P1 to P4. P1: Position of the thread in the cheese (outer or inner side) P2: Country of origin of the yarn (e.g., Country A or Country B). P3: Lot identifier of the stain used for staining (e.g., Identifier 1 or Identifier 2) P4: Illumination direction (e.g., diagonal left, front, or diagonal right) when reading the yarn (S130 (FIG. 10)) for generating the warp yarn partial image IMpp

[0107] The parameters P1-P3 indicate variables related to the yarns. The parameter P4 indicates variables related to the acquisition of the warp portion image IMpp. When the second light source 132 (FIG. 2) is used, the illumination direction is frontal. In this embodiment, the reference pattern D1 (FIG. 1) represents appropriate parameters P1-P4 for each of the multiple warp yarns Wp and multiple weft yarns Wf included in the fabric 700. The processor 210 determines the parameters P1-P4 of the target warp yarn Wp by referring to the reference pattern D1.

[0108] In S250b, the processor 210 determines a threshold value for each class color. In this embodiment, the processor 210 determines the threshold value using a color determination model associated with a dominant color component among the color determination models associated with red R, green G, and blue B, respectively. Here, the dominant color component used is the dominant color component of the warp portion Pp associated with the target warp thread Wp (S230b (FIG. 11)). Normally, the dominant color component is the same among multiple warp portions Pp associated with the target warp thread Wp. If the dominant color components differ among multiple warp portions Pp, the processor 210 may determine the dominant color component by majority vote.

[0109] FIG. 12(B) is a diagram showing an example of a color determination model. FIG. 12(B) shows an example of a determination model CMb2 used when the dominant color component is blue B. Like the determination model CMb in FIG. 9, the determination model CMb2 defines a reference threshold and an adjustment amount associated with a variation parameter for each class color (here, blue CL1, navy blue CL2, and light blue CL3). The determination model CMb2 also defines a color priority order, which is the order of priority for class colors (here, blue, navy blue, and light blue). Like the example in FIG. 9, the color priority order is an order of decreasing likelihood of misclassification.

[0110] The total number of candidate colors CCp used to determine the color Cp of the target warp thread Wp (i.e., the total number of warp thread portions Pp associated with the target warp thread Wp) is greater than 3 (e.g., greater than or equal to 10 and less than or equal to 100). Therefore, the reference threshold value is larger than the reference threshold value in FIG. 9.

[0111] Furthermore, the determination model CMb2, unlike the determination model CMb in Fig. 9, defines an adjustment amount associated with each of the multiple parameters P1-P4. Similar to the adjustment amount in Fig. 9, the adjustment amount of each variable parameter may be experimentally determined in advance so as to mitigate the influence of changes in the likelihood of obtaining votes due to the variable parameter on the determination of a color.

[0112] The processor 210 adjusts the reference threshold using the total value of the adjustment amounts of the plurality of variation parameters P1-P4, thereby determining the thresholds for each of the class colors CL1, CL2, and CL3.

[0113] Although not shown in the drawings, the configurations of the determination models associated with red R and green G are similar to the configuration of the determination model CMb2 for blue B.

[0114] In S255b (FIG. 11), the processor 210 determines the color Cp of the target warp thread Wp using multiple candidate colors CCp (FIG. 12(A)) associated with the target warp thread Wp and threshold values ​​for each class color. The method for determining the color Cp of the target warp thread Wp is the same as the method for determining the representative color Cw in S255 (FIG. 6). The processor 210 counts the number of votes for each class color of the color classification model associated with the dominant color component. Then, the processor 210 identifies a class color CLc having a number of votes equal to or greater than the threshold value, and determines the color Cp of the target warp thread Wp to be that class color CLc. If multiple class colors have a number of votes equal to or greater than the threshold value associated with the class color, the processor 210 determines the color Cp of the target warp thread Wp in accordance with color priority order.

[0115] In S262b, the processor 210 determines whether or not the processing of all warp threads Wp is complete. If unprocessed warp threads Wp remain (S262b: No), the processor 210 proceeds to S242b and processes the unprocessed warp threads Wp. If the processing of all warp threads Wp is complete (S262b: Yes), the processor 210 ends the processing of Fig. 11, i.e., the processing of S153b in Fig. 10.

[0116] In S156b, the processor 210 determines the color of each of the multiple wefts Wf. The process for determining the color of the weft Wf is similar to the process for determining the color Cp of the warp Wp (FIG. 11) by replacing the warp portion Pp with the weft portion Pf (FIG. 5(C)) and the warp Wp with the weft Wf (not shown). The processor 210 can determine the correspondence between the multiple weft portions Pf and the multiple wefts Wf by using the cluster CLf described in FIG. 5(C).

[0117] Note that the length of the combined image IMrc (FIG. 5(B)) in the first direction Dx is longer than the length in the second direction Dy. Therefore, the total number of weft portions Pf associated with one weft Wf is greater than the total number of warp portions Pp associated with one warp Wp. In this case, the color determination model for the weft Wf may be different from the color determination model for the warp Wp (e.g., determination model CMb2 (FIG. 12(B))). For example, the reference threshold for the weft Wf may be greater than the reference threshold for the warp Wp.

[0118] In addition, in determining the variable parameters P1-P4 for the weft yarn Wf, as described in S245 (FIG. 6), the processor 210 may determine the correspondence between the multiple clusters CLf and the multiple weft yarns Wf in the fabric 700 according to the reference correspondence.

[0119] As described above, in S150b (FIG. 10), the processor 210 determines the color of each of the multiple warp yarns Wp and multiple weft yarns Wf represented by the combined image IMrc. In S160, the processor 210 inspects the color of each of the multiple yarns. The processing of S160 is the same as the processing of S160 in FIG. 4.

[0120] The subsequent processes of S170 and S180 are the same as the processes of S170 and S180 in Fig. 4. If the inspection is not complete (S170: No), in S180, the processor 210 transports the fabric 700 in the transport direction Df by a predetermined transport distance. After S180, the processor 210 proceeds to S130 and inspects a new portion of the fabric 700. If the inspection is complete (S170: Yes), the processor 210 ends the inspection process in Fig. 10.

[0121] As described above, in this embodiment, the processor 210 (FIG. 1) executes the following processes in accordance with the program 231. In steps S210b-S225b (FIG. 11), the processor 210 acquires multiple warp thread portion images IMpp representing the warp thread portion Pp (FIG. 12(A)). The warp thread portion image IMpp is an example of a read thread image, which is a read image of the target warp thread Wp included in the fabric 700. The warp thread portions Pp represented by the warp thread portion images IMpp differ among the multiple warp thread portion images IMpp representing the target warp thread Wp. The warp thread portion Pp represented by the warp thread portion image IMpp (i.e., the read position) is an example of a read condition. In step S235b, the processor 210 uses each of the multiple warp thread portion images IMpp to determine a candidate color CCp for each warp thread portion image IMpp. The candidate color CCp is a candidate color for the target warp thread Wp. In S255b, the processor 210 determines the color Cp of the target warp thread Wp using the multiple candidate colors CCp of the multiple warp thread portion images IMpp.

[0122] With this configuration, the processor 210 can appropriately determine the color Cp of the target warp thread Wp. For example, the apparent color of some of the multiple warp thread portions Pp may differ from the actual color due to various causes (e.g., yarn curvature, etc.). In this embodiment, the processor 210 determines the color Cp of the target warp thread Wp using multiple candidate colors CCp of multiple warp thread portion images IMpp associated with different reading conditions (here, different warp thread portions Pp). Therefore, even if some of the multiple candidate colors CCp indicate an incorrect color different from the actual color, the processor 210 can appropriately determine the color Cp of the target warp thread Wp.

[0123] Furthermore, in this embodiment, as described in FIG. 5(C), the threads Wp and Wf form multiple thread portions Pp and Pf that are visible on the fabric 700 and are not hidden by other threads. As shown in FIG. 12(A), multiple warp thread portion images IMpp representing the warp thread Wp include multiple warp thread portion images IMpp representing different warp thread portions Pp. As described above, the warp thread portion Pp represented by the warp thread portion image IMpp (i.e., the reading position) is an example of a reading condition. Thus, the multiple reading conditions associated with the multiple warp thread portion images IMpp include multiple conditions indicating different warp thread portions Pp represented by the warp thread portion images IMpp. In S255b (FIG. 11), the processor 210 determines the color Cp of the target warp thread Wp using the candidate colors CCp of each of the multiple warp thread portion images IMpp representing different warp thread portions Pp. Therefore, the processor 210 can appropriately determine the color Cp of the target warp thread Wp.

[0124] In this embodiment, the processor 210 executes the following process in S255b (FIG. 11). The processor 210 uses multiple candidate colors CCp associated with the target warp thread Wp to calculate the number of votes for each class color CL1, CL2, and CL3. One or more candidate colors CCp belonging to one class color are an example of a same-color group, which is a group of candidate colors indicating the same color. The processor 210 determines the color Cp of the target warp thread Wp as the class color CLc of the same-color group having a number of votes equal to or greater than a threshold. That is, the processor 210 refers to the total number of candidate colors CCp in each same-color group (i.e., the number of votes) to determine the class color CLc of the same-color group having a number of votes equal to or greater than a threshold, and determines the color Cp of the target warp thread Wp as that class color CLc. The class color CLc used as the color Cp of the target warp thread Wp is an example of a representative color representing multiple candidate colors CCp (also referred to as the representative color CLc). The group of multiple candidate colors CCp associated with the target warp thread Wp is an example of a processing group used to determine the representative color CLc. Furthermore, in S245b and S250b, the processor 210 determines the threshold value using the variable parameters P1-P4. As a result, even if the variable parameters P1-P4 vary, the processor 210 can appropriately determine the representative color CLc, and ultimately the color Cp of the target warp thread.

[0125] In this embodiment, as shown in Fig. 12(B), the processor 210 determines the threshold value using the variable parameters P1-P4. Therefore, when the variable parameters P1-P4 vary, the processor 210 can appropriately determine the color Cp of the target warp thread Wp.

[0126] The process of FIG. 11 includes various processes having algorithms common to the processes included in the process of FIG. 6. For example, in S255b (FIG. 11), when multiple class colors have a number of votes equal to or greater than the threshold value associated with the class color, the processor 210 determines the representative color CLc, and therefore the color Cp of the target warp thread Wp, as the color with the highest priority based on the color priority order. Also, in S235b, the processor 210 inputs the warp thread partial image IMpp (FIG. 12(A)) into one of the trained color classification models M2r, M2g, and M2b to determine the candidate color CCp of the warp thread partial image IMpp. The color priority order (e.g., FIG. 12(B)) is configured so that colors more likely to be misclassified by the color classification models M2r, M2g, and M2b have a lower priority. As described above, this embodiment can provide various advantages similar to those of the embodiment of FIG. 6. In addition, in S255b (Figure 11), if multiple class colors have a number of votes equal to or greater than the threshold value associated with the class color, the processor 210 may determine the representative color CLc, and therefore the color Cp of the target warp thread Wp, to be the color of the same color group that includes the most candidate colors (i.e., the class color with the most votes).

[0127] Although not shown in the drawings, the processor 210 determines the color of the weft thread Wf in a manner similar to the method for determining the color Cp of the warp thread Wp. Therefore, the processor 210 can appropriately determine the color of the weft thread Wf in the same manner as the color Cp of the warp thread Wp.

[0128] C. Variations: (1) Because the fabric 700 is soft, it may deform in various ways. For example, the fabric 700 may be skewed. In this case, in the combined image IMrc ( FIG. 5(B) ), the warp threads Wp may be skewed with respect to the second direction Dy, and the weft threads Wf may be skewed with respect to the first direction Dx. The processor 210 may perform skew correction on the scanned image (e.g., the combined image IMrc). The skew correction may be performed by various processes (e.g., affine transformation) to correct the scanned image so that it represents a non-skewed fabric 700. Various methods may be used to determine the skew correction parameters. For example, the processor 210 may use a Hough transform to detect the direction parallel to the warp threads Wp and the direction parallel to the weft threads Wf. Alternatively, the processor 210 may determine the skew correction parameters according to information input by the operator. Furthermore, the warp threads Wp and the weft threads Wf may meander in the scanned image. In this case, the processor 210 may classify the multiple thread portions into groups by tracing the extension direction of each thread portion. Various methods may be used to determine the extension direction of the thread portion. For example, the processor 210 may obtain a thin line by thinning the region of the thread portion, calculate an approximate straight line of the thin line, and use a direction parallel to the approximate straight line as the extension direction of the thread portion. For example, Hilditch, Zhang-Suen, or Nagendraprasad-Wang-Gupta thinning may be used as the thinning method.

[0129] (2) The color priority order (e.g., FIG. 9 and FIG. 12(B)) may be various orders other than the order of decreasing probability of misclassification. For example, the color priority order may be the order of increasing frequency of color use (i.e., increasing frequency of thread use).

[0130] (3) The color classification models M2r, M2g, and M2b (FIG. 1) may be various other models instead of convolutional neural networks. For example, the color classification models M2r, M2g, and M2b may be formed by multiple fully connected layers. A common color classification model may be used regardless of the dominant color component. In this case, steps S230 (FIG. 6) and S230b (FIG. 11) may be omitted. The method for determining candidate colors is not limited to using a machine learning model, and various methods for determining representative colors of an image may be used. For example, the processor 210 may determine candidate colors based on various summary statistics (e.g., average color, most frequent color, etc.) of multiple color values ​​of multiple pixels included in the image.

[0131] (4) The total number of light sources used to read the fabric 700 may be any number equal to or greater than one. K light sources (where K is an integer equal to or greater than two) associated with different lighting directions may be used. In the embodiment of FIG. 6, the processor 210 may determine the representative color Cw using K candidate colors associated with the K lighting directions. In the embodiment of FIG. 11, a color determination model (e.g., color determination model CMb2 (FIG. 12(B))) may specify adjustment amounts for each of the K lighting directions.

[0132] (4) The variation parameters used in determining the threshold value (e.g., S250 (FIG. 6), S250b (FIG. 11)) may include one or more parameters arbitrarily selected from the above parameters P1-P4. For example, the determination models CMr, CMg, and CMb (FIG. 9) used in S250 (FIG. 6) may be configured to determine the threshold value according to parameter P2 (country of origin of the raw yarn) or parameter P3 (lot identifier of the dye). Furthermore, the determination models CMr, CMg, and CMb may be configured to determine the threshold value according to two or more variation parameters. The two or more variation parameters may be selected from the above variation parameters P1-P3. Furthermore, one or more parameters arbitrarily selected from the parameters P1-P4 may be omitted from the determination model used in S250b (FIG. 11) (e.g., determination model CMb2 (FIG. 12(B))). In either case, the variation parameters may include parameters indicating different types of variation factors from the above parameters P1-P4.

[0133] (5) The method of determining one color (e.g., representative colors Cw and CLc (FIGS. 7B and 12A)) from multiple candidate colors may be various methods for determining a representative color, instead of the method using thresholds determined for each class color as shown in FIGS. 9 and 12B. For example, a common threshold may be applied to multiple class colors. Processor 210 may determine one color by majority vote. Processor 210 may employ various summary statistics of multiple candidate colors (e.g., average color, most frequent color, etc.).

[0134] (6) The process for determining the thread color may be various other processes instead of the processes of the above-described embodiments and modifications. For example, in the embodiments of FIGS. 10 and 11, similar to the embodiment of FIG. 4, the processor 210 may use multiple light sources to acquire multiple scanned images associated with multiple illumination directions. Then, in steps S245b-S255b (FIG. 11), the processor 210 may determine the color Cp of the target warp thread Wp using multiple candidate colors associated with multiple combinations of multiple illumination directions and multiple warp thread portions. Here, the variable parameter P4 (illumination direction) may be omitted from the determination of the threshold. Furthermore, a portion of the combined image IMrc may be used to determine the thread color.

[0135] (7) The object detection model M1 (FIG. 1) may be various object detection models capable of detecting thread portions instead of RTMDet (e.g., Mask R-CNN, PaDIM, YOLO, etc.). Furthermore, the method for detecting thread portions may be various other methods instead of using a machine learning model. For example, the processor 210 may detect thread portions by template matching using multiple template images representing thread portions.

[0136] (8) In each of the above embodiments and modifications, the processor 210 may cause the GPU 260 to execute various calculations. For example, the processor 210 may cause the GPU 260 to execute some or all of the calculations performed by the object detection model M1. The processor 210 may also cause the GPU 260 to execute some or all of the calculations performed by the color classification models M2r, M2g, and M2b. The GPU 260 may be omitted.

[0137] (9) The reading device for reading the fabric 700 is not limited to the digital cameras 111-114, but may be any of various devices having an image sensor (CCD, CIS, etc.). For example, the fabric 700 may be read by a so-called flatbed scanner.

[0138] Furthermore, the method of acquiring a read yarn image (e.g., a yarn portion image IMw (FIG. 7(A))), which is a read image of a specific yarn contained in the fabric 700, is not limited to the above method and may be any method. For example, a control device (e.g., a computer different from the data processing device 200) that controls the reading device may generate data for the read yarn image using the read image of the fabric 700. The processor 210 may request the read yarn image from the control device and acquire the data for the read yarn image from the control device.

[0139] (10) Data processing device 200 in Fig. 1 may be a device of a type different from a personal computer (e.g., a digital camera, a scanner, or a smartphone). Furthermore, multiple devices (e.g., computers) that can communicate with each other via a network may share some of the data processing functions of the data processing device and collectively provide the data processing functions (a system including these devices corresponds to a data processing device).

[0140] In each of the above embodiments, some of the configurations implemented by hardware may be replaced with software, and conversely, some or all of the configurations implemented by software may be replaced with hardware. For example, the processing by the object detection model M1 may be performed by a dedicated hardware circuit such as an Application Specific Integrated Circuit (ASIC) instead of a program module. Similarly, the processing by the color classification models M2r, M2g, and M2b may be performed by a dedicated hardware circuit instead of a program module.

[0141] Furthermore, when some or all of the functions of the present disclosure are realized by a computer program, the program can be provided in a form stored on a computer-readable recording medium (e.g., a non-transitory recording medium). The program can be used in a state stored on the same or a different recording medium (computer-readable recording medium) from when it was provided. The "computer-readable recording medium" is not limited to portable recording media such as memory cards and CD-ROMs, but can also include internal storage devices within a computer, such as various ROMs, and external storage devices connected to a computer, such as a hard disk drive.

[0142] The above-described examples and modifications can be combined as appropriate. The above-described examples and modifications are provided to facilitate understanding of the present disclosure and are not intended to limit the present invention. The present invention may be modified or improved without departing from the spirit thereof, and the present invention includes equivalents thereof. [Explanation of symbols]

[0143] 10...Inspection device, 111-114...Digital camera, 120...Rotary encoder, 131...First light source, 132...Second light source, 133...Third light source, 200...Data processing device, 210...Processor, 215...Storage device, 220...Volatile storage device, 230...Non-volatile storage device, 231...Program, 240...Display unit, 250, 980...Operation unit, 270...Communication interface, 700...Fabric, 700e1...Left end, 700e2...Right end, 70 0F...flat portion, 900...conveying device, 910...first roller, 920...second roller, 950...conveying mechanism, 990...control unit, Db...reverse direction, Df...forward direction (conveying direction), Dt...vertical direction, Dx...first direction, Dy...second direction, M1...object detection model, M2r...first color classification model, M2g...second color classification model, M2b...third color classification model, P1, P2, P3, P4...variation parameters, Pf...weft portion, Pp...warp portion, Wf...weft, Wp...warp

Claims

1. A program, A function of acquiring multiple scanned yarn images, which are multiple images scanned under different scanning conditions of a specific yarn contained in a fabric containing multiple yarns; a candidate color determination function that determines a candidate color that is a candidate for the color of the specific yarn for each of the plurality of read yarn images using each of the plurality of read yarn images; a color determination function that determines the color of the specific yarn using a plurality of candidate colors of the plurality of read yarn images; A program that makes the computer realize the above.

2. 2. The program according to claim 1, the plurality of candidate colors includes a processing group that is a group of a plurality of candidate colors that is at least a part of the plurality of candidate colors; The color determination function comprises: When the processing group includes a same-color group, which is a group of candidate colors that indicate the same color, and the same-color group includes a number of candidate colors equal to or greater than a threshold value, determining the representative color of the processing group to be the color of the same-color group; determining a color of the particular yarn using the representative color; The program further comprises: causing a computer to perform a function of determining the threshold value using variables related to the acquisition of the scanned yarn image or the specific yarn; program.

3. 3. The program according to claim 2, The color determination function comprises: When the processing group includes a plurality of same-color groups that indicate mutually different colors, and each of the plurality of same-color groups includes a number of candidate colors equal to or greater than a threshold value associated with the same-color group, a representative color of the processing group is determined to be the color of the same-color group that includes the largest number of candidate colors. program.

4. 3. The program according to claim 2, The color determination function comprises: When the processing group includes a plurality of same-color groups that indicate mutually different colors, and each of the plurality of same-color groups includes a number of candidate colors equal to or greater than a threshold value associated with the same-color group, a representative color of the processing group is determined to be the color with the highest priority based on a predetermined color priority order among the plurality of colors of the plurality of same-color groups. program.

5. 5. The program according to claim 4, the candidate color determination function determines the candidate color of the read yarn image by inputting the read yarn image into a trained machine learning model, and the machine learning model has been trained to classify the color of the yarn represented by the image input to the machine learning model into one of a plurality of colors; The color priority order is configured such that colors that are more likely to be misclassified by the machine learning model have a lower priority. program.

6. 6. The program according to claim 2, The variables are: When the dyeing method of the specific yarn is cheese dyeing, a portion corresponding to the specific yarn among an inner peripheral portion and an outer peripheral portion of the raw yarn wound for the cheese dyeing; The country of origin of the specific yarn; a lot identifier of the dye used to dye the particular yarn; an illumination direction when reading the specific yarn to generate the read yarn image; including one or more of the following: program.

7. 6. The program according to claim 1, the plurality of reading conditions associated with the plurality of read yarn images include a plurality of conditions indicating different illumination directions when reading the specific yarn to generate the read yarn images; program.

8. 8. The program according to claim 7, The specific yarn forms a plurality of yarn portions that are visible portions on the fabric without being hidden by other yarns, the plurality of reading conditions include a plurality of conditions indicating different thread portions represented by the read thread image, One or both of the thread portion and the illumination direction are different among the plurality of reading conditions, The color determination function comprises: determining a representative color of the common thread portion using a plurality of candidate colors of a plurality of read thread images that represent the common thread portion and that correspond to different illumination directions; determining a color of the particular thread using a representative color of each of the plurality of thread portions; program.

9. 6. The program according to claim 1, The specific yarn forms a plurality of yarn portions that are visible portions on the fabric without being hidden by other yarns, the plurality of reading conditions include a plurality of conditions indicating different thread portions represented by the read thread image, the color determination function determines the color of the specific thread using the candidate colors of the plurality of read thread images representing different thread portions from each other; program.

10. 6. The program according to claim 1, the candidate color determination function determines the candidate color of the read yarn image by inputting the read yarn image into a trained machine learning model, and the machine learning model has been trained to classify the color of the yarn represented by the image input to the machine learning model into one of a plurality of colors. program.

11. 1. A data processing device, comprising: an acquisition unit that acquires a plurality of read yarn images, which are a plurality of images of a specific yarn included in a fabric including a plurality of yarns, read under different reading conditions; a candidate color determination unit that determines a candidate color that is a candidate for the color of the specific yarn for each of the plurality of read yarn images using each of the plurality of read yarn images; a color determination unit that determines the color of the specific yarn using a plurality of candidate colors of the plurality of read yarn images; A data processing device comprising:

Images