Computer program, data processing device, and method of manufacturing sheet

Abstract

To provide a computer program, a data processing device, and a method of manufacturing a sheet that improve detection of a defect of an object that is performed using a machine learning model.SOLUTION: A method comprises: acquiring an object image that shows an object and is generated using an image sensor; determining whether or not a specific condition which indicates that a pseudo defect different from a defect to be detected is included in the object shown in the object image is satisfied; detecting a defect of the object shown in the object image by executing detection processing including processing that is executed using a machine learning model on the object image; performing first detection processing as detection processing in a case where it is determined that the specific condition is satisfied; and performing second detection processing different from the first detection processing as detection processing in a case where it is determined that the specific condition is not satisfied.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present specification relates to a computer program, a data processing device, and a method for manufacturing a sheet. [Background technology]

[0002] Patent Literature 1 discloses a technology for inspecting labels attached to products using a captured image containing the labels. In this technology, a label area and the type of label in the captured image are identified using an object detection model. Then, the image of the labeled area is input to a machine learning model selected according to the type of label, thereby inspecting the label. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2023-168966 Summary of the Invention [Problem to be solved by the invention]

[0004] This specification discloses a new technique that can improve the detection of defects in objects performed using machine learning models. [Means for solving the problem]

[0005] The techniques disclosed in this specification can be implemented in the following application examples.

[0006] [Application Example 1] A computer program that causes a computer to realize the following: an acquisition function that acquires a target image showing an object, the target image being generated using an image sensor; a judgment function that determines whether a specific condition is satisfied that indicates that the object shown in the target image may include a pseudo-defect that is different from the defect to be detected; and a detection function that performs a detection process on the target image, including processing that is executed using a machine learning model, to detect defects in the object shown in the target image, the detection function executing a first detection process as the detection process if it is determined that the specific condition is satisfied, and executing a second detection process different from the first detection process as the detection process if it is determined that the specific condition is not satisfied.

[0007] According to the above configuration, the detection function for detecting defects in an object using a machine learning model executes a first detection process when it is determined that a specific condition indicating that a pseudo defect may exist in the object is satisfied, and executes a second detection process different from the first detection process when it is determined that the specific condition is not satisfied. As a result, the detection process is executed according to the specific condition indicating that a pseudo defect may be included in the object, and therefore the detection of defects in an object executed using a machine learning model can be improved (for example, detection efficiency and detection accuracy can be improved).

[0008] [Application Example 2] A method for manufacturing a sheet-like object, comprising: a creation step for creating the sheet-like object; an acquisition step for acquiring a target image showing the sheet-like object, the target image being generated using an image sensor; a judgment step for determining whether a specific condition indicating that the sheet-like object shown in the target image may include a pseudo-defect different from the defect to be detected is satisfied; a detection step for performing a detection process including processing performed using a machine learning model on the target image to detect defects in the sheet-like object shown in the target image, the detection process including a first detection process and a second detection process different from the first detection process, and the first detection process is performed if it is determined that the specific condition is satisfied, and the second detection process is performed if it is determined that the specific condition is not satisfied; and an addition step for adding a marker to the sheet-like object to identify the detected defect if the defect is detected in the detection step.

[0009] According to the above configuration, in the detection process for detecting defects in an object using a machine learning model, a first detection process is executed when it is determined that a specific condition indicating that a pseudo defect may exist in the object is satisfied, and a second detection process different from the first detection process is executed when it is determined that the specific condition is not satisfied. As a result, the detection process is executed according to the specific condition indicating that a pseudo defect may exist in the object, and therefore the detection of defects in the object executed using the machine learning model can be improved (e.g., detection efficiency and detection accuracy can be improved). Therefore, for example, markers for identifying defects can be added efficiently and accurately.

[0010] The technology disclosed in this specification can be realized in various forms, such as a method for detecting defects in an object, a data processing device for detecting defects in an object, a method for manufacturing a sheet-like object, a computer program for realizing these methods and devices, a recording medium on which the computer program is recorded, etc. [Brief explanation of the drawings]

[0011] [Figure 1] Flowchart of the fabric manufacturing method. [Figure 2] FIG. 1 is a diagram showing the configuration of an inspection system 1000. [Figure 3] FIG. 1 is a perspective view showing a schematic configuration of an inspection device 10. [Figure 4] 10 is a flowchart showing the procedure of an inspection process. [Figure 5] 10 is a first conceptual diagram showing the transition of the conveyance state of the cloth 700 in the inspection process. FIG. [Figure 6] FIG. 10 is a second conceptual diagram showing the transition of the conveyance state of the cloth 700 in the inspection process. [Figure 7] 10 is a flowchart of an interrupt process when a defect is detected. [Figure 8] 4 is a flowchart of an inspection process according to the first embodiment. [Figure 9] 10 is a flowchart of a normal detection process. [Figure 10] FIG. 10 is a diagram showing an example of an image used in normal detection processing. [Figure 11] FIG. 1 is a first explanatory diagram showing an outline of a defect detection process. [Figure 12] FIG. 2 is a second explanatory diagram showing an outline of the defect detection process. [Figure 13] 10 is a flowchart of a special detection process. [Figure 14] FIG. 10 is a diagram showing an example of an image used in the special detection process. [Figure 15] 10 is a flowchart of an inspection process according to a second embodiment. [Figure 16] FIG. 2 is a diagram showing an example of a luminance histogram HG of a target image PI. DETAILED DESCRIPTION OF THE INVENTION

[0012] A. First Example: A1. Fabric manufacturing method FIG. 1 is a flowchart of a method for manufacturing a cloth. In S1, a weaving process is carried out in which a cloth (a woven fabric in this embodiment) is woven from yarn. For example, the weaving process is carried out using a known loom. Note that the cloth is not limited to a woven fabric, but may also be a knitted fabric or a nonwoven fabric. If the cloth is a knitted fabric, instead of the weaving process, a process of knitting the cloth from yarn using a known knitting machine is carried out. If the cloth is a nonwoven fabric, instead of the weaving process, a process of forming a nonwoven fabric by collecting and bonding fibrous materials (e.g., natural fibers, synthetic fibers) using a known manufacturing device is carried out. The fabric produced in S1 is wound around a core material to form a roll (described below).

[0013] In S2, an inspection process is performed to inspect the manufactured fabric. The inspection process is a process of detecting defects that may be present in the fabric and adding marks to the fabric to identify the detected defects. The inspection process is performed using an inspection system 1000.

[0014] A2. Inspection system configuration Fig. 2 is a diagram showing the configuration of an inspection system 1000. The inspection system 1000 in Fig. 2 includes a data processing device 200 and an inspection device 10. The data processing device 200 and the inspection device 10 are communicatively connected. This allows the data processing device 200 to receive image data generated by digital cameras 111-114 (described later), control signals from a control unit 990 of a conveyance device 900, and the like.

[0015] The data processing device 200 is, for example, a personal computer, and performs various data processing operations for inspecting the appearance of the fabric 700 produced in S1.

[0016] The data processing device 200 includes a processor 210, a storage device 215, a display unit 240, an operation unit 250, a graphics processing unit (GPU) 260, and a communication interface 270. These elements are connected to each other via a bus. The storage device 215 includes a volatile storage device 220 and a non-volatile storage device 230.

[0017] The processor 210 is a device configured to perform data processing, such as a central processing unit (CPU) or a system on a chip (SoC). The volatile storage device 220 is, for example, a dynamic random access memory (DRAM), and the non-volatile storage device 230 is, for example, a flash memory.

[0018] The non-volatile storage device 230 stores a computer program PG. The computer program PG includes three machine learning models as program modules, specifically, an object detection model MD1, a shadow removal model MD2, and a feature extraction model MD3. These machine learning models will be described later. The processor 210 executes the computer program PG to perform the inspection process, which will be described later.

[0019] The display unit 240 is a device configured to display images, such as a liquid crystal display or an organic EL display. The operation unit 250 is a device configured to receive operations by a user, such as a button, a lever, or a touch panel overlaid on the display unit 240. The display unit 240 and the operation unit 250 may form a so-called touch screen. The user can input various requests and instructions to the data processing device 200 by operating the operation unit 250. The display unit 240 may display operation elements (e.g., buttons, sliders, etc.), and the displayed elements may be operated through operation of the operation unit 250.

[0020] The GPU 260 is a computing device configured to execute various numerical calculations such as image processing and machine learning. The GPU 260 executes various calculations in accordance with instructions from the processor 210. In this embodiment, the GPU 260 executes calculations for the object detection model MD1, the shadow removal model MD2, and the feature extraction model MD3 in accordance with instructions from the processor 210.

[0021] The communication interface 270 is an interface for communicating with the inspection device 10 (for example, it includes one or more of a USB interface, a wired LAN interface, an IEEE802.11 wireless interface, and an industrial camera interface (for example, CameraLink, CoaXPress, etc.)). In this embodiment, the communication interface 270 is connected to the inspection device 10.

[0022] The inspection device 10 is a device that performs inspection of the fabric 700, which is the object of inspection in this embodiment, in cooperation with the data processing device 200. The inspection device 10 includes digital cameras 111-114, a rotary encoder 120, a light source 130, and a conveying device 900.

[0023] The digital cameras 111-114 are devices that generate and acquire images of an object (subject) by photographing the object using an image sensor including an imaging element such as a CCD or CMOS. The digital cameras 111-114 are used to photograph the cloth 700 to be inspected. The rotary encoder 120 is used to calculate the relative position of the cloth 700 with respect to the conveying device 900 (details will be described later). The light source 130 irradiates the cloth 700 with light to clearly photograph the cloth to be inspected.

[0024] The conveying device 900 is a device that conveys the fabric 700, and includes a conveying mechanism 950, an operation unit 980 that receives operations by a user, and a control unit 990.

[0025] FIG. 3 is a perspective view showing a schematic configuration of the inspection device 10. The conveying mechanism 950 includes a plurality of rollers including rollers 951 and 952, a support plate 953, an upstream end detection sensor 955, and a conveying motor (not shown). The cloth 700 is conveyed by driving at least a portion of the plurality of rollers by the conveying motor. The support plate 953 is a flat plate disposed between the upstream roller 951 and the downstream roller 952, and supports a portion of the cloth 700 to be inspected that is to be photographed by the digital cameras 111-114. In FIG. 3, the rotary encoder 120 and the operation unit 980 and control unit 990 of the conveying device 900 are not shown.

[0026] The partial path Pth in the figure indicates the portion of the conveyance path of the cloth 700 between the rollers 951 and 952, i.e., the portion along the support plate 953. The direction Df in the figure indicates the conveyance direction on the partial path Pth (direction Df is also referred to as the conveyance direction Df). The orthogonal direction Dt indicates a direction parallel to the flat portion 700F and perpendicular to the partial path Pth. Hereinafter, the upstream side of the conveyance direction Df of the conveyance path of the cloth 700 will be simply referred to as the upstream side, and the downstream side of the conveyance direction Df of the conveyance path of the cloth 700 will be simply referred to as the downstream side.

[0027] The cloth 700 to be inspected is a long piece of cloth that is longer than the partial path Pth. The cloth 700 is wound around a core material 31 to form an upstream roll 710. The upstream roll 710 is disposed in a mounting section (not shown) upstream of the partial path Pth. The upstream mounting section includes, for example, a holding member including two or more rollers, and is configured to rotatably hold the upstream roll 710. The cloth 700 pulled out from the upstream roll 710 is transported from the position of the upstream roller 951 to the position of the downstream roller 952 along the partial path Pth.

[0028] The fabric 700 conveyed to the downstream roller 952 is wound around the core material 32 downstream of the partial path Pth. The fabric 700 wound around the core material 32 forms the downstream roll 720. The downstream roll 720 is disposed in a mounting section (not shown) downstream of the partial path Pth. The downstream mounting section includes, for example, a holding member including two or more rollers, and is configured to rotatably hold the downstream roll 720.

[0029] Between the rollers 951 and 952, the cloth 700 is supported by the support plate 953 to form a flat portion, that is, a flat portion 700F. The light source 130 is disposed at a position where it irradiates the flat portion 700F with light.

[0030] 3, the imaging range Ar, which is the area captured by the digital cameras 111-114, is hatched. The imaging range Ar is a rectangular area having two sides Ar1 and Ar2 parallel to the partial path Pth and two sides Ar3 and Ar4 perpendicular to the partial path Pth. The first side Ar1 and the second side Ar2 of the imaging range Ar are located outside the cloth 700.

[0031] In the figure, partial areas R11-R14 indicate areas photographed by digital cameras 111-114, respectively. In this embodiment, the digital cameras 111-114 are arranged side by side in the orthogonal direction Dt, and therefore the partial areas R11-R14 are also arranged side by side in the orthogonal direction Dt. The photographing range Ar is the entire partial areas R11-R14.

[0032] The upstream end detection sensor 955 detects the upstream end of the long cloth 700 at a predetermined position upstream of the partial path Pth, i.e., upstream of the imaging range Ar; in this embodiment, near the upstream roll 710. More specifically, the upstream end detection sensor 955 detects that the upstream end of the cloth 700 has separated from the core material 31 when the entire cloth 700 is pulled out from the upstream roll 710 and the upstream end of the cloth 700 has separated from the core material 31. The upstream end detection sensor 955 is, for example, a known contact sensor that has a contact portion that comes into contact with the conveyed cloth 700 near the upstream roll 710. The upstream end detection sensor 955 optically or physically detects a transition from a state in which the contact portion and the cloth 700 are in contact to a state in which the contact portion and the cloth 700 are not in contact. The upstream end detection sensor 955 may have another configuration, such as a photoelectric sensor that detects the presence or absence of cloth at a predetermined position without contact. The upstream end detection sensor 955 transmits a detection signal indicating the detection result to the control unit 990.

[0033] The operation unit 980 (FIG. 2) of the transport device 900 includes a plurality of switches (not shown) and is configured to receive operations from an operator. In this embodiment, the plurality of switches are foot switches operated by the operator with their feet, and include at least a transport start switch and a transport stop switch. Alternatively, the plurality of switches may be push switches operated by the operator with their hands, or buttons displayed on a touch panel.

[0034] The control unit 990 (FIG. 2) of the conveying device 900 is connected to, for example, the operation unit 980, the upstream end detection sensor 955, and a conveying motor and power supply (not shown). The control unit 990 is an electric circuit configured to control the conveying motor in response to the operation of the operation unit 980 and the detection signal of the upstream end detection sensor 955. The control unit 990 may be configured using a computer or dedicated hardware (such as an Application Specific Integrated Circuit (ASIC)). For example, when a conveyance start switch is pressed while the cloth 700 is not being conveyed, the control unit 990 starts conveying the cloth 700. When a stop switch is pressed while the cloth 700 is being conveyed, the conveying device 900 stops conveying the cloth 700. When the conveying device 900 receives a detection signal from the upstream end detection sensor 955 indicating that the upstream end of the cloth 700 has been detected during conveyance of the cloth 700, the conveying device 900 automatically stops conveying the cloth 700.

[0035] Furthermore, the control unit 990 is connected to the data processing device 200 via a known communication interface (for example, a USB interface). The data processing device 200 transmits information such as the conveyance state of the cloth 700 and the detection result by the upstream end detection sensor 955 to the data processing device 200 via the communication interface.

[0036] A rotary encoder 120 is connected to the conveying device 900 to detect the direction and amount of position change due to conveyance. For example, the rotary encoder 120 is connected to a roller (e.g., the upstream roller 951). The rotary encoder 120 may have various configurations for detecting the direction and amount of position change due to conveyance. For example, the rotary encoder 120 may be an incremental encoder. An incremental encoder alternately outputs A pulses and B pulses in response to position changes. The number of output pulses indicates the amount of movement. The phase difference (positive or negative) between the A pulse and the B pulse indicates the direction of movement. The data processing device 200 (FIG. 1) can calculate the current conveyance position of the cloth 700 conveyed by the conveying device 900 (i.e., the relative position of the cloth 700 with respect to the conveying device 900) by counting the number of pulses output from the rotary encoder 120 according to the phase difference (i.e., the direction). Note that a counter that counts the number of pulses according to the direction may be connected to the rotary encoder 120. The data processing device 200 may use the information from the counter to obtain the current relative position of the fabric 700 .

[0037] A3. Inspection process The inspection process (S2 in FIG. 1) will be further described. FIG. 4 is a flowchart showing the procedure of the inspection process. FIGS. 5 and 6 are conceptual diagrams showing the transition of the transport state of the cloth 700 in the inspection process. The transport state of the cloth 700 transitions in chronological order as shown in FIGS. 5(A), (B), (C), and 6(A), (B), (C).

[0038] In S10 of FIG. 4, an operator sets the cloth 700 to be inspected in the inspection device 10. FIG. 5(A) is a conceptual diagram of a state in which a first cloth 700a, which is the first to be inspected in the inspection process, is set in the inspection device 10. The first cloth 700a is wound around a core 31a to form a roll. The operator attaches the roll of first cloth 700a to an attachment section (not shown) on the upstream side of the conveyance mechanism 950 as an upstream roll 710a. The operator pulls out a downstream end 704a of the first cloth 700a from the upstream roll 710a. Specifically, the downstream end 704a passes through a partial path Pth (FIG. 3) that passes through the imaging range Ar on the support plate 953, is pulled out until it reaches a position downstream of the support plate 953, and is then wound around the core 32a. The worker attaches the core 32a around which the end 704a is wound to a mounting portion (not shown) on the downstream side of the conveying mechanism 950. As a result, the first cloth 700a is set so as to be inspectable by the inspection device 10, as shown in Fig. 5(A). Note that in the state shown in Fig. 5(A), the portion of the first cloth 700a located downstream of the imaging range Ar in the conveying direction Df cannot be inspected by the inspection process (described later) by the data processing device 200, and therefore is inspected visually by the worker, for example.

[0039] In S15, the data processing device 200 starts the inspection process. For example, when an operator operates the operation unit 250 of the data processing device 200 to start the computer program PG and input an instruction to start the inspection process, the data processing device 200 (processor 210) starts the inspection process. The inspection process is a process for inspecting the cloth 700 using captured images of the cloth 700 in synchronization with the conveyance of the cloth 700 by the conveyance device 900. Details of the inspection process will be described later.

[0040] In S20, the conveying device 900 starts conveying the cloth 700 based on the operation of the worker. Specifically, the worker presses a conveyance start switch on the operation unit 980 of the conveying device 900. In response to the pressing of the conveyance start switch, the control unit 990 of the conveying device 900 drives the conveyance motor to start conveying the cloth 700 in the conveyance direction Df.

[0041] When S15 and S20 are executed, the fabric 700 is inspected by an inspection process for each portion that passes through the imaging range Ar. For example, FIG. 5(B) illustrates a state in which the conveyance of the first fabric 700a has progressed from the state in FIG. 5(A). The inspected portion of the first fabric 700a is sent downstream in the conveyance direction Df and wound around the core 32a, so in FIG. 5(B), a downstream roll 720a is formed downstream of the imaging range Ar. Furthermore, because the first fabric 700a is pulled out from the upstream roll 710a during conveyance, the diameter of the upstream roll 710a in the state in FIG. 5(B) is smaller than in the state in FIG. 5(A).

[0042] As the conveyance of the inspected cloth 700 progresses, the cloth 700 is completely pulled out from the upstream roll 710 and the upstream end of the cloth 700 separates from the core material 31. In S30, the upstream end detection sensor 955 detects the upstream end of the cloth 700. When the upstream end of the cloth 700 is detected by the upstream end detection sensor 955, in S35 the conveyance device 900 automatically stops conveying the cloth 700. Specifically, the upstream end detection sensor 955 transmits a detection signal indicating that it has detected the upstream end of the cloth 700 to the control unit 990 of the conveyance device 900. Upon receiving the detection signal, the control unit 990 stops driving the conveyance motor and stops conveying the cloth 700.

[0043] 5(C) shows a state in which the upstream end 701a of the first cloth 700a has separated from the core material 31a. In this state, the conveyance of the first cloth 700a is stopped. In this state, the portion of the first cloth 700a near the upstream end 701a (hereinafter referred to as the upstream end 703a) has not passed through the photographing range Ar.

[0044] When the conveyance of the cloth 700 is automatically stopped, in S40, an operator loads a new roll of cloth 700 into the loading section on the upstream side of the conveyance mechanism 950. The new cloth 700 means the cloth to be inspected next after the cloth 700 currently being inspected.

[0045] In Fig. 6(A), the roll of the second cloth 700b to be inspected next to the first cloth 700a is shown as the upstream roll 710b. The upstream roll 710b in Fig. 6(A) is the roll that is attached in S40.

[0046] In S45, an operator connects the upstream end of the cloth 700 currently being inspected to the downstream end of a new cloth 700. For example, as shown in FIG. 6(A), an upstream end 701a of a first cloth 700a and a downstream end 702b of a second cloth 700b are connected using a jig 20. The jig 20 is, for example, a string-like cable tie made of resin. In the example of FIG. 6(A), the upstream end 701a of the first cloth 700a and the downstream end 702b of the second cloth 700b are connected using the jig 20 at three locations: both ends and the center of the cloths 700a and 700b in the width direction (the horizontal direction in FIG. 6(A)).

[0047] In S50, the conveying device 900 resumes conveying the cloth 700 based on the operator's operation. Specifically, the operator presses the conveying start switch on the operation unit 980 of the conveying device 900. In response to the pressing of the conveying start switch, the control unit 990 of the conveying device 900 drives the conveying motor to resume conveying the cloth 700 in the conveying direction Df.

[0048] When the conveyance of the cloth 700 is resumed, the joint portion of the two cloths 700 moves in the conveyance direction Df, so that the joint portion of the two cloths passes through the photographing range Ar and reaches the downstream roll 720 (FIG. 3). For example, in the example of FIG. 6, the joint portion between the upstream end 701a of the first cloth 700a and the downstream end 702b of the second cloth 700b is located upstream of the photographing range Ar in FIG. 6(A). When the conveyance of the cloth is resumed in S50, the joint portion reaches the downstream roll 720a, as shown in FIG. 6(B).

[0049] In S55, when the joint portion of the two cloths 700 reaches the downstream roll 720, the conveying device 900 stops conveying the cloth 700 based on the operator's operation. In the example of FIG. 6, the operator visually monitors the conveyance of the cloths 700a and 700b in FIG. 6, and presses the stop switch of the operation unit 980 of the conveying device 900 when the state shown in FIG. 6(B) is reached, i.e., when the joint portion between the upstream end 701a of the first cloth 700a and the downstream end 702b of the second cloth 700b reaches the downstream roll 720a. In response to the pressing of the stop switch, the control unit 990 of the conveying device 900 stops the conveying motor and stops the conveyance of the cloths 700a and 700b.

[0050] When the conveyance of the cloth 700 is stopped, in S60, the worker replaces the downstream roll 720. In the example of FIG. 6, the worker removes the jig 20 in the state of FIG. 6(B), thereby disconnecting the upstream end 701a of the first cloth 700a from the downstream end 702b of the second cloth 700b. The worker removes the downstream roll 720a around which the first cloth 700a is wound from the downstream mounting portion and moves it to a predetermined storage location. The worker prepares a core 32b without any cloth wound around it and wraps the downstream end 704b of the second cloth 700b around the core 32b. The worker mounts the core 32b around which the downstream end 704b of the second cloth 700b is wound to a mounting portion (not shown) on the downstream side of the conveyance mechanism 950. As a result, as shown in FIG. 6(C), the first cloth 700a is removed from the inspection device 10, and only the second cloth 700b is set in the inspection device 10 so as to be inspectable.

[0051] When the cloth 700 replacement work is completed, in S65, the conveying device 900 resumes conveying the cloth 700 based on the operation of the worker. Specifically, as in S50, the worker presses the conveying start switch on the operation unit 980 of the conveying device 900. In response to the pressing of the conveying start switch, the control unit 990 of the conveying device 900 drives the conveying motor to resume conveying the cloth 700 in the conveying direction Df.

[0052] After S65, the inspection process returns to S30. That is, when S65 is executed, in the example of FIG. 6, the second cloth 700b is inspected by the inspection process for each portion that has passed through the imaging range Ar. As the conveyance of the second cloth 700b progresses, the second cloth 700b is completely pulled out from the upstream roll 710b, and the upstream end of the second cloth 700b separates from the core material 31b. At that point, the upstream end detection sensor 955 detects the upstream end of the second cloth 700b (S30). In this manner, S30 to S65 are repeatedly executed, and the multiple cloths 700 are inspected one by one. When inspection of all of the cloths 700 to be inspected has been completed, the inspection process ends.

[0053] Next, the interrupt process during defect detection will be described. The interrupt process during defect detection is an interrupt process that is executed when a defect is detected in the inspection process started in S15 of FIG. 5 during the inspection process. FIG. 7 is a flowchart of the interrupt process during defect detection. In S80, the conveying device 900 stops conveying the cloth 700 based on a defect detection signal from the data processing device 200. Specifically, when a defect is detected in the inspection process, the data processing device 200 transmits a defect detection signal to the conveying device 900. When a defect is detected in the inspection process, the conveying device 900 is conveying the cloth 700. For this reason, when the control unit 990 of the conveying device 900 receives the defect detection signal, it stops driving the conveying motor to stop conveying the cloth 700. Since the inspection process of the data processing device 200 is performed in synchronization with the conveyance of the cloth 700, the inspection process is also interrupted when the conveyance of the cloth 700 is stopped.

[0054] In S82, the worker marks the defect in the cloth 700 based on the defect information displayed by the data processing device 200. Specifically, as will be described later, when a defect is detected in the inspection process, the data processing device 200 displays defect information indicating the position of the defect in the cloth 700 on the display unit 240. Immediately after the defect is detected, the conveyance of the cloth 700 is stopped, and therefore the detected defect is present in a portion of the cloth 700 that is located on the support plate 953. After the worker understands the position of the defect by looking at the display unit 240, the worker confirms the defect in the actual cloth 700 and marks the cloth 700 so that the defect can be identified later. The mark may be, for example, a sticker attached to the cloth 700 or a mark drawn with chalk or the like.

[0055] In S84, the conveying device 900 resumes conveying the cloth 700 based on the operator's operation. Specifically, the operator presses a conveyance start switch on the operation unit 980 of the conveying device 900. In response to the pressing of the conveyance start switch, the control unit 990 of the conveying device 900 drives the conveyance motor to resume conveying the cloth 700 in the conveyance direction Df. Since the inspection process of the data processing device 200 is performed in synchronization with the conveyance of the cloth 700, when the conveyance of the cloth 700 is resumed, the inspection process is also resumed.

[0056] A4. Inspection processing Next, the inspection process by the data processing device 200 will be described. Fig. 8 is a flowchart of the inspection process of the first embodiment. When the inspection process is started, the processor 210 of the data processing device 200 executes a normal detection process in S210. The normal detection process is a process for detecting defects in the cloth 700 using a photographed image obtained by photographing the cloth 700 in synchronization with the conveyance of the cloth 700 by the conveying device 900. The normal detection process will be described in detail later.

[0057] In S220, the processor 210 determines whether the feeding of the cloth 700 has been stopped due to the upstream end of the cloth 700 being detected by the upstream end detection sensor 955. As described above in S30 and S35 of FIG. 4 , the feed device 900 stops feeding the cloth 700 when the upstream end of the cloth 700 is detected by the upstream end detection sensor 955. As described above, the control unit 990 of the feed device 900 is communicatively connected to the data processing device 200. The control unit 990 of the feed device 900 sequentially transmits information regarding the operating status of the feed device 900, such as information indicating that the upstream end of the cloth 700 has been detected by the upstream end detection sensor 955 and information indicating that the operation of the feed device 900 has been stopped, to the data processing device 200. Therefore, the processor 210 can determine whether the feeding of the cloth 700 has been stopped due to the upstream end of the cloth 700 being detected by the upstream end detection sensor 955, based on the information transmitted from the feed device 900. In a modified example, the data processing device 200 may receive a detection signal from the upstream end detection sensor 955 indicating that the upstream end of the cloth 700 has been detected, and when the detection signal is received, may determine that the upstream end of the cloth 700 has been detected by the upstream end detection sensor 955 and that the conveyance of the cloth 700 has been stopped.

[0058] If the detection of the upstream end of the cloth 700 does not stop the conveyance of the cloth 700 (S220: NO), the processor 210 returns to S210. That is, in this case, the state in which the normal detection process is executed is maintained.

[0059] If the feed of the cloth 700 is stopped due to detection of the upstream end of the cloth 700 (S220: YES), the processor 210 determines in S230 whether the feed of the cloth 700 has been resumed based on an instruction from the operator. As described above, the control unit 990 of the feed device 900 sequentially transmits information regarding the operating status of the feed device 900 to the data processing device 200. Therefore, even if the feed of the cloth 700 is resumed based on an instruction from the operator, information indicating this is transmitted from the feed device 900 to the data processing device 200. The processor 210 can determine whether the feed of the cloth 700 has been resumed based on the information transmitted from the feed device 900. In a modified example, the data processing device 200 may receive a signal indicating that the feed start switch has been pressed from the operation unit 980 of the feed device 900, and, upon receiving this signal, determine that the feed of the cloth 700 has been resumed based on an instruction from the operator.

[0060] If the conveyance of the cloth 700 is stopped due to the detection of the upstream end of the cloth 700 (S220: YES), the upstream end of the cloth 700 being inspected (e.g., the upstream end 701a of the first cloth 700a) is separated from the core 31 (e.g., the core 31a in FIG. 5C) as shown in FIG. 5C. If the conveyance of the cloth 700 is subsequently resumed based on an instruction from the operator (S230: YES), as shown in FIG. 6A, a new cloth to be inspected next (e.g., the second cloth 700b) is connected, and the conveyance of the cloth 700 is resumed (S40-50 in FIG. 4). Therefore, after the conveyance is resumed, the ends of the two cloths 700 (in the example of FIG. 6A, the upstream end 703a of the first cloth 700a and the downstream end 704b of the second cloth 700b) pass through the imaging range Ar as the inspection targets.

[0061] If the conveyance of the cloth 700 is not resumed based on the operator's instruction (S230: NO), the processor 210 waits until the conveyance of the cloth 700 is resumed. If the conveyance of the cloth 700 is resumed based on the operator's instruction (S230: YES), the processor 210 executes the special detection process in S240. The special detection process, like the normal detection process, is a process for detecting defects in the cloth 700 using captured images obtained by photographing the cloth 700 in synchronization with the conveyance of the cloth 700 by the conveying device 900. The special detection process is a process designed to detect defects more accurately than the normal detection process when wrinkles are present in the inspected cloth 700. As will be described in detail later, the edge of the cloth 700 is more likely to wrinkle than portions other than the edge of the cloth 700. Therefore, the detection process to be executed is switched from the normal detection process to the special detection process in S240. The special detection process will be described in detail later.

[0062] As can be seen from the above explanation, the fact that the conveyance of the cloth 700 is stopped upon detection of the upstream end of the cloth 700 (YES in S220) and then the conveyance of the cloth 700 is resumed based on an instruction from the operator (YES in S230) is a condition indicating that the captured image captured in the shooting range Ar thereafter, i.e., the target image PI to be processed in the detection process, may include the edge of the cloth 700. Furthermore, because wrinkles are more likely to occur at the edge of the cloth 700 than in parts other than the edge of the cloth 700, the condition indicating that the target image PI to be processed may include the edge of the cloth 700 can be said to be a condition indicating that the target image PI to be processed may include wrinkles.

[0063] In S250, the processor 210 determines whether the conveyance of the cloth 700 has been stopped based on an instruction from the operator. As described above, the control unit 990 of the conveyance device 900 sequentially transmits information regarding the operating status of the conveyance device 900 to the data processing device 200. Therefore, even when the conveyance of the cloth 700 has been stopped based on an instruction from the operator, information indicating this is transmitted from the conveyance device 900 to the data processing device 200. The processor 210 can determine whether the conveyance of the cloth 700 has been stopped based on the information transmitted from the conveyance device 900. In a modified example, the data processing device 200 may receive a signal indicating that the stop switch has been pressed from the operation unit 980 of the conveyance device 900, and, upon receiving the signal, may determine that the conveyance of the cloth 700 has been stopped based on an instruction from the operator.

[0064] If the conveyance of the cloth 700 is not stopped based on the instruction of the operator (S250: NO), the processor 210 returns to S240. That is, in this case, the state in which the special detection process is being executed is maintained.

[0065] If the conveyance of the cloth 700 is stopped based on the instruction of the worker (S250: YES), the processor 210 determines in S260 whether the conveyance of the cloth 700 is resumed based on the instruction of the worker. For example, similar to S230, the processor 210 can determine whether the conveyance of the cloth 700 is resumed based on information transmitted from the conveying device 900.

[0066] When the conveyance of the cloth 700 is stopped based on an instruction from the operator (S250: YES), as shown in FIG. 6(B), the ends of two cloths 700 (e.g., the upstream end 703a of the first cloth 700a and the downstream end 704b of the second cloth 700b) pass through the photographing range Ar and reach the vicinity of the downstream roll 720 (e.g., 720a in FIG. 6(B)) (S55 in FIG. 4). Then, when the conveyance of the cloth 700 is resumed based on an instruction from the operator (S260: YES), as shown in FIG. 6(C), the downstream roll 720 is replaced and the conveyance of the cloth 700 is resumed (S60-65 in FIG. 4). For this reason, after the conveyance is resumed, a portion other than the end of the cloth 700 (in the example of FIG. 6(C) , a portion other than the end of the second cloth 700b) passes through the photographing range Ar as an inspection target.

[0067] If the conveyance of the cloth 700 is not resumed based on the instruction of the worker (S260: NO), the processor 210 waits until the conveyance of the cloth 700 is resumed. If the conveyance of the cloth 700 is resumed based on the instruction of the worker (S260: YES), the processor 210 executes the normal detection process in S210. Because wrinkles are less likely to occur in portions other than the end of the cloth 700, the detection process to be executed is returned from the special detection process to the normal detection process in S210.

[0068] As can be seen from the above explanation, in the inspection process of this embodiment, the processor 210 switches between the special detection process and the normal detection process so as to perform the special detection process when inspecting the edge of the cloth 700 and to perform the normal detection process when inspecting a part other than the edge of the cloth 700.

[0069] A5. Normal detection process Next, the normal detection process of S210 in FIG. 8 will be described. FIG. 9 is a flowchart of the normal detection process. In S300, the processor 210 of the data processing device 200 determines whether the photographing timing has arrived. For example, the first photographing timing is the start of the normal detection process. The nth photographing timing (n is an integer equal to or greater than 2) is the timing at which the inspection target cloth is conveyed from the (n-1)th photographing timing by the length AH (FIG. 3) of the photographing range Ar in the conveying direction Df. Therefore, in FIG. 3, when a portion of the cloth 700 located within the photographing range Ar is photographed at the nth photographing timing, the portion of the cloth 700 photographed at the (n-1)th photographing timing is a portion located within the adjacent range Arb adjacent to the photographing range Ar on the downstream side in the conveying direction Df. This makes it possible to sequentially generate multiple target images PI (described later) showing portions of the conveyed cloth 700 at different positions in the conveying direction Df. In a modified example, in order to inspect the cloth 700 more reliably and without omission, the timing of the nth photograph may be set to the timing when the cloth to be inspected has been transported a length slightly shorter than length AH (Figure 3) from the (n-1)th photograph.

[0070] The processor 210 can recognize the amount of transport of the cloth 700 based on a count value, which is the number of pulses obtained from the rotary encoder 120. For this reason, the processor 210 determines that the timing for photographing has arrived based on the count value each time the cloth 700 is transported by the length AH of the photographing range Ar in the transport direction Df.

[0071] If the timing to capture an image has not arrived (S300: NO), the processor 210 ends the normal detection process. As shown in FIG. 8, the normal detection process is repeatedly executed until the feed of the cloth 700 is stopped by detecting the upstream end of the cloth 700 (S220 in FIG. 8). Therefore, if the feed of the cloth 700 has not been stopped by detecting the upstream end of the cloth 700 (NO in S220 in FIG. 8), the processor 210 starts the normal detection process again. Therefore, the processor 210 waits until the timing to capture an image arrives unless the feed of the cloth 700 is stopped. If the timing to capture an image arrives (S300: YES), the processor 210 acquires a captured image using the digital cameras 111-114 in S310. Specifically, the processor 210 supplies a capture instruction to each of the digital cameras 111-114. In response to the image capture instruction, the digital cameras 111-114 capture images of the fabric 700 and generate captured images IM1-IM4. The processor 210 acquires data of the captured images IM1-IM4 from each of the digital cameras 111-114.

[0072] FIG. 10 is a diagram showing an example of an image used in the normal detection process. FIG. 10(A) shows examples of captured images IM1-IM4 obtained from the digital cameras 111-114 (FIG. 2), respectively. Each of the captured images IM1-IM4 is a rectangular image having two sides parallel to the first direction Dx and two sides parallel to a second direction Dy perpendicular to the first direction Dx. The second direction Dy is a direction roughly parallel to the transport direction Df (FIG. 2). The data of each of the captured images IM1-IM4 is bitmap data representing the color values ​​of a plurality of pixels arranged in a matrix along the first direction Dx and the second direction Dy. The color values ​​are, for example, RGB values ​​including the respective gradation values ​​of red R, green G, and blue B (for example, values ​​greater than or equal to zero and less than or equal to 255).

[0073] As shown in FIG. 3, partial regions R11-R14 corresponding to captured images IM1-IM4 (FIG. 5(A)) are arranged side by side in the orthogonal direction Dt. The first captured image IM1 and the fourth captured image IM4 represent the cloth 700 and the background BG, respectively. The second captured image IM2 and the third captured image IM3 represent the cloth 700, respectively. These captured images IM1-IM4 as a whole represent the entire shooting range Ar. Although not shown, the background BG may represent various objects located outside the cloth 700, such as a part of the conveying device 900.

[0074] In the example of FIG. 10(A), the portion of the cloth 700 represented by the second captured image IM2 has a linear defect SCa. A linear defect can be formed due to various causes. For example, a defect in the thread forming the cloth 700 can cause the linear defect. Also, a linear defect (e.g., a scratch or a linear drawing) can be formed due to contact between the cloth 700 and another member (e.g., a device for carrying the cloth 700, a writing implement, etc.).

[0075] In S315, the processor 210 combines the four captured images IM1-IM4 to generate data for one target image PI. Figure 10(B) shows a target image PIa generated from the captured images IM1-IM4 of Figure 10(A) as an example of the target image PI. The target image PI is a strip-shaped image representing a portion of the cloth 700 within the capturing range Ar (Figure 3).

[0076] Various methods may be used to generate the target image PI. For example, the partial regions R11-R14 (FIG. 3) may be arranged side by side in the orthogonal direction Dt within the shooting range Ar without any gaps and without overlapping each other. In this case, the processor 210 may generate data for the target image PI by connecting the respective ends of the captured images IM1-IM4 (FIG. 10(A)) arranged side by side in the first direction Dx. Alternatively, the partial regions R11-R14 may be arranged so that two adjacent partial regions partially overlap each other. In this case, two adjacent images among the captured images IM1-IM4 include overlapping portions. In this case, the processor 210 combines the captured images IM1-IM4 by using the corresponding portion of one of the two images as the image of the overlapping portion of the two images.

[0077] In steps S320-S340, the processor 210 executes a defect detection process on the target image PI to detect defects contained in the target image PI. The defect detection process of this embodiment is executed based on an anomaly detection mechanism that uses a machine learning model called PaDiM (a Patch Distribution Modeling Framework for Anomaly Detection and Localization). PaDiM is disclosed in the paper "T. Defard, A. Setkov, A. Loesch, and R. Audigier, "Padim: a patch distribution modeling framework for anomaly detection and localization," arXiv:2011.08785(2020), https: / / arxiv.org / abs / 2011.08785, posted on 17 Nov 2020."

[0078] In S320, the processor 210 inputs the target image PI into the feature extraction model MD3 to generate N feature maps fm. Figures 11 and 12 are explanatory diagrams showing an overview of the defect detection process.

[0079] The feature extraction model MD3 shown in FIG. 12(A) executes calculation processing on the input image II using a plurality of calculation parameters, and generates a feature map fm (described later) corresponding to the input image II.

[0080] The feature extraction model MD3 is a known model used, for example, as an image recognition model that outputs image recognition results. The feature extraction model MD3 in this embodiment uses a known model called ResNet18. This model is disclosed, for example, in the paper "K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in ICML, 2016."

[0081] The feature extraction model MD3 has multiple layers LY1 to LY4. Each layer is a convolutional neural network (CNN) including multiple convolution layers. Each convolution layer performs convolution processing using a filter of a predetermined size to generate a feature map. The calculated value of each convolution processing is added with a bias and then input to a predetermined activation function for conversion. The feature map output from each convolution layer is input to the next processing layer (next convolution layer or layer). A known function such as the so-called ReLU (Rectified Linear Unit) is used as the activation function.

[0082] The weights and biases of the filters used in the convolution process are computation parameters that are adjusted through a learning process. The feature extraction model MD3 in this embodiment is a trained machine learning model that uses a publicly available set of computation parameters. For example, the trained machine learning model is a model that has been trained to classify a dataset classified into 1,000 classes in ImageNet.

[0083] The first layer LY1 generates n1 feature maps fm1 (FIG. 11(B)). The n1 feature maps fm1 are input to the second layer LY2. The number n1 of feature maps fm1 (also referred to as the number of channels) is, for example, 64. The second layer LY2 generates n2 feature maps fm2 (FIG. 11(B)). The n2 feature maps fm2 are input to the third layer LY3. The number n2 of channels of the feature map fm2 is, for example, 128. The third layer LY3 generates n3 feature maps fm3 (FIG. 11(B)). The n3 feature maps fm3 are input to the fourth layer LY4. The number n3 of channels of the feature map fm3 is, for example, 256. The fourth layer LY4 generates n4 feature maps fm4. Four feature maps fm4 are not used in this embodiment, and therefore are not generated in this embodiment. In this embodiment, a total of N (N is an integer equal to or greater than 3) feature maps fm1 to fm3 are generated (N=n1+n2+n3, N=448 in this embodiment).

[0084] Since the size of the input image II is a fixed size, the processor 210 divides the target image PI into multiple input images II and inputs them to the feature extraction model MD3, thereby generating N feature maps for each input image II.

[0085] In S330, the processor 210 generates a feature matrix FM for the target image PI using the generated N feature maps fm.

[0086] Specifically, the processor 210 adjusts the size (number of pixels in the vertical and horizontal directions) of the generated feature maps fm to make all feature maps fm the same size. In this embodiment, of the N feature maps fm, the feature map fm1 generated in the first layer LY1 is the largest ( FIG. 11(B) ). To achieve this, in this embodiment, the processor 210 performs a known enlargement process on the feature map fm2 generated in the second layer LY2 to generate a feature map fm2r of the same size as the feature map fm1 ( FIG. 11(C) ). Similarly, the processor 210 performs an enlargement process on the feature map fm3 generated in the third layer LY3 to generate a feature map fm3r of the same size as the feature map fm1 ( FIG. 11(C) ).

[0087] The processor 210 selects L usable maps Um to be used in generating the feature matrix FM from the N size-adjusted feature maps fm generated using one input image II (FIG. 11(D)). The number L of usable maps Um is an integer between 1 and N, for example, approximately 50 to 200. The L usable maps Um are selected, for example, randomly.

[0088] The processor 210 generates a feature matrix FM for one input image II using the selected L usage maps Um. The feature matrix FM is a matrix whose elements are feature vectors V(i, j), which correspond one-to-one to each pixel in the size-adjusted feature map fm. (i, j) indicates the coordinates of the corresponding pixel in the feature map fm. The feature vector is a vector whose elements are the pixel values ​​at coordinates (i, j) in the L usage maps Um. As shown in FIG. 11(E), one feature vector is an L-dimensional vector (a vector with L elements). Here, the feature matrix FM is generated for each input image II obtained by dividing the target image PI.

[0089] In S340, the processor 210 generates an anomaly map AM based on the feature matrix FM of the target image PI (ie, the feature matrix FM of each of the multiple input images II) and the Gaussian matrix GM (FIG. 2).

[0090] First, the Gaussian matrix GM will be described. The Gaussian matrix GM is a matrix that indicates the characteristics of a normal object (in this embodiment, the cloth 700). The Gaussian matrix GM is generated in advance by the process described below and stored in the nonvolatile storage device 230 of the data processing device 200 (FIG. 2).

[0091] As shown in FIG. 12(A), data of a predetermined number (assumed to be K) of normal images NI (NI1-NIK) is prepared. The normal images NI are generated, for example, by photographing normal cloth 700 that does not contain any abnormalities such as defects using a digital camera. The number K of normal images NI is, for example, an integer equal to or greater than 1, for example, approximately 10 to 100. The size of the normal images NI is the size of the input image II of the feature extraction model MD3.

[0092] Next, a feature matrix FMn (FMn1-FMnK) indicating the features of each of the K normal images NI (NI1-NIK) is generated (FIG. 12(B)). The method for generating the feature matrix FMn is the same as the method for generating the feature matrix FM of the target image PI described above (S320-S340 in FIG. 9, FIG. 11). That is, the normal image NI is input to the feature extraction model MD3 (FIG. 1), N feature maps fm of the normal image NI are generated, L usage maps Um are selected from the N feature maps fm, and the feature matrix FMn of the normal image NI is generated using the L usage maps Um. Since the number of normal images NI is K, K feature matrices FMn (FMn1-FMnK) are generated.

[0093] Then, a Gaussian matrix GM is generated using the K feature matrices FMn1-FMnK of the normal image N1. The Gaussian matrix GM is a matrix whose elements are Gaussian parameters that correspond one-to-one to each pixel of the size-adjusted feature map fm. The Gaussian parameters corresponding to the pixel at coordinates (i, j) include a mean vector μ(i, j) and a covariance matrix Σ(i, j). The mean vector μ(i, j) is the average of the feature vectors V(i, j) of the K feature matrices FMn1-FMnK of the normal image N1. The covariance matrix Σ(i, j) is the covariance matrix of the feature vectors V(i, j) of the K feature matrices FMn1-FMnK of the normal image N1. The mean vector μ(i, j) and the covariance matrix Σ(i, j) are statistical data calculated using the K feature vectors V(i, j). One Gaussian matrix GM is generated for the K normal images N11-N1K.

[0094] At S340 in FIG. 9, the processor 210 generates an anomaly map AMm using the feature matrix FM of the target image PI, i.e., each of the feature matrices FM of the multiple input images II obtained by dividing the target image PI, and the Gaussian matrix GM stored in the non-volatile memory device 230.

[0095] The anomaly map AMm is an image of the same size as the feature matrix FM and the Gaussian matrix GM. The value of each pixel in the anomaly map AMm is the Mahalanobis distance (FIG. 12(E)). The Mahalanobis distance D(i, j) at the coordinate (i, j) is calculated by performing a calculation process according to a known formula using the feature vector V(i, j) of the feature matrix FM (FIG. 12(D)) and the mean vector μ(i, j) and covariance matrix Σ(i, j) of the Gaussian matrix GM (FIG. 12(D)). The Mahalanobis distance D(i, j) is an evaluation value indicating the degree of difference between the target image and the K normal images NI1-NIK at the coordinate (i, j). Therefore, it can be said that the Mahalanobis distance D(i, j) is a value indicating the degree of anomaly of the target image at the coordinate (i, j).

[0096] Since the anomaly map AMm and the feature matrix FM are generated for each input image II obtained by dividing the target image PI, the processor 210 combines these anomaly maps AMm to generate one anomaly map AM corresponding to the target image PI.

[0097] FIG. 10C shows an example of an anomaly map AM, which is an anomaly map AMa corresponding to the target image PIa in FIG. 10A. The anomaly map AMa in FIG. 10A shows a defect SPa consisting of multiple abnormal pixels. That is, the defect SCa included in the target image PIa in FIG. 10A can be detected by the anomaly map AMa. An abnormal pixel is, for example, a pixel whose anomaly level is equal to or greater than a threshold value TH1. In this way, by referring to the anomaly map AM, it is possible to identify the position, size, and shape of a defect such as a scratch included in the target image PI. If the target image PI does not include a defect, the anomaly map AM will not identify the defect either.

[0098] In S350, the processor 210 determines whether the number of abnormal pixels in the abnormality map AM is equal to or greater than the threshold value TH2. If the number of abnormal pixels is equal to or greater than the threshold value TH2 (S350: YES), the processor 210 executes a defect notification process in S360. The defect notification process includes a process of displaying a notification screen (not shown) on the display unit 240 to notify the operator that a defect has been detected, and a process of transmitting a defect detection signal indicating that a defect has been detected to the control unit 990 of the conveyance device 900.

[0099] The notification screen includes, for example, an abnormality map AM (FIG. 10(C)), and is configured to enable the operator to identify the detected position, size, and shape. When the defect detection signal is transmitted to the control unit 990 of the conveying device 900, the interrupt process at the time of defect detection in FIG. 7 is executed as described above.

[0100] When the defect notification process of S360 is executed, the processor 210 ends the normal detection process. If the number of abnormal pixels is equal to or greater than the threshold value TH2 (S350: NO), the processor 210 ends the normal detection process without executing the defect notification process of S360. As shown in FIG. 8, the normal detection process is repeatedly executed until the upstream end of the cloth 700 is detected and the conveyance of the cloth 700 is stopped (S220 in FIG. 8). Therefore, if the conveyance of the cloth 700 is not stopped by the detection of the upstream end of the cloth 700 (NO in S220 in FIG. 8), the processor 210 starts the normal detection process again. That is, the processor 210 returns to S300 and waits until the next photographing timing arrives. As a result, multiple target images PI showing different portions of the cloth 700 are sequentially acquired (S300-S315), and the processes of S320-S360 described above are executed for each of the multiple target images PI as a processing target.

[0101] A6. Special detection process Next, the special detection process of S240 in Fig. 8 will be described. Fig. 13 is a flowchart of the special detection process. In S400, similar to S300 in Fig. 9, the processor 210 of the data processing device 200 determines whether or not the photographing timing has arrived. For example, the first photographing timing is the start of the special detection process. The mth photographing timing (m is an integer of 2 or more) is the timing when the cloth to be inspected has been transported by the length AH (Fig. 3) of the photographing range Ar in the transport direction Df from the (m-1)th photographing timing.

[0102] If the timing for capturing an image has not arrived (S400: NO), the processor 210 ends the special detection process. As shown in FIG. 8, the special detection process is repeatedly executed until the conveyance of the cloth 700 is stopped based on an instruction from the operator (S250 in FIG. 8). Therefore, if the conveyance of the cloth 700 has not been stopped based on an instruction from the operator (NO in S250 in FIG. 8), the processor 210 starts the special detection process again. Therefore, the processor 210 waits until the timing for capturing an image arrives unless the conveyance of the cloth 700 is stopped. If the timing for capturing an image arrives (S400: YES), the processor 210 acquires a captured image using the digital cameras 111-114 in S410, similar to S310 in FIG. 9.

[0103] In S415, similar to S315 in Fig. 9, the processor 210 combines the four captured images acquired from the digital cameras 111-114 to generate data for one target image PI. Fig. 14 is a diagram showing an example of an image used in the special detection process. Fig. 14(A) shows a target image PIb representing the connection portion between the first cloth 700a and the second cloth 700b as an example of the target image PI.

[0104] The target image PI captured in the normal detection process described above (for example, PIa in FIG. 10(B)) does not include the edge of the cloth 700. In contrast, the target image PI captured in the special detection process may include the edge of the cloth 700.

[0105] The end 703a of the first cloth 700a included in the target image PIb in FIG. 14(A) includes multiple wrinkles Wr. The upstream end 701a of the first cloth 700a and the downstream end 702b of the second cloth 700b are connected at only three points by the jig 20. As a result, uneven stress is applied to the end 703a of the first cloth 700a and the end 704b of the second cloth 700b, making wrinkles Wr more likely to occur. Because a slope is created along the wrinkles Wr, the angle at which light from the light source 130 is irradiated in this area is different from that of the flat area. As a result, one of the two areas along the wrinkles Wr becomes a dark area Sd that is darker than the flat area, and the other becomes a bright area Ba that is brighter than the flat area. The dark area Sd can also be considered a shadow created by the wrinkles Wr.

[0106] The wrinkles Wr are a phenomenon that occurs temporarily due to the application of biased stress during transport, and would not occur if biased stress were not applied. For this reason, the wrinkles Wr do not affect the quality of the cloth 700 and are different from defects that should be detected in the inspection process. On the other hand, the wrinkles Wr have a linear shape and are similar to defects such as linear scratches. For this reason, the wrinkles Wr can be said to be pseudo defects (hereinafter also referred to as pseudo defects) that are different from defects that should be detected but have characteristics similar to defects.

[0107] The target image PIb in FIG. 14(A) further includes a linear defect SCb located near one wrinkle Wr of the second cloth 700b.

[0108] In S420, the processor 210 inputs the target image PI into the object detection model MD1 to generate a wrinkle region image WI. The object detection model MD1 is a known machine learning model including a convolutional neural network (CNN). In this example, the object detection model MD1 is a model called "RTMDet," which is disclosed in the following paper: Chengqi Lyu, Wenwei Zhang, Haian Huang, Yue Zhou, Yudong Wang, Yanyi Liu, Shilong Zhang, and Kai Chen. "Rtmdet: An Empirical Study of Designing Real-Time Object Detectors," arXiv.2212.07784, December 16, 2022, https: / / doi.org / 10.48550 / arXiv.2212.07784.

[0109] RTMDet is a model that detects the bounding box and category (i.e., object type) of an object and performs pixel-level region segmentation called instance segmentation. In this embodiment, the object detection model MD1 is trained in advance to detect wrinkles Wr as the detection target and to detect the bounding box (rectangular region) surrounding the wrinkle and the pixel-level region where the wrinkle is located (hereinafter referred to as the wrinkle region). The object detection model MD1 may be trained using various methods, such as the training method described in the above-mentioned paper by RTMDet. Note that RTMDet is a model that can distinguish and detect multiple types of objects, but in this embodiment, it is trained to detect only wrinkles. The results of detecting the wrinkle region at the pixel level are output as a mask image indicating the wrinkle region and regions other than the wrinkle region.

[0110] In this embodiment, the designated size, which is the size of an image that can be input to the object detection model MD1, is smaller than the target image PI. To achieve this, the processor 210 divides the target image PI into multiple partial images of the designated size and inputs the multiple partial images to the object detection model MD1. The processor 210 combines mask images that indicate the detection results of wrinkle regions in each of the multiple partial images to generate a wrinkle region image WI corresponding to the target image PI.

[0111] Fig. 14(B) shows a wrinkle region image WI corresponding to the target image PIb in Fig. 14(A). The wrinkle region image WI in Fig. 14(B) includes three wrinkle regions WA, and it can be seen that the wrinkle region image WI identifies the wrinkle region WA in which the three wrinkles Wr in the target image PIb in Fig. 14(A) are located.

[0112] In S430, the processor 210 inputs the target image PI to the shadow removal model MD2 to generate a processed target image MI. The shadow removal model MD2 is a known machine learning model that includes a convolutional neural network (CNN). In this embodiment, the shadow removal model MD2 is a model called "DeshadowNet," which is disclosed in the following paper: Liangqiong Qu, Jiandong Tian, ​​Shengfeng He, Yandong Tang, and Rynson WH Lau, "DeshadowNet: A Multi-context Embedding Deep Network for Shadow Removal," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 21-26 July 2017, https: / / doi.org / 10.1109 / CVPR.2017.248

[0113] DeshadowNet is pre-trained to output an image in which the shadow has been removed from a captured image containing a shadow. The training method for the feature extraction model MD3 may be various, such as the training method for DeshadowNet described in the above-mentioned paper.

[0114] In this embodiment, the specified size, which is the size of an image that can be input to the shadow removal model MD2, is smaller than the target image PI. To achieve this, the processor 210 divides the target image PI into multiple partial images of the specified size, inputs the multiple partial images to the shadow removal model MD2, and generates a processed image in which the shadow has been removed for each of the multiple partial images. The processor 210 combines the multiple processed images corresponding to the multiple partial images to generate a processed target image MI corresponding to the target image PI.

[0115] Figure 14(C) shows a processed target image MI corresponding to the target image PIb in Figure 14(A). Like the target image PIb, the processed target image MI in Figure 14(C) includes three wrinkles Wr, but does not include the dark areas Sd that are the shadow areas along these wrinkles Wr. The processed target image MI in Figure 14(C) is almost identical to the target image PIb in Figure 14(A), except that the shadow areas have been removed.

[0116] In S440, the processor 210 executes defect detection processing on the processed target image MI to detect defects contained in the processed target image MI. This defect detection processing is the same processing as the above-described S320-S340 of the normal detection processing in FIG. 9, and is executed based on an anomaly detection mechanism using a machine learning model called PaDiM. That is, in S440, the processor 210 generates an anomaly map AM corresponding to the processed target image MI using a feature extraction model MD3 and a Gaussian matrix GM (FIG. 2). Since the processed target image MI is an image generated based on the target image PI, the anomaly map AM is also an image corresponding to the target image PI.

[0117] FIG. 14(D) shows, as an example of the abnormality map AM, an abnormality map AMa corresponding to the target image PIa in FIG. 14(A) and the processed target image MI in FIG. 14(C). The abnormality map AMb in FIG. 14(A) shows a defect SPb consisting of multiple abnormal pixels. As described above, an abnormal pixel is a pixel whose abnormality is equal to or greater than the threshold value TH1. For example, the linear defect SCb included in the target image PIb in FIG. 14(A) can be detected by the abnormality map AMa. Furthermore, the abnormality map AMb in FIG. 14(A) also shows a wrinkle Wr, which is a pseudo defect included in the target image PIb and should not be detected, as a defect SPb.

[0118] In S450, processor 210 deletes from the abnormality map AM, among the multiple abnormal pixels detected in the abnormality map AM, abnormal pixels located in the wrinkled region WA. For example, processor 210 identifies the wrinkled region WA by referring to the wrinkled region image WI in FIG. 14(B). Processor 210 replaces all pixels included in the region in the abnormality map AM corresponding to the wrinkled region WA with pixels different from the abnormal pixels. The abnormality map from which the abnormal pixels located in the wrinkled region WA have been deleted is designated as the processed abnormality map FI.

[0119] FIG. 14(E) shows a processed abnormality map FI corresponding to the abnormality map AMb in FIG. 14(D). The processed abnormality map FI in FIG. 14(E) shows a portion of the defect SPb consisting of multiple abnormal pixels that represents a linear defect SCb included in the target image PIb in FIG. 14(A). Furthermore, the processed abnormality map FI does not show a portion of the target image PIb that represents a wrinkle Wr that should not be detected. In this way, in S440 and S450, the feature extraction model MD3 is used to detect defects SCb located in an area different from the wrinkle area WA in the target image PI.

[0120] In S460, the processor 210 determines whether the number of abnormal pixels in the processed abnormality map FI is equal to or greater than the threshold value TH2. If the number of abnormal pixels is equal to or greater than the threshold value TH2 (S460: YES), the processor 210 executes a defect notification process in S470. The defect notification process is the same process as S360 in Fig. 9, and includes a process of displaying a notification screen (not shown) on the display unit 240 to notify the operator that a defect has been detected, and a process of transmitting a defect detection signal to the control unit 990 of the conveyance device 900.

[0121] When the defect notification process of S470 is executed, the processor 210 ends the special detection process. If the number of abnormal pixels is equal to or greater than the threshold value TH2 (S460: NO), the processor 210 ends the special detection process without executing the defect notification process of S470. As shown in FIG. 8, the special detection process is repeatedly executed until the conveyance of the cloth 700 is stopped based on an instruction from the operator (S250 in FIG. 8). Therefore, if the conveyance of the cloth 700 is not stopped based on an instruction from the operator (NO in S250 in FIG. 8), the processor 210 restarts the special detection process. That is, the processor 210 returns to S400 and waits until the next photographing timing arrives. As a result, multiple target images PI showing different portions of the cloth 700 are sequentially acquired (S400-S415), and the processes of S420-S470 described above are executed for each of the multiple target images PI as a processing target.

[0122] According to the present embodiment described above, the processor 210 of the data processing device 200 acquires the target image PI generated by the digital camera 111-114 that generates an image using an image sensor (S310 and S315 in FIG. 9 , and S410 and S415 in FIG. 13 ). The processor 210 determines whether a specific condition indicating that the cloth 700 shown in the captured images IM1-IM4, and thus the target image PI generated using the captured images IM1-IM4, may contain wrinkles Wr is satisfied (S220, S230, S250, and S260 in FIG. 8 ). If the specific condition is satisfied (YES in S220, YES in S230), the processor 210 executes the special detection process (S240 in FIG. 8 ). If the specific condition is not satisfied (YES in S250, YES in S260), the processor 210 executes the normal detection process (S210 in FIG. 8 ), which is different from the special detection process. As a result, a detection process is executed according to specific conditions that indicate that the cloth 700 shown in the target image PI may contain wrinkles Wr, which are pseudo defects, and this can improve the detection of defects in the cloth 700 that is executed using a machine learning model. For example, this can improve the efficiency and accuracy of detecting defects in the cloth 700.

[0123] A more detailed explanation will be given below. Although wrinkles Wr, which are pseudo defects, should not be detected, they are likely to be mistaken for linear scratches and erroneously detected as defects in normal detection processing. For this reason, when inspecting areas where wrinkles Wr are likely to occur (for example, the edges of the cloth 700), it is possible to perform visual inspection by an operator instead of detection processing using a machine learning model. However, visual inspection depends on the ability of the operator, and therefore there is a large variation in inspection accuracy and inspection speed, making it easy for defects to be overlooked and for the inspection speed to decrease.

[0124] For this reason, when inspecting a portion where wrinkles Wr are likely to occur, it is preferable to perform special detection processing that takes into account the possibility that wrinkles Wr may be included in the target image PI. When inspecting a portion where wrinkles Wr are unlikely to occur, it is preferable to perform normal detection processing that assumes that wrinkles Wr are not present, from the viewpoints of inspection accuracy, inspection speed, and inspection processing load.

[0125] In this embodiment, as described above, a preferred detection process is executed between the normal detection process and the special detection process depending on specific conditions that indicate that the cloth 700 shown in the target image PI may contain wrinkles Wr, which are pseudo-defects, thereby improving the detection of defects in the cloth 700 executed using a machine learning model.

[0126] Furthermore, according to this embodiment, the processor 210 acquires multiple target images PI showing portions of the cloth 700 at different positions in the conveyance direction Df (S300-S315 in FIG. 9, S400-S415 in FIG. 3). The processor 210 executes a detection process for each of the multiple target images PI as a processing target (S320-S360 in FIG. 9, S420-S470 in FIG. 13). As described above, the specific condition is a condition indicating that the target image PI to be processed may include an end of the cloth 700 in the conveyance direction Df (e.g., the end 703a, 704b in FIG. 6A). As described above, wrinkles Wr are likely to occur at the end of the cloth 700 in the conveyance direction Df. In this embodiment, different detection processes can be performed depending on whether the target image PI to be processed includes an end of the cloth 700 or not. Therefore, different detection processes can be performed depending on whether wrinkles Wr are likely to occur or not.

[0127] Furthermore, according to this embodiment, the processor 210 executes the special detection process (S240 in FIG. 8) from the time when the upstream end detection sensor 955 detects the upstream end 701a of the first cloth 700a at a predetermined position (specifically, near the upstream roll 710) until the termination condition is satisfied (in this embodiment, until it is determined in S260 in FIG. 8 that conveyance has resumed). In other words, in this embodiment, it is determined that the specific condition is satisfied from the time when the upstream end detection sensor 955 detects the upstream end 701a of the first cloth 700a at a predetermined position until the termination condition is satisfied. In this way, in this embodiment, the upstream end detection sensor 955 can be used to determine the specific condition for switching between the special detection process and the normal detection process. Therefore, for example, the special detection process can be executed when the target image PI includes an end portion in the conveyance direction Df of the cloth 700 that is prone to wrinkles Wr.

[0128] Furthermore, according to this embodiment, the above-mentioned termination condition indicates that the downstream end 704b of the second cloth 700b, which is to be inspected after the first cloth 700a, has passed through the imaging range Ar. Specifically, the termination condition is that after the upstream end 701a of the first cloth 700a is detected and the conveyance of the cloth 700 is stopped (YES in S220 of FIG. 8), the conveyance of the cloth 700 is resumed based on an instruction from the operator (YES in S230), and then the conveyance of the cloth 700 is stopped based on an instruction from the operator (YES in S250). With this configuration, the special detection process is performed on the upstream end 703a of the first cloth 700a and the downstream end 704b of the second cloth 700b, and the normal detection process is performed on portions other than these ends 703a and 704b. As a result, appropriate detection processes can be performed on the end of the cloth 700 in the conveyance direction Df and on portions other than the end.

[0129] Furthermore, according to this embodiment, after the downstream end 704b of the second cloth 700b passes through the photographing range Ar, the conveyance of the second cloth 700b is stopped based on an instruction from the operator (S55 in FIG. 4, FIG. 6(B)). While the conveyance is stopped, the operator performs the work of replacing the cloth 700 (S60 in FIG. 4). Then, the conveyance of the second cloth 700b is resumed based on an instruction from the operator (S65 in FIG. 4). When the processor 210 determines that the conveyance has been stopped in S55 in FIG. 4 (YES in S250) and that the conveyance has been resumed in S65 in FIG. 4 (YES in S260), the processor 210 executes the normal detection process instead of the special detection process (S210 in FIG. 8). That is, the condition for terminating the special detection process can be said to be that an instruction to stop and resume the conveyance of the second cloth 700b is input by the operator. According to this configuration, by switching between special detection processing and normal detection processing in response to transport instructions from the operator, appropriate detection processing can be performed for the end of the cloth 700 in the transport direction Df and for parts other than the end.

[0130] Furthermore, according to this embodiment, in the special detection process, processor 210 identifies wrinkled region WA in target image PI using object detection model MD1 (S420 in FIG. 13). Processor 210 detects defects SCb located in a region other than wrinkled region WA in target image PI using feature extraction model MD3, which is a machine learning model different from that used for object detection model MD1 (S440-S450 in FIG. 13). As a result, when wrinkles Wr may occur, defects SCb located in a region other than wrinkled region WA are detected using multiple machine learning models, thereby preventing erroneous detection of wrinkles Wr.

[0131] Furthermore, according to this embodiment, the special detection process includes a shadow removal process that removes dark areas Sd (shadows) included in the target image PI (S430 in FIG. 13). The defect detection process of S440 is performed on the processed target image MI (FIG. 14(C)) after the shadow removal process has been performed. As described above, dark areas Sd (shadows) are likely to occur near wrinkles Wr (FIG. 14(A)). Because the brightness of the dark areas Sd is low, even if a defect such as a scratch exists in the dark areas Sd, the characteristics of the defect are unlikely to appear in the image, and the defect may be difficult to detect in the defect detection process. According to this embodiment, the defect detection process is performed on the processed target image MI in which the brightness of the dark areas Sd has been corrected to be the same as that of the surrounding area by the shadow removal process. As a result, when a defect such as a scratch exists in the dark areas Sd, the accuracy of detecting the defect can be improved.

[0132] Furthermore, in this embodiment, the special detection process includes a defect detection process using the feature extraction model MD3 (S440 in FIG. 13) and processes performed to address wrinkles Wr (S420, S430, and S450 in FIG. 13). In contrast, the normal detection process includes a defect detection process using the feature extraction model MD3 common to the special detection process (S320-S340 in FIG. 9), but does not include processes performed to address wrinkles Wr. As a result, when a specific condition indicating that wrinkles Wr may be present is met, the processes performed to address wrinkles Wr are executed, thereby preventing a decrease in the accuracy of defect detection due to wrinkles Wr. Furthermore, when the specific condition is not met, the normal detection process is executed, which does not include processes performed to address wrinkles Wr, thereby preventing a decrease in the efficiency of the defect detection process. For example, because the processing time of the normal detection process is shorter than the processing time of the special detection process, the conveyance speed of the cloth 700 can be increased when the normal detection process is executed compared to when the special detection process is executed. In this way, for example, according to this embodiment, the cloth 700 can be inspected more efficiently than when the special detection process is always executed.

[0133] Furthermore, the manufacturing method of the cloth 700 in this embodiment includes a weaving process (S1 in FIG. 1) for creating the cloth 700, an acquisition process for acquiring a target image PI showing the cloth 700 (S310, S315 in FIG. 9, S410, S415 in FIG. 13), and a judgment process for determining whether specific conditions indicating that the cloth 700 shown in the target image PI may contain wrinkles Wr are satisfied (S220, S230, S250, S260 in FIG. 8). Furthermore, the manufacturing method includes a detection step in which, if it is determined that the specific condition is satisfied (YES in S220, YES in S230), a special detection process is executed (S240 in FIG. 8 ), and, if it is determined that the specific condition is not satisfied (YES in S250, YES in S260), a normal detection process (S210 in FIG. 8 ) different from the special detection process is executed; and an addition step (S82 in FIG. 7 ) in which, if defects SCa and SCb are detected in the detection step, marks are added to the fabric 700 to identify the detected defects SCa and SCb. According to this manufacturing method, the detection step can improve the detection of defects in the fabric 700, which is performed using a machine learning model, as described above. Therefore, for example, marks for identifying defects can be added efficiently and accurately.

[0134] As can be seen from the above explanation, the processes of S420-S470 of the special detection process (FIG. 13) are an example of the first detection process, and the processes of S320-S340 of the normal detection process (FIG. 9) are an example of the second detection process. The wrinkles Wr are an example of a pseudo defect, the cloth 700 is an example of a sheet-like object, and the task of replacing the cloth 700 is an example of a predetermined task. Furthermore, S420 in FIG. 13 is an example of the pseudo defect detection process and the second process, and S440 in FIG. 13 is an example of the defect detection process and the first process. Each process (S320 in FIG. 9, S420, S430, S440 in FIG. 13) executed using the object detection model MD1, the shadow removal model MD2, and the feature extraction model MD3 is an example of the process executed using a machine learning model.

[0135] B. Second Example In the first embodiment, the detection process is switched when the cloth feed is stopped or restarted based on an instruction from the operator. In the second embodiment, the detection process is switched according to the analysis result of the target image PI.

[0136] Fig. 15 is a flowchart of the inspection process of the second embodiment. In the second embodiment, the inspection process of Fig. 15 is executed instead of the inspection process of Fig. 8. Other configurations of the second embodiment are the same as those of the first embodiment.

[0137] In S500, similarly to S300 in Fig. 9, the processor 210 of the data processing device 200 determines whether or not the photographing timing has arrived. For example, the first photographing timing is the start of the special detection process. The mth (m is an integer of 2 or more) photographing timing is the timing when the inspection target cloth has been conveyed by the length AH (Fig. 3) of the photographing range Ar in the conveying direction Df from the (m-1)th photographing timing.

[0138] If the timing to take a photograph has not arrived (S500: NO), processor 210 waits until the timing to take a photograph arrives. If the timing to take a photograph arrives (S500: YES), processor 210 acquires a photographed image using digital cameras 111-114 in S510, similar to S310 in FIG.

[0139] In S515, similarly to S315 in FIG. 9, the processor 210 combines the four captured images acquired by the digital cameras 111-114 to generate data for one target image PI.

[0140] In S520, the processor 210 generates a luminance histogram HG of the target image PI. The histogram HG is obtained by classifying the pixels constituting the target image PI into a plurality of classes according to their luminance. In this embodiment, 256 gradation luminance values ​​are calculated, and the data of the histogram HG is generated with each of the 256 gradation values ​​representing one class. The histogram HG shows the distribution of luminance of the target image PI.

[0141] Fig. 16 is a diagram showing an example of a histogram HG of the brightness of a target image PI. The graph of the histogram HG in Fig. 16 is a graph in which the horizontal axis represents 256 brightness values ​​and the vertical axis represents the number of pixels having each brightness value. The histogram HGa in Fig. 16(A) is the histogram HG of the target image PI showing the cloth 700 without wrinkles Wr, specifically, the target image PIa in Fig. 10(B). The histogram HGb in Fig. 16(B) is the histogram HG of the target image PI showing the cloth 700 with wrinkles Wr, specifically, the target image PIb in Fig. 14(A).

[0142] The cloth 700 shown in the target image PIa has uniform brightness because it does not contain wrinkles Wr. The cloth 700 shown in the target image PIb contains wrinkles Wr, and therefore includes bright areas Ba and dark areas Sd that occur along the wrinkles Wr. For this reason, the histogram HGa of the target image PIa, which does not contain wrinkles Wr, contains, for example, one peak P1 corresponding to the uniform brightness of the cloth 700. In contrast, the histogram HGb of the target image PIb, which does contain wrinkles Wr, contains, for example, three peaks P1-P3. Peak P1 corresponds to the brightness of the portion of the cloth 700 that does not have wrinkles Wr. Peak P2 corresponds to the brightness of the dark areas Sd of the cloth 700 that are along the wrinkles Wr, and peak P3 corresponds to the brightness of the bright areas Ba of the cloth 700 that are along the wrinkles Wr. Thus, the brightness variation of the target image PIb, which contains wrinkles Wr, is greater than the brightness variation of the target image PIa, which does not contain wrinkles Wr.

[0143] In S530, processor 210 calculates the variance σ of the brightness of target image PI. The variance σ increases as the variation in brightness of target image PI increases. Therefore, the variance σb of the brightness of target image PIb that includes wrinkles Wr is greater than the variance σa of the brightness of target image PIa that does not include wrinkles Wr.

[0144] In S540, the processor 210 determines whether the variance σ of the target image PI is equal to or greater than a threshold TH3. The threshold TH3 is experimentally determined in advance so that the presence or absence of wrinkles Wr can be determined based on the variance σb.

[0145] If the variance σ of the target image PI is less than the threshold value TH3 (S540: NO), it can be determined that the target image PI does not contain wrinkles Wr. Therefore, in this case, processor 210 executes the normal detection process in S550. Specifically, processor 210 executes S320-S360 in FIG. 9 described above. The processes of S310 and S315 in FIG. 9 correspond to S510 and S515 in FIG. 15 in the second embodiment, and have already been executed, so they are not executed in S550.

[0146] If the variance σ of the target image PI is equal to or greater than the threshold value TH3 (S540: YES), it can be determined that the target image PI includes wrinkles Wr. Therefore, in this case, processor 210 executes special detection processing in S560. Specifically, processor 210 executes S420-S470 in FIG. 13 described above. The processing of S410 and S415 in FIG. 13 corresponds to S510 and S515 in FIG. 15 in the second embodiment, and has already been executed, so it is not executed in S560.

[0147] After the normal detection process of S550 or the special detection process of S560 is executed, processor 210 returns to S500 and waits until the next photographing timing arrives. As a result, while multiple target images PI showing different parts of cloth 700 are sequentially acquired (S500-S515), either the normal detection process of S550 or the special detection process of S560 is executed for each of the multiple target images PI as a processing target.

[0148] According to the present embodiment described above, the processor 210 analyzes the target image PI to determine whether the cloth 700 shown in the target image PI contains a pseudo defect. Specifically, as described above, if the variance σb of the brightness of the target image PI is equal to or greater than the threshold value TH3 (YES in S540), it is determined that the cloth 700 shown in the target image PI contains a wrinkle Wr, which is a pseudo defect. Then, if it is determined that the cloth 700 shown in the target image PI contains a pseudo defect, the processor 210 executes the special detection process (S560 in FIG. 15). In this embodiment, the processor 210 analyzes the target image PI to determine whether each target image PI contains a wrinkle Wr. Therefore, even if the wrinkle Wr exists in a portion of the cloth 700 that is not in the conveyance direction Df, the processor 210 can execute the special detection process for the target image PI in which the wrinkle Wr exists. As a result, the deterioration of detection accuracy due to the presence of the wrinkle Wr can be further suppressed. Furthermore, since it is not necessary to recognize the stop and restart of conveyance by the conveyance device 900, the configuration of the inspection system 1000 is simplified.

[0149] As can be seen from the above explanation, the variance σb of the brightness of the target image PI in this embodiment being equal to or greater than the threshold value TH3 is an example of a specific condition.

[0150] C. Variations (1) In the second embodiment described above, the process of determining whether or not the target image PI contains wrinkles Wr using the variance σ of the luminance of the target image PI is merely an example, and is not limited thereto. For example, processor 210 may determine whether or not the target image PI contains wrinkles Wr using a machine learning model. In this case, for example, processor 210 inputs the target image PI into an object detection model MD1 used in the special detection process to detect wrinkled regions WA in the target image PI. If the area of ​​the detected wrinkled regions WA is equal to or greater than a threshold, processor 210 determines that the target image PI contains wrinkles Wr and executes the special detection process. If the area of ​​the detected wrinkled regions WA is less than the threshold, processor 210 determines that the target image PI does not contain wrinkles Wr and executes the normal detection process.

[0151] (2) In the above embodiment, the pseudo defect is a wrinkle Wr. In cases where other pseudo defects, not limited to wrinkles Wr, may be included, a special detection process may be performed to suppress false detection of the pseudo defect. The pseudo defect may be another structure or attachment that is different from the defect to be detected but is similar to the defect to be detected and may be falsely detected.

[0152] For example, a gap NT ( FIG. 14(A) ) that may occur between the upstream end 701 a of the first cloth 700 a and the downstream end 702 b of the second cloth 700 b, or a background structure (e.g., a support plate 953) visible through the gap NT, may be a false defect. In this case, for example, the special detection process may include a process of detecting the gap NT area in the target image PI using an object detection model trained to detect the gap NT area, and a process of detecting defects in an area different from the detected gap NT area. Furthermore, because threads are likely to fray at the upstream and downstream ends of the cloth 700, lint is likely to separate from the cloth 700. Because lint separated from the cloth can be easily removed even if it adheres to the cloth 700, such lint is not a defect that should be detected, but it is likely to be erroneously detected as a linear defect and may be a false defect. In this case, for example, the special detection process may include a process of detecting an area in the target image PI where the lint is located using an object detection model trained to detect lint areas, and a process of detecting defects in an area different from the area where the detected lint is located.

[0153] (3) In the above embodiment, the object to be inspected is cloth 700, but it may be another object. The other object may be, for example, a sheet-like object, such as a metal foil, a resin film, or paper. Furthermore, the object to be inspected may be an object having a shape other than a sheet, such as various products or parts thereof, such as automobiles or electrical appliances. Depending on the object to be inspected, the defects to be detected and the pseudo defects that should not be detected may take various forms. Examples of pseudo defects include structures resulting from temporary deformations that can be easily restored (e.g., wrinkles, folds, bends, twists), easily removable attachments (e.g., dirt, debris), and objects other than the object reflected in the target image (e.g., objects in the background of the object to be inspected).

[0154] (4) In the first embodiment described above, the special detection process and the normal detection process are switched based on specific conditions related to the stoppage or resumption of the conveyance of the cloth 700 based on a user instruction or detection by the upstream end detection sensor 955 ( FIG. 7 ). The conditions for switching between the special detection process and the normal detection process may be various other conditions. For example, if the length of the cloth 700 in the conveyance direction Df is predetermined, the special detection process and the normal detection process may be switched based on a condition related to the conveyance amount of the cloth 700. For example, if the length of the cloth 700 in the conveyance direction Df is 10 meters, it is considered that a portion of the first cloth 700a other than the end thereof passes through the imaging range Ar during the period from when the downstream end of the first cloth 700a is inspected until the cloth 700a is conveyed for 9 meters. For this reason, the processor 210 may execute the normal detection process during the period from when the downstream end of the first cloth 700a is inspected until the cloth 700a is conveyed for 9 meters. From the time when the 9-meter transport has been completed until the 2-meter transport has been completed, it is considered that a portion including the upstream end 703a of the first cloth 700a and the downstream end 704b of the second cloth 700b passes through the photographing range Ar. From the time when the 9-meter transport has been completed until the 2-meter transport has been completed, the processor 210 A special detection process can be performed.

[0155] Furthermore, switching between the special detection process and the normal detection process may be performed based on a combination of conditions related to the stoppage and resumption of the feed of the cloth 700 and conditions related to the feed amount. For example, it is considered that a portion including the upstream end 703a of the first cloth 700a and the downstream end 704b of the second cloth 700b passes through the imaging range Ar during the period from when the upstream end 701a of the first cloth 700a is detected by the upstream end detection sensor 955 until the predetermined length (e.g., 2 meters) of feed is performed. For this reason, the processor 210 may perform the special detection process during the period from when the upstream end 701a of the first cloth 700a is detected by the upstream end detection sensor 955 until the predetermined length (e.g., 2 meters) of feed is performed. The processor 210 may perform the normal detection process during the period from when the predetermined length (e.g., 2 meters) of feed is performed until the upstream end 701a of the second cloth 700b is detected by the upstream end detection sensor 955.

[0156] In the first embodiment, the special detection process is switched to the normal detection process when the feed of the cloth 700 is stopped based on an instruction from the operator (YES in S250 of FIG. 8) and then resumed based on an instruction from the operator (YES in S260 of FIG. 8). Alternatively, the processor 210 may switch from the special detection process to the normal detection process when the feed of the cloth 700 is stopped based on an instruction from the operator (YES in S250 of FIG. 8). In other words, even if the operator has not input both instructions to stop and restart the feed of the cloth 700, the special detection process may be switched to the normal detection process when the operator inputs an instruction to stop the feed of the cloth 700.

[0157] (5) The defect detection process used in the normal detection process and the special detection process in the above embodiment is a process using a mechanism called PaDiM (S320 in FIG. 9, FIGS. 11 and 12). The defect detection process is not limited to this, and may be another process using a machine learning model. For example, the defect detection process may be a process of detecting defects contained in the target image PI by inputting the target image PI into an object detection model that has been trained in advance to detect defects such as scratches. The object detection model may be, for example, a machine learning model that realizes instance segmentation, such as the above-mentioned RTMDet or Mask R-CNN, or a machine learning model that realizes semantic segmentation, such as YOLO (You only look once).

[0158] (6) The object detection model MD1 used in the special detection process of the above embodiment is a machine learning model called RTMDet that realizes instance segmentation, but it may be another machine learning model. For example, the object detection model MD1 may be a machine learning model that realizes other instance segmentation, such as Mask R-CNN, or a machine learning model that realizes semantic segmentation, such as YOLO (You only look once).

[0159] The shadow removal model MD2 used in the special detection process of the above embodiment is a machine learning model called DeshadowNet, but may also be another machine learning model, for example, a stacked condition generative adversic network (ST-CGAN).

[0160] The feature extraction model MD3 used in PaDiM in the above embodiment is a machine learning model called ResNet18, but other machine learning models may also be used. For example, any image recognition model, such as VGG16 or VGG19, may be used as the feature extraction model MD3.

[0161] (7) The special detection process and normal detection process in the above embodiment are merely examples and may be modified as appropriate. For example, in the special detection process of Fig. 13, the process of removing the dark areas Sd (shadows) of the target image PI using the shadow removal model MD2 in S240 may be omitted. Also, in the special detection process of Fig. 13, a process of reducing the brightness of the bright areas Ba of the target image PI may be added.

[0162] (8) The total number of digital cameras used to photograph the cloth 700 is not limited to four and may be any number greater than or equal to one. Furthermore, the device used to photograph the cloth 700 may include a line sensor as an image sensor instead of an area sensor such as a digital camera. In either case, the processor 210 detects defects in the cloth 700 using the target images acquired using these devices. For example, if there is one digital camera, the captured image generated by the single digital camera may be used as the target image PI as is.

[0163] (9) In the above embodiment and modified examples, the processor 210 may cause the GPU 260 to execute various calculations. For example, the processor 210 may cause the GPU 260 to execute some or all of the calculations performed by the object detection model MD1, the shadow removal model MD2, and the feature extraction model MD3. Note that the GPU 260 may be omitted.

[0164] (10) The data processing device 200 in Fig. 2 may be a device of a type different from a personal computer (e.g., a digital camera, a scanner, or a smartphone). Also, multiple devices (e.g., computers) that can communicate with each other via a network may share some of the data processing functions of the data processing device and collectively provide the data processing functions (a system including these devices corresponds to a data processing device).

[0165] (11) In each of the above embodiments, a part of the hardware configuration may be replaced with software, and conversely, a part or all of the software configuration may be replaced with hardware. For example, the processing by the object detection model MD1, the shadow removal model MD2, and the feature extraction model MD3 (FIG. 2) may be performed by a dedicated hardware circuit such as an Application Specific Integrated Circuit (ASIC).

[0166] Furthermore, when some or all of the functions of the present disclosure are realized by a computer program, the program can be provided in a form stored on a computer-readable recording medium (e.g., a non-transitory recording medium). The program can be used in a state stored on the same or a different recording medium (computer-readable recording medium) from when it was provided. The "computer-readable recording medium" is not limited to portable recording media such as memory cards and CD-ROMs, but can also include internal storage devices within a computer, such as various ROMs, and external storage devices connected to a computer, such as a hard disk drive.

[0167] The above-described examples and modifications can be combined as appropriate. The above-described examples and modifications are provided to facilitate understanding of the present disclosure and are not intended to limit the present invention. The present invention may be modified or improved without departing from the spirit thereof, and the present invention includes equivalents thereof. [Explanation of symbols]

[0168] 1000... inspection system, 10... inspection device, 111-114... digital camera, 120... rotary encoder, 130... light source, 20... jig, 200... data processing device, 210... processor, 215... storage device, 220... volatile storage device, 230... non-volatile storage device, 240... display unit, 250... operation unit, 260... graphics processing unit, 270... communication interface, 31, 32... core material, 700... cloth, 710... upstream roll, 720... downstream roll, 900... conveying device Position, 950...conveyance mechanism, 951...upstream roller, 952...downstream roller, 953...support plate, 955...upstream end detection sensor, 980...operation unit, 990...control unit, AM...abnormality map, Ar...shooting range, FI...processed abnormality map, FM...feature matrix, GM...Gaussian matrix, HG...histogram, MD1...object detection model, MD2...shadow removal model, MD3...feature extraction model, MI...processed target image, NI...normal image, PG...computer program, PI...target image, Wr...wrinkle

Claims

1. A computer program comprising: an acquisition function for acquiring an object image showing an object, the object image being generated using an image sensor; a judgment function for judging whether a specific condition indicating that the object shown in the target image may include a pseudo defect different from a defect to be detected is satisfied; a detection function that executes a detection process including a process executed using a machine learning model on the target image to detect defects of the object shown in the target image, the detection function executing a first detection process as the detection process when it is determined that the specific condition is satisfied, and executing a second detection process different from the first detection process as the detection process when it is determined that the specific condition is not satisfied; and A computer program that enables a computer to realize the above.

2. 2. The computer program of claim 1, the target object is a sheet-like object, The pseudo defects are wrinkles that occur in the sheet-like object.

3. 2. The computer program of claim 1, the target object is a sheet-like object, the acquisition function acquires a plurality of target images generated by photographing the sheet-like object conveyed in a conveyance direction by the image sensor, the target images showing portions of the sheet-like object at different positions in the conveyance direction; the detection function executes the detection process for each of the plurality of target images as a processing target; The specific condition is a condition indicating that the target image to be processed may include an edge of the sheet-like object in the conveying direction.

4. 4. A computer program according to claim 3, comprising: a conveying mechanism that conveys the sheet-like object includes a sensor that detects an upstream end of the conveyed sheet-like object in the conveying direction at a predetermined position upstream of an imaging range of the image sensor; A computer program in which, from the time the upstream end of the first sheet-like object is detected at the specified position by the sensor until an end condition is met, it is determined that the specific condition is met, and the first detection process is executed.

5. 5. A computer program according to claim 4, the termination condition is a condition indicating that an end portion of a second sheet-like object to be inspected next to the first sheet-like object on a downstream side in the conveying direction has passed through an imaging range of the image sensor, A computer program in which, from the time the termination condition is satisfied until the upstream end of the second sheet-like object is detected at the specified position by the sensor, it is determined that the specific condition is not satisfied, and the second detection process is executed.

6. 6. A computer program according to claim 5, After the downstream end of the second sheet-like object has passed through the photographing range, conveyance of the second sheet-like object is stopped; a predetermined task is performed by an operator while the transport of the second sheet-like object is stopped, and then the transport of the second sheet-like object is resumed; The computer program, wherein the termination condition is that the operator inputs at least one of an instruction to stop and an instruction to restart the transport of the second sheet-like object.

7. 2. The computer program of claim 1, the determination function includes an analysis function that determines whether the target object shown in the target image includes the pseudo defect by analyzing the target image, The judgment function determines that the specific condition is satisfied when the analysis function determines that the pseudo defect is included.

8. 2. The computer program of claim 1, The first detection process includes: a pseudo defect detection process for identifying the pseudo defect area of ​​the target image using a pseudo defect detection model, which is a machine learning model trained to detect a pseudo defect area where the pseudo defect of the input image is located; a defect detection process for detecting the defect located in a region different from the pseudo defect region of the target image using a machine learning model different from the pseudo defect detection model; a computer program comprising:

9. 9. A computer program according to claim 8, comprising: the target object is a sheet-like object, the pseudo defects are wrinkles that occur in the sheet-like object, the first detection process further includes a shadow removal process of removing a shadow included in the target image; The defect detection process is executed on the target image after the shadow removal process has been executed.

10. 2. The computer program of claim 1, the first detection process includes a first process for detecting the defect included in the target image using a specific machine learning model, and a second process for dealing with the pseudo defect; A computer program, wherein the second detection process includes the first process but does not include the second process.

11. 1. A data processing device, comprising: an acquisition unit that acquires a target image showing an object, the target image being generated using an image sensor; a determination unit that determines whether a specific condition indicating that the object shown in the target image may include a pseudo defect different from a defect to be detected is satisfied; a detection unit that executes a detection process including a process executed using a machine learning model on the target image to detect defects of the object shown in the target image, the detection unit executing a first detection process as the detection process when it is determined that the specific condition is satisfied, and executing a second detection process different from the first detection process as the detection process when it is determined that the specific condition is not satisfied; and A data processing device comprising:

12. A method for manufacturing a sheet-like object, comprising: a creating step of creating the sheet-like object; acquiring an object image showing the sheet-like object, the object image being generated using an image sensor; a determining step of determining whether a specific condition indicating that the sheet-like object shown in the target image may include a pseudo defect different from a defect to be detected is satisfied; a detection process for detecting defects in the sheet-like object shown in the target image by executing a detection process including a process executed using a machine learning model on the target image, the detection process executing a first detection process as the detection process when it is determined that the specific condition is satisfied, and executing a second detection process different from the first detection process as the detection process when it is determined that the specific condition is not satisfied; and an adding step of adding a mark to the sheet-like object to identify the detected defect when the defect is detected in the detecting step; A manufacturing method comprising:

Images