Computer program and data processing device

Abstract

To detect defects in an object using multiple object images.SOLUTION: A first object image includes a first portion of an object and does not include a second portion adjacent to the first portion. A second object image includes the second portion and does not include at least a portion of the first portion. First processing is executed for detecting defects in the object using the first object image, and second processing is executed for detecting defects in the object using the second object image. If specific conditions including a first defect being detected in the first processing and a second defect being detected in the second processing are met, a third object image including a portion of the first object image containing the first defect and a portion of the second object image containing the second defect is acquired, and the defect in the object is detected using the third object image. Alternatively, if the specific conditions are met, a detection result for a specific portion of the object including the first and second portions is generated using detection results of the first and second processing.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present specification relates to a technique for detecting defects in an object using an image of the object. [Background technology]

[0002] Various techniques for inspecting objects have been proposed. Patent Document 1 discloses a technique for inspecting the surface of a web, such as an aluminum sheet or a plastic sheet. In this technique, a surface defect detector detects defects present on the surface of the web and outputs a timing signal. Based on the timing signal, a stop control means slows down the web transport and then stops the defect at a predetermined visual inspection position within the web transport path. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2-038958 Summary of the Invention [Problem to be solved by the invention]

[0004] This specification discloses a new technique for detecting defects in an object using multiple images of the object. [Means for solving the problem]

[0005] The techniques disclosed in this specification can be implemented in the following application examples.

[0006] [Application Example 1] A computer program, comprising: a first acquisition function that acquires a first object image and a second object image showing an object, the first object image and the second object image being images generated using an image sensor, the first object image including an image of a first portion of the object and not including an image of a second portion adjacent to the first portion of the object, and the second object image including an image of the second portion and not including at least a part of the image of the first portion; a first process that detects defects of the object shown in the first object image using the first object image; and a second process that detects defects of the object shown in the first object image using the second object image. a second acquisition function that, when a specific condition is satisfied including that a first defect is detected in the first processing and a second defect is detected in the second processing, acquires a third object image that includes a portion of the first object image that includes the first defect and a portion of the second object image that includes the second defect; and a second detection function that, when the specific condition is satisfied, executes a third process that uses the third object image to detect defects in the object shown in the third object image.

[0007] When a first defect is detected in the first process using the first object image and a second defect is detected in the second process using the second object image, for example, a defect spanning from the first portion to the second portion of the object may be detected as the first defect and the second defect. According to the above configuration, when specific conditions are satisfied, including the first defect being detected in the first process and the second defect being detected in the second process, a defect in the object shown in the third object image including the first defect and the second defect is detected. As a result, a defect spanning from the first portion to the second portion of the object can be appropriately detected using multiple object images (first to third object images).

[0008] [Application Example 2] A computer program comprising: a first acquisition function that acquires a first object image and a second object image showing an object, the first object image and the second object image being images generated using an image sensor, the first object image including an image of a first portion of the object and not including an image of a second portion adjacent to the first portion of the object, and the second object image including an image of the second portion and not including at least a portion of the image of the first portion; a first detection function that executes a first process that detects defects in the object shown in the first object image using the first object image; and a second process that detects defects in the object shown in the second object image using the second object image, the detection results of the first process and the second process including information indicating at least one of a detected defect area and a reliability of defect detection; a result generation function that, when a specific condition is satisfied, including that a first defect is detected in the first process and a second defect is detected in the second process, generates the detection result of a specific portion including the first portion and the second portion of the object using the detection result of the first process and the detection result of the second process; A computer program comprising:

[0009] When a first defect is detected in a first process using a first object image and a second defect is detected in a second process using a second object image, for example, a defect spanning from a first portion to a second portion of the object may be detected as a first defect and a second defect. According to the above configuration, when a specific condition is satisfied, including the first defect being detected in the first process and the second defect being detected in the second process, a detection result of a specific portion including the first portion and the second portion of the object is generated using the detection results of the first process and the detection results of the second process. As a result, a defect spanning from the first portion to the second portion of the object can be appropriately detected using multiple object images (first and second object images).

[0010] The technology disclosed in this specification can be realized in various forms, such as a method for detecting defects in an object, a data processing device for detecting defects in an object, a computer program for realizing these methods and devices, a recording medium on which the computer program is recorded, etc. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram showing the configuration of an inspection system 1000. [Figure 2] FIG. 1 is a perspective view showing a schematic configuration of an inspection device 10. [Figure 3] 4 is a flowchart of an inspection process according to the first embodiment. [Figure 4] FIG. 10 is a diagram showing an example of an image used in the inspection process. [Figure 5] 4 is a flowchart of a defect detection process according to the first embodiment. [Figure 6] FIG. 10 is a diagram showing an example of an image used in the defect detection process. [Figure 7] FIG. 10 is a diagram showing an example of a result display screen. [Figure 8] 10 is a flowchart of a defect detection process according to a second embodiment. [Figure 9] 10 is a flowchart of a defect detection process according to a second embodiment. [Figure 10] 10 is a table showing a method for determining a priority detection result. [Figure 11] 10 is a flowchart of a defect detection process according to a third embodiment. [Figure 12] 10 is a flowchart of a defect detection process according to a fourth embodiment. [Figure 13] 10 is a flowchart of a defect detection process according to a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0012] A. First Example A1. Inspection system configuration FIG. 1 is a diagram showing the configuration of an inspection system 1000. The inspection system 1000 is a system for inspecting a cloth 700. The cloth to be inspected is a known fabric such as a woven fabric, a knitted fabric, or a nonwoven fabric. The inspection system 1000 in FIG. 1 includes a data processing device 200 and an inspection device 10. The data processing device 200 and the inspection device 10 are communicatively connected. This allows the data processing device 200 to receive image data generated by digital cameras 111-114 (described later), control signals from a control unit 990 of a conveying device 900, and the like.

[0013] The data processing device 200 is, for example, a personal computer. The data processing device 200 includes a processor 210, a storage device 215, a display unit 240, an operation unit 250, a graphics processing unit (GPU) 260, and a communication interface 270. These elements are connected to each other via a bus. The storage device 215 includes a volatile storage device 220 and a non-volatile storage device 230.

[0014] The processor 210 is a device configured to perform data processing, such as a central processing unit (CPU) or a system on a chip (SoC). The volatile storage device 220 is, for example, a dynamic random access memory (DRAM), and the non-volatile storage device 230 is, for example, a flash memory.

[0015] The non-volatile storage device 230 stores a computer program PG. The computer program PG includes an object detection model MD as a program module. The object detection model MD will be described later. The processor 210 executes the computer program PG to perform an inspection process, which will be described later.

[0016] The display unit 240 is a device configured to display images, such as a liquid crystal display or an organic EL display. The operation unit 250 is a device configured to receive operations by a user, such as a button, a lever, or a touch panel overlaid on the display unit 240. The display unit 240 and the operation unit 250 may form a so-called touch screen. The user can input various requests and instructions to the data processing device 200 by operating the operation unit 250. The display unit 240 may display operation elements (e.g., buttons, sliders, etc.), and the displayed elements may be operated through operation of the operation unit 250.

[0017] The GPU 260 is a computing device configured to perform various numerical operations such as image processing and machine learning. The GPU 260 performs various operations in accordance with instructions from the processor 210. In this embodiment, the GPU 260 performs operations for the object detection model MD in accordance with instructions from the processor 210.

[0018] The communication interface 270 is an interface for communicating with the inspection device 10 (for example, it includes one or more of a USB interface, a wired LAN interface, an IEEE802.11 wireless interface, and an industrial camera interface (for example, CameraLink, CoaXPress, etc.)). In this embodiment, the communication interface 270 is connected to the inspection device 10.

[0019] The inspection device 10 is a device that performs inspection of the fabric, which is the object of inspection in this embodiment, in cooperation with a data processing device 200. The inspection device 10 includes digital cameras 111-114, a rotary encoder 120, a light source 130, and a conveying device 900.

[0020] The digital cameras 111-114 are devices that use image sensors including imaging elements such as CCDs and CMOSs ​​to capture images of objects (subjects) and generate and acquire captured images of the objects. The digital cameras 111-114 are used to capture images of the fabric to be inspected. The rotary encoder 120 is used to calculate the relative position of the fabric with respect to the conveying device 900 (details will be described later). The light source 130 irradiates light onto the fabric to be inspected in order to capture a clear image of the fabric.

[0021] The conveying device 900 is a device that conveys the fabric to be inspected, and includes a conveying mechanism 950, an operation unit 980 that receives operations by the user, and a control unit 990.

[0022] FIG. 2 is a perspective view showing a schematic configuration of the inspection device 10. The conveying mechanism 950 includes a plurality of rollers including rollers 951 and 952, a support plate 953, and a conveying motor (not shown). At least a portion of the plurality of rollers is driven by the conveying motor to convey the cloth 700 to be inspected. The support plate 953 is a flat plate disposed between the upstream roller 951 and the downstream roller 952, and supports a portion of the cloth 700 to be inspected that is to be photographed by the digital cameras 111-114. In FIG. 2, the rotary encoder 120 and the operation unit 980 and control unit 990 of the conveying device 900 are not shown.

[0023] The partial path Pth in the figure indicates the portion of the conveyance path of the cloth 700 between the rollers 951 and 952, i.e., the portion along the support plate 953. The direction Df in the figure indicates the conveyance direction on the partial path Pth (direction Df is also referred to as the conveyance direction Df). The orthogonal direction Dt indicates a direction parallel to the flat portion 700F and perpendicular to the partial path Pth. Hereinafter, the upstream side of the conveyance direction Df of the conveyance path of the cloth 700 will be simply referred to as the upstream side, and the downstream side of the conveyance direction Df of the conveyance path of the cloth 700 will be simply referred to as the downstream side.

[0024] The cloth 700 to be inspected is a long piece of cloth that is longer than the partial path Pth. The cloth 700 is wound around a core 31 to form an upstream roll 710. The upstream roll 710 is disposed in a mounting section (not shown) upstream of the partial path Pth. The upstream mounting section includes, for example, a holding member including two or more rollers, and is configured to rotatably hold the upstream roll 710. The cloth 700 pulled out from the upstream roll 710 is transported from the position of the upstream roller 951 to the position of the downstream roller 952 along the partial path Pth.

[0025] The fabric 700 conveyed to the downstream roller 952 is wound around the core 32 downstream of the partial path Pth. The fabric 700 wound around the core 32 forms the downstream roll 720. The downstream roll 720 is disposed in a mounting section (not shown) downstream of the partial path Pth. The downstream mounting section includes, for example, a holding member including two or more rollers, and is configured to rotatably hold the downstream roll 720.

[0026] Between the rollers 951 and 952, the cloth 700 is supported by the support plate 953 to form a flat portion, that is, a flat portion 700F. The light source 130 is disposed at a position where it irradiates the flat portion 700F with light.

[0027] 2, the imaging range Ar, which is the area captured by the digital cameras 111-114, is hatched. The imaging range Ar is a rectangular area having two sides Ar1 and Ar2 parallel to the partial path Pth and two sides Ar3 and Ar4 perpendicular to the partial path Pth. The first side Ar1 and the second side Ar2 of the imaging range Ar are located outside the cloth 700.

[0028] In the figure, partial areas R11-R14 indicate areas photographed by digital cameras 111-114, respectively. In this embodiment, the digital cameras 111-114 are arranged side by side in the orthogonal direction Dt, and therefore the partial areas R11-R14 are also arranged side by side in the orthogonal direction Dt. The photographing range Ar is the entire partial areas R11-R14.

[0029] The operation unit 980 (FIG. 1) of the transport device 900 includes a plurality of switches (not shown) and is configured to receive operations from an operator. In this embodiment, the plurality of switches are foot switches operated by the operator with their feet, and include at least a transport start switch and a transport stop switch. Alternatively, the plurality of switches may be push switches operated by the operator with their hands, or buttons displayed on a touch panel.

[0030] The control unit 990 (FIG. 1) of the conveying device 900 is connected to, for example, the operation unit 980 and a conveying motor and power supply (not shown). The control unit 990 is an electric circuit configured to control the conveying motor in response to the operation of the operation unit 980 or a signal from the data processing device 200. The control unit 990 may be configured using a computer or dedicated hardware (such as an Application Specific Integrated Circuit (ASIC)). For example, when a conveyance start switch is pressed while the cloth 700 is not being conveyed, the control unit 990 starts conveying the cloth 700. When a stop switch is pressed while the cloth 700 is being conveyed, the conveying device 900 stops conveying the cloth 700. When the control unit 990 receives a conveyance stop signal from the data processing device 200 requesting that conveyance be stopped while the cloth 700 is being conveyed, the control unit 990 stops conveying the cloth 700.

[0031] A rotary encoder 120 is connected to the conveying device 900 to detect the direction and amount of position change due to conveyance. For example, the rotary encoder 120 is connected to a roller (e.g., the upstream roller 951). The rotary encoder 120 may have various configurations for detecting the direction and amount of position change due to conveyance. For example, the rotary encoder 120 may be an incremental encoder. An incremental encoder alternately outputs A pulses and B pulses in response to position changes. The number of output pulses indicates the amount of movement. The phase difference (positive or negative) between the A pulse and the B pulse indicates the direction of movement. The data processing device 200 (FIG. 1) can calculate the current conveyance position of the cloth 700 conveyed by the conveying device 900 (i.e., the relative position of the cloth 700 with respect to the conveying device 900) by counting the number of pulses output from the rotary encoder 120 according to the phase difference (i.e., the direction). Note that a counter that counts the number of pulses according to the direction may be connected to the rotary encoder 120. The data processing device 200 may use the information from the counter to obtain the current relative position of the fabric 700 .

[0032] A2.Inspection process The inspection process for the cloth 700 is performed, for example, after the cloth manufacturing process or before manufacturing products such as clothing using the cloth. In the inspection process for the cloth 700, an operator mounts the roll of cloth 700 to be inspected as the upstream roll 710 on an upstream mounting portion (not shown) of the conveying mechanism 950. The operator pulls out the downstream end of the cloth 700 from the upstream roll 710. Specifically, the downstream end of the cloth 700 passes through a partial path Pth ( FIG. 2 ) that passes through the imaging range Ar on the support plate 953, is pulled out until it reaches a position downstream of the support plate 953, and is wound around the interlining 32. The operator mounts the interlining 32 around which the downstream end of the cloth 700 is wound on an downstream mounting portion (not shown) of the conveying mechanism 950. As a result, the cloth 700 to be inspected is set in the inspection device 10 so that it can be inspected, as shown in FIG. 2. At the start of the inspection, the portion of the cloth 700 to be inspected that is located downstream of the shooting range Ar in the conveying direction Df cannot be inspected by the inspection process (described later) by the data processing device 200, so it is inspected visually by an operator, for example.

[0033] When the operator sets the cloth 700 to be inspected on the inspection device 10, he or she operates the operation unit 250 of the data processing device 200 to start the computer program PG and inputs an instruction to start the inspection process. This causes the data processing device 200 to start the inspection process. The inspection process is a process for inspecting the cloth 700 using captured images of the cloth 700 in synchronization with the conveyance of the cloth 700 by the conveying device 900. Details of the inspection process will be described later.

[0034] When the data processing device 200 starts the inspection process, the operator presses the conveyance start switch on the operation unit 980 of the conveyance device 900. In response to the pressing of the conveyance start switch, the control unit 990 of the conveyance device 900 drives the conveyance motor to start conveying the cloth 700 in the conveyance direction Df.

[0035] When the conveyance of the cloth 700 begins, the cloth 700 is inspected by an inspection process for each portion that passes through the imaging range Ar. FIG. 3 is a flowchart of the inspection process of the first embodiment. In S100, the processor 210 of the data processing device 200 determines whether the imaging timing has arrived. For example, the first imaging timing is the start of the inspection process. The nth (n is an integer greater than or equal to 2) imaging timing is the timing at which the inspected cloth has been conveyed from the (n-1)th imaging timing by a distance AH (FIG. 2) of the imaging range Ar in the conveying direction Df. Therefore, in FIG. 2, when a portion of the cloth 700 located within the imaging range Ar is imaged at the nth imaging timing, the portion of the cloth 700 imaged at the (n-1)th imaging timing is a portion located within the adjacent range Arb adjacent to the imaging range Ar on the downstream side in the conveying direction Df in FIG. 2. This makes it possible to sequentially generate multiple inspection images IMs (described below) showing portions of the conveyed cloth 700 that are positioned at different positions in the conveying direction Df. In a modified example, in order to inspect the cloth 700 more reliably and without omission, the timing of the nth photograph may be set to the timing when the cloth to be inspected has been transported a length slightly shorter than length AH (Figure 2) from the timing of the (n-1)th photograph.

[0036] The processor 210 can recognize the amount of transport of the cloth 700 based on a count value, which is the number of pulses obtained from the rotary encoder 120. For this reason, the processor 210 determines that the timing for photographing has arrived based on the count value each time the cloth 700 is transported by the length AH of the photographing range Ar in the transport direction Df.

[0037] If the timing to capture images has not arrived (S100: NO), the processor 210 waits until the timing to capture images arrives. If the timing to capture images arrives (S100: YES), the processor 210 acquires captured images using the digital cameras 111-114 in S110. Specifically, the processor 210 provides a capture instruction to each of the digital cameras 111-114. In response to the capture instruction, the digital cameras 111-114 capture images of the cloth 700 and generate captured images IM1-IM4. The processor 210 acquires data of the captured images IM1-IM4 from each of the digital cameras 111-114.

[0038] FIG. 4 is a diagram showing an example of an image used in the inspection process. FIG. 4(A) shows examples of captured images IM1-IM4 obtained from the digital cameras 111-114 (FIG. 1), respectively. Each of the captured images IM1-IM4 is a rectangular image having two sides parallel to the first direction Dx and two sides parallel to a second direction Dy perpendicular to the first direction Dx. The second direction Dy is a direction roughly parallel to the conveying direction Df (FIG. 1). The data of each of the captured images IM1-IM4 is bitmap data representing the color values ​​of a plurality of pixels arranged in a matrix along the first direction Dx and the second direction Dy. The color values ​​are, for example, RGB values ​​including the respective gradation values ​​of red R, green G, and blue B (for example, values ​​greater than or equal to zero and less than or equal to 255).

[0039] As shown in FIG. 2, partial regions R11-R14 corresponding to captured images IM1-IM4 (FIG. 4(A)) are arranged side by side in the orthogonal direction Dt. The first captured image IM1 and the fourth captured image IM4 represent the cloth 700 and the background BG, respectively. The second captured image IM2 and the third captured image IM3 represent the cloth 700, respectively. These captured images IM1-IM4 as a whole represent the entire capturing range Ar. Although not shown, the background BG may represent various objects located outside the cloth 700, such as a part of the conveying device 900.

[0040] In the example of FIG. 4(A), the captured image IM3 includes a defect FD (e.g., a linear scratch) in the cloth 700. The defect FD can be formed due to various causes. For example, a defect in the thread forming the cloth 700 can form the linear defect. Furthermore, a linear defect (e.g., a scratch or a linear drawing) can be formed due to contact between the cloth 700 and another member (e.g., a device for carrying the cloth 700, a writing implement, etc.).

[0041] In S120, the processor 210 combines the four captured images IM1-IM4 to generate data for a single inspection image IMs. Fig. 4(B) shows the inspection image IMs generated from the captured images IM1-IM4 of Fig. 4(A). The inspection image IMs is a strip-shaped image that represents the portion of the cloth 700 within the imaging range Ar (Fig. 2).

[0042] Various methods may be used to generate the inspection image IMs. For example, the partial regions R11-R14 (FIG. 2) may be arranged side by side in the orthogonal direction Dt within the imaging range Ar without any gaps and without overlapping each other. In this case, the processor 210 may generate data for the inspection image IMs by connecting the ends of the captured images IM1-IM4 (FIG. 4(A)) arranged side by side in the first direction Dx. Alternatively, the partial regions R11-R14 may be arranged so that two adjacent partial regions partially overlap each other. In this case, two adjacent images among the captured images IM1-IM4 include overlapping portions. In this case, the processor 210 combines the captured images IM1-IM4 by using the corresponding portion of one of the two images as the image of the overlapping portion of the two images.

[0043] In S130, the processor 210 divides the inspection image IMs into image sizes (also referred to as input sizes) that can be input to the object detection model MD. As a result, k (k is an integer of 2 or greater) partial images PI are generated from one inspection image IMs. To distinguish each of the k partial images PI, an identification number i (i is an integer of 1 or greater and k or less) is assigned to the partial image PI. For example, the first partial image PI is expressed as partial image PI1, and the i-th partial image PI is expressed as partial image PIi.

[0044] FIG. 4C shows an example of k partial images PI (PI1-PIk). The arrangement of the partial images PI1-PIk on the inspection image IMs is predetermined. FIG. 4B shows the range of the partial images PI1-PIk on the inspection image IMs. In this embodiment, the partial images PI1-PIk are aligned in the first direction Dx on the inspection image IMs. Among the partial images PI1-PIk, two adjacent partial images partially overlap. For example, as shown in FIGS. 4B and 4C, the right edge of partial image PI1 and the left edge of partial image PI2 represent the same area IA1 (also referred to as overlapping area IA1) of the inspection image IMs. The right edge of partial image PIi and the left edge of partial image PIi+1 represent the same area IAi (also referred to as overlapping area IAi) of the inspection image IMs. All parts of the inspection image IMs are included in at least one of the k partial images PI.

[0045] 4(C), partial image PIi includes partial defect FD1, which is a left portion of defect FD included in inspection image IMs. Partial image PIi+1 includes partial defect FD2, which is a right portion of defect FD included in inspection image IMs. Partial images PIi and PIi+1 include an overlapping area IAi, so partial defect FD1 in partial image PIi and partial defect FD2 in partial image PIi+1 partially overlap.

[0046] In S140, the processor 210 executes a defect detection process using the k partial images PI. The defect detection process is a process for detecting defects (e.g., FD) in the cloth 700 shown in the inspection images IMs from which the k partial images PI are derived. Details of the defect detection process will be described later.

[0047] When the defect detection process of S140 is completed, the processor 210 returns to S100 and waits until the next photographing timing arrives. As a result, a plurality of inspection images IMs showing different portions of the cloth 700 are sequentially acquired (S110-S120), a plurality of partial images PI1-PIk are generated based on each inspection image IMs (S130), and the defect detection process (S140) is performed using the plurality of partial images PI1-PIk. Although not shown in the figure, the inspection process ends, for example, when the inspection of the entire cloth 700 is completed.

[0048] A3. Defect detection processing Next, the defect detection process of S140 in Fig. 3 will be described. Fig. 5 is a flowchart of the defect detection process of the first embodiment. In S210, the processor 210 inputs each of the k partial images PI to the object detection model MD, and detects defects contained in the inspection image IMs that is the basis of the k partial images PI.

[0049] The object detection model MD is a well-known machine learning model including a convolutional neural network (CNN). In this example, the object detection model MD is a model called "RTMDet," which is disclosed in the following paper: Chengqi Lyu, Wenwei Zhang, Haian Huang, Yue Zhou, Yudong Wang, Yanyi Liu, Shilong Zhang, and Kai Chen. "Rtmdet: An Empirical Study of Designing Real-Time Object Detectors," arXiv.2212.07784, December 16, 2022, https: / / doi.org / 10.48550 / arXiv.2212.07784.

[0050] The RTMDet is a model that detects the bounding box and category (i.e., object type) of an object and performs pixel-level region division called instance segmentation. In this embodiment, the object detection model MD is pre-trained to output multiple detection results for multiple types of defects that may exist in the cloth 700, specifically, linear scratches, holes, chalk marks, etc., as detection targets. The object detection model MD may be trained using various methods, such as the training method described in the above-mentioned paper by the RTMDet.

[0051] One detection result includes box information indicating a bounding box (rectangular region) surrounding the defect, type information indicating the type of defect, the reliability of the detection (also called certainty), and mask data indicating the pixel-level region where the defect is located. The bounding box and mask data are examples of region information indicating the region where the defect is located.

[0052] The bounding box includes information indicating, for example, the coordinates of the center of the rectangular area, the length in the first direction Dx (height of the rectangle), and the length in the second direction Dy (width of the rectangle). The type information indicates, for example, one of multiple types of defects to be detected. The reliability is, for example, a numerical value greater than 0 and less than 1. The higher the reliability, the more likely the detection result is to be correct. The mask data is, for example, binary image data indicating one or more defective pixels that constitute the defect among the multiple pixels included in the bounding box (rectangular area). The mask is an area indicated by one or more defective pixels.

[0053] By inputting one partial image PI into the object detection model MD, multiple (e.g., several tens) detection results are output. The processor 210 identifies, from among the multiple detection results, detection results having a reliability equal to or greater than a predetermined threshold THd as valid detection results. The threshold THd is a value experimentally determined in advance so as to enable detection of defects in fabric. For example, if the partial image PI does not contain a defect to be detected, the valid detection result obtained by inputting the partial image PI into the object detection model MD is zero. For example, if the partial image PI contains a defect to be detected, inputting the partial image PI into the object detection model MD will obtain a valid detection result indicating the defect. The processor 210 uses the valid detection results to generate mask images MI1-MIk indicating the defect detection results.

[0054] FIG. 4(D) shows examples of mask images MI1-MIk that indicate valid detection results for partial images PI1-PIk in FIG. 4(C). Mask images MI1-MIk are mask images corresponding to partial images PI1-PIk. Mask image MIi corresponding to partial image PIi that includes partial defect FD1 includes a mask MS1 (a region consisting of one or more defective pixels) that indicates the location of partial defect FD1. Mask image MIi+1 corresponding to partial image PIi+1 that includes partial defect FD2 includes a mask MS2 that indicates the location of partial defect FD2. Mask images MI1, MI2, MIi-1, and MIk corresponding to partial images PI1, PI2, PIi-1, and PIk that do not include defects do not include a mask.

[0055] In Fig. 4(D), the dashed rectangles indicate bounding boxes. In Fig. 4(D), a bounding box BB1 that surrounds the mask MS1 (partial defect FD1) is associated with the mask MS1 of the mask image MIi. A bounding box BB2 that surrounds the mask MS2 (partial defect FD2) is associated with the mask MS2 of the mask image MIi+1.

[0056] As can be seen from the above description, the process of S210 includes a process of detecting defects in the cloth 700 shown in each partial image PI using each of the k partial images PI.

[0057] In S220, the processor 210 generates a detection result image DI using the generated mask images MI1-MIk. FIG. 4(E) shows the detection result image DI corresponding to the inspection image IMs of FIG. 4(B). The detection result image DI is, for example, an image of the same size as the inspection image IMs. The detection result image DI describes the masks and bounding boxes of valid detection results. In the example of FIG. 4(E), the detection result image DI includes masks MS1 and MS2 and bounding boxes BB1 and BB2 associated with the masks. Note that the masks MS1 and MS2 correspond to the same defect FD in the inspection image IMs (FIG. 4(B)). Therefore, the masks MS1 and MS2 overlap each other in the area of ​​the detection result image DI corresponding to the overlapping area IAi (FIGS. 4(B) and 4(C)) between the partial images PIi and PIi+1. In the detection result image DI, the masks MS1 and MS2 overlap each other, but are not combined by taking a logical sum, and are held as distinguishable data.

[0058] In S230, processor 210 determines whether a defect has been detected. If there is one or more valid detection results, i.e., detection results having a reliability equal to or greater than threshold THd, processor 210 determines that a defect has been detected. If there is no valid detection result, processor 210 determines that a defect has not been detected. If no defect has been detected (S230: NO), processor 210 proceeds to S280, which will be described later.

[0059] If a defect is detected (S230: YES), the processor 210 transmits a conveyance stop signal to the conveyance device 900 in S235, requesting that conveyance be stopped. Upon receiving the conveyance stop signal, the control unit 990 of the conveyance device 900 stops conveyance of the fabric 700. In this way, if a defect is detected (S230: YES), conveyance of the fabric 700 is stopped.

[0060] In S240, processor 210 determines whether the detected defect is a defect that straddles two partial images. In this embodiment, if the following three conditions are met, the detected defect is determined to be a defect that straddles two partial images.

[0061] Condition 1: A defect is detected in one partial image. Condition 2: A defect is detected in one partial image and another partial image adjacent to it. Condition 3: At least a part of a defect detected in one partial image overlaps with at least a part of a defect detected in another partial image.

[0062] For example, in the example of Fig. 4, a partial defect FD1 is detected in partial image PIi of Fig. 4(C), as shown by the mask MS1 and bounding box BB1 of mask image MIi of Fig. 4(E). And, a partial defect FD2 is detected in partial image PIi+1, which is adjacent to partial image PIi of Fig. 4(C), as shown by the mask MS2 and bounding box BB2 of mask image MIi+1 of Fig. 4(E). Therefore, in the example of Fig. 4, conditions 1 and 2 are satisfied.

[0063] Condition 3 is determined, for example, using a mask indicating the detected defect. In the detection result image DI of FIG. 4(E), a mask MS1 of mask image MIi is a mask indicating partial defect FD1 detected in partial image PIi. A mask MS2 of mask image MIi+1 of FIG. 4(E) is a mask indicating partial defect FD2 detected in partial image PIi+1. The mask MS1 of mask image MIi and the mask MS2 of mask image MIi+1 overlap (FIG. 4(E)). For this reason, in the example of FIG. 4, condition 3 is satisfied. For this reason, in the example of FIG. 4, the detected defect is determined to be a defect that straddles two partial images.

[0064] Condition 3 can be said to be a condition indicating that a defect (e.g., partial defect FD1) detected in one partial image (e.g., partial image PIi) and a defect (e.g., partial defect FD2) detected in another partial image (e.g., partial image PIi+1) constitute one continuous defect (e.g., defect FD).

[0065] If a defect FDx indicated by a dashed line in Fig. 4(C) is detected in partial image PI1 but no defect is detected in partial image PI2, then mask image MI1 in Fig. 4(D) includes mask MSx and mask image MI2 does not include a mask. In this case, conditions 2 and 3 are not satisfied, and therefore the detected defect is determined not to be a defect that straddles two partial images.

[0066] Condition 3 may be determined using a bounding box. For example, if at least a portion of a bounding box of a defect detected in one partial image (e.g., BB1 in FIG. 4D) overlaps with at least a portion of a bounding box of a defect detected in another partial image (e.g., BB2 in FIG. 4D) in the detection result image DI, the detected defect may be determined to be a defect that straddles two partial images.

[0067] Alternatively, condition 3 may be omitted, and if conditions 1 and 2 are satisfied, the detected defect may be determined to be a defect that straddles two partial images.

[0068] If the detected defect does not straddle two partial images (S240: NO), the processor 210 proceeds to S280, which will be described later.

[0069] If the detected defect is a defect that straddles two partial images (S240: YES), in S250, processor 210 generates a new partial image N1 that includes the defect that straddles two partial images. For example, processor 210 determines the center of gravity CC of the defect that straddles two partial images. Processor 210 generates the new partial image N1 by cutting out, from inspection image IMs, an image that is centered in the first direction Dx at the center of gravity CC and has the input size of object detection model MD.

[0070] The calculation of the center of gravity CC is determined, for example, using a mask indicating the detected defect. In the example of Fig. 4, the processor 210 calculates the overall center of gravity of the mask MS1 of the mask image MIi and the mask MS2 of the mask image MIi+1 in the detection result image DI as the center of gravity CC of the defect spanning two partial images (Fig. 4(E)). Alternatively, the calculation of the center of gravity CC may be determined, for example, using a bounding box of the detected defect. For example, the midpoint between the center of gravity of the bounding box BB1 in Fig. 4(E) and the center of gravity of the bounding box BB2 in Fig. 4(E) may be calculated as the center of gravity CC of the defect spanning two partial images.

[0071] Since the detection result image DI and the inspection image IMs are images of the same size, determining the center of gravity CC in the detection result image DI is equivalent to determining the center of gravity CC in the inspection image IMs. In the example of FIG. 4, as shown in FIG. 4(B), a new partial image NI is cut out from the inspection image IMs with the center of gravity CC at the center in the first direction Dx. FIG. 6 is a diagram showing an example of an image used in the defect detection process. FIG. 6(A) shows a partial image NI cut out from the inspection image IMs. The partial image NI in FIG. 6(A) includes the entire defect FD in the inspection image IMs. In this way, the partial image NI is generated with the center of gravity CC at the center in the first direction Dx, and therefore the partial image NI is generated with the entire partial defect FD1 and the partial defect FD2, i.e., the defect FD is positioned at the center.

[0072] In S260, the processor 210 inputs a new partial image NI into the object detection model MD and detects defects contained in the partial image NI. That is, the processor 210 generates a plurality of (e.g., several tens of) detection results by inputting the partial image NI into the object detection model MD. The processor 210 identifies, among the plurality of detection results, a detection result having a reliability equal to or greater than a predetermined threshold THd as a valid detection result. The processor 210 uses the valid detection results to generate a mask image NM corresponding to the partial image NI.

[0073] 6B shows an example of a mask image NM corresponding to the partial image NI in FIG. 6A. The mask image NM includes one mask MSf indicating the location of the defect FD in the inspection image IMs. A bounding box BBf surrounding the mask MSf is associated with the mask MSf.

[0074] In S270, processor 210 modifies the detection result image DI (FIG. 4(E)) generated in S220 using the newly generated mask image NM. As a result, the modified detection result image DIf of FIG. 6(C) is generated. Specifically, processor 210 replaces the portion of detection result image DI within target area NA corresponding to partial image NI of inspection image IMs with the newly generated mask image NM. Specifically, target area NA is an area centered in the first direction Dx at the center of gravity CC described above, and has the input size of object detection model MD.

[0075] In S280, the processor 210 displays a screen showing the detection results (also referred to as a result display screen) on the display unit 240. FIG. 7 is a diagram showing an example of the result display screen. The result display screen W1 in FIG. 7 is an example of a screen that is displayed when a defect is detected (YES in S230 in FIG. 5) and the defect straddles two partial images (YES in S240 in FIG. 5). The result display screen W1 in FIG. 7 includes a message MG1 notifying that a defect has been detected, an inspection image IMs, and a corrected detection result image DIf superimposed on the inspection image IMs. The result display screen W1 also includes text TX indicating the type of the detected defect, the reliability of the detection, and the size of the defect (e.g., the length of a linear scratch).

[0076] The information on the type of defect included in the text TX is information based on the type information included in the detection result output from the object detection model MD in S260. The reliability of the detection included in the text TX is the reliability included in the detection result output from the object detection model MD. The size of the defect included in the text TX is calculated based on, for example, the size (number of pixels in the vertical or horizontal direction) of the mask MSf or the bounding box BBf included in the detection result image DIf.

[0077] The worker recognizes that a defect has been detected by looking at the result display screen W1. At this point, the conveying device 900 stops conveying the cloth 700, so the portion of the cloth 700 containing the detected defect is stopped on the support plate 953. For example, after the worker recognizes the location of the defect by looking at the result display screen W1, the worker confirms the defect in the actual cloth 700 and adds a marker to the cloth 700 for later identification. Then, the worker presses the conveyance start switch on the operation unit 980 of the conveying device 900. In response to the pressing of the conveyance start switch, the control unit 990 of the conveying device 900 drives the conveyance motor to resume conveying the cloth 700 in the conveyance direction Df. The inspection process in FIG. 3 is performed synchronously with the conveyance of the cloth 700 (S100 in FIG. 3). Therefore, when the conveyance of the cloth 700 is resumed, the inspection process is also resumed.

[0078] If no defect is detected in the inspection image IMs (NO in S230 of FIG. 5), a result display screen (not shown) including, for example, a message indicating that no defect is detected and the inspection image IMs is displayed. In this case, a conveyance stop signal is not sent to the conveying device 900, so conveyance of the cloth 700 is not stopped.

[0079] Furthermore, if a defect is detected (YES in S230 in FIG. 5) and the defect does not straddle two partial images (NO in S240 in FIG. 5), a display screen (not shown) similar to the result display screen W1 in FIG. 7 is displayed. However, in the result display screen displayed in this case, the detection result image superimposed on the inspection image IMs is the detection result image DI generated in S220. This is because the process of correcting the detection result image DI in S270 is not performed in this case. Furthermore, in this case, a conveyance stop signal is sent to the conveying device 900 in S235, and the conveyance of the cloth 700 is stopped. In this case, as described above, the operator marks the detected defects, and then the conveyance of the cloth 700 is resumed.

[0080] According to the present embodiment described above, the processor 210 acquires a plurality of partial images PI, including partial images PIi and PIi+1, showing the cloth 700 (S110-S130 in FIG. 3, FIG. 4C). Here, of the cloth 700 shown in the inspection image IMs in FIG. 4B, a portion p1 (FIG. 4B) including approximately the left half of the defect FD is defined as a first portion, and a portion p2 (FIG. 4B) including approximately the right half of the defect FD is defined as a second portion. The second portion p2 is a portion adjacent to the right side of the first portion p1 (FIG. 4B). The boundary between the first portion p1 and the second portion p2 is the right edge of the overlapping region IAi between the partial images PIi and PIi+1. It can be said that the partial image PIi in FIG. 4C is an image that includes an image of the first portion p1 of the cloth 700 but does not include an image of the second portion p2 of the cloth 700. It can be said that the partial image PIi+1 in Figure 4(C) is an image that includes an image of the second portion p2 of the cloth 700, but does not include an image of the first portion p1 of the cloth 700 excluding the overlapping area IAi.

[0081] Processor 210 executes a detection process including a first process for detecting defects using partial image PIi and a first process for detecting defects using partial image PIi+1 (S210 in FIG. 5). When specific conditions (conditions 1 to 3 above) are satisfied, including that partial defect FD1 is detected in the first process using partial image PIi and partial defect FD2 is detected in the second process using partial image PIi (YES in S240 in FIG. 5), processor 210 acquires partial image NI (FIG. 6(A)) that includes a portion of partial image PIi that includes partial defect FD1 and a portion of partial image PIi+1 that includes partial defect FD2—in other words, partial image NI that includes the entire defect FD that straddles the two partial images (S250 in FIG. 5). When the specific conditions are satisfied (YES in S240 in FIG. 5), processor 210 executes a third process (S260 in FIG. 5) that uses partial image NI to detect defects in the fabric 700 shown in partial image NI.

[0082] When a defect is detected in both adjacent partial images PIi and PIi+1, a defect FD that straddles partial images PIi and PIi+1 may be detected as partial defects FD1 and FD2, as shown in the example of FIG. 4 . In this case, because only a portion of the defect FD is included in both partial images PIi and PIi+1, the first process using partial image PIi and the second process using partial image PIi+1 may not correctly generate information indicating the defect area (mask or bounding box) or defect type information. For example, the determination result of the defect type included in the detection result of the first process may differ from the determination result of the defect type included in the detection result of the second process. According to this embodiment, when specific conditions are satisfied, including the detection of partial defect FD1 in the first process using partial image PIi and the detection of partial defect FD2 in the second process using partial image PIi+1, a defect in the cloth 700 is detected using partial image NI including partial defects FD1 and FD2. As a result, the plurality of partial images PIi, PIi+1, and NI can be used to appropriately detect a defect FD that straddles the first portion p1 and the second portion p2 of the cloth 700. For example, in this embodiment, the detection result finally obtained using the partial image NI that includes the entire defect FD is adopted, so that type information of the defect FD, a mask corresponding to the entire defect FD, and the like can be generated with high accuracy.

[0083] Furthermore, according to this embodiment, the first process using the partial image PIi, the second process using the partial image PIi+1, and the third process using the partial image NI are processes that use the object detection model MD, which is a trained machine learning model (S210, S260 in FIG. 5). As a result, the machine learning model can be used to appropriately detect the defect FD in the fabric 700.

[0084] Furthermore, according to this embodiment, the processor 210 generates a partial image NI in which the defect FD is positioned at the center (FIGS. 4(B) and 6(A)). As a result, the processor 210 increases the likelihood that the entire defect FD will be included in the partial image NI. Therefore, for example, it increases the likelihood that type information and a mask of the defect FD will be generated based on the overall characteristics of the defect FD. As a result, for example, the type information and a mask of the defect FD can be generated with greater accuracy.

[0085] Furthermore, in this embodiment, the machine learning model used in the first process using partial image PIi and the machine learning model used in the second process using partial image PIi+1 are the same model (object detection model MD in this embodiment). As a result, both partial defect FD1 included in partial image PIi and partial defect FD2 included in partial image PIi+1 can be detected using the same determination criteria.

[0086] Furthermore, in this embodiment, the specific condition includes a condition (condition 3 in the above embodiment) indicating that a defect (e.g., partial defect FD1) detected in one partial image (e.g., partial image PIi) and a defect (e.g., partial defect FD2) detected in another partial image (e.g., partial image PIi+1) constitute a single continuous defect (e.g., defect FD). As a result, when a single continuous defect (e.g., defect FD) exists that straddles two partial images PI, a third process using a new partial image NI can be executed. Therefore, a single continuous defect that straddles two partial images PI can be appropriately detected.

[0087] As can be seen from the above explanation, in this embodiment, partial image PIi is an example of the first object image, partial image PIi+1 is an example of the second object image, and partial image NI is an example of the third object image. Also, partial defect FD1 is an example of the first defect, and partial defect FD2 is an example of the second defect.

[0088] B. Second Example In the second embodiment, the content of the defect detection process differs from that in the first embodiment. Other configurations of the second embodiment are similar to those of the first embodiment. FIG. 8 is a flowchart of the defect detection process in the second embodiment. FIG. 9 is an explanatory diagram of the second embodiment. In the flowchart of FIG. 8, steps that are the same as those in the flowchart of FIG. 5 are given the same reference numerals as in FIG. 5, and steps that differ from those in the flowchart of FIG. 5 are given the suffix "B" to the reference numeral. Below, we will explain the parts of the defect detection process in FIG. 8 that are different from the defect detection process in FIG. 5.

[0089] As shown in Fig. 8, in the defect detection process of the second embodiment, S250B to S275B in Fig. 8 are executed instead of S250 to S270 in Fig. 5. That is, in the second embodiment, when a detected defect straddles two partial images, the process executed is different from that in the first embodiment.

[0090] In S250B, processor 210 determines whether the defect types in the defect detection results of the two partial images are different. In the example of Fig. 4, it determines whether the type of partial defect FD1 in the detection result of partial image PIi is different from the type of partial defect FD2 in the detection result of partial image PIi+1.

[0091] If the types of defects in the detection results of the two partial images are different (S250B: YES), processor 210 determines in S255B which detection result (also referred to as a priority detection result) should be prioritized from the detection results of the defects in the two partial images. In the example of Fig. 4, the priority detection result is determined from the detection result of partial defect FD1 detected in partial image PIi and the detection result of partial defect FD2 detected in partial image PIi+1.

[0092] 10 is a table showing a method for determining a prioritized detection result. The processor 210 calculates and compares the area of ​​the partial defect FD1 (also referred to as the defect area S1) with the area of ​​the partial defect FD2 (also referred to as the defect area S2). For example, the area (number of pixels) of the mask MS1 of the partial defect FD1 is used as the defect area S1, and for example, the area (number of pixels) of the mask MS2 of the partial defect FD2 is used as the defect area S2. Alternatively, the areas of the bounding boxes BB1 and BB2 of the partial defects FD1 and FD2 may be used as the defect areas S1 and S2. Furthermore, the processor 210 compares the reliability C1 included in the detection result of the partial defect FD1 with the reliability C2 included in the detection result of the partial defect FD2.

[0093] 10, in a first case where the defect area S1 is larger than the defect area S2 and the reliability C1 is larger than the reliability C2, the processor 210 determines the detection result of the partial defect FD1 as the priority detection result. In a second case where the defect area S2 is larger than the defect area S1 and the reliability C2 is larger than the reliability C1, the processor 210 determines the detection result of the partial defect FD2 as the priority detection result.

[0094] When neither the first nor second case applies, the processor 210 calculates a priority detection result based on an index value V1 (V1=C1*S1) obtained by multiplying the defect area S1 by the reliability C1, and an index value V2 (V2=C2*S2) obtained by multiplying the defect area S2 by the reliability C2. Specifically, as shown in FIG. 10, in the third case where the index value V1 is greater than the index value V2, the processor 210 determines the detection result of the partial defect FD1 as the priority detection result. In the fourth case where the index value V2 is greater than the index value V1, the processor 210 determines the detection result of the partial defect FD2 as the priority detection result.

[0095] If the index value V1 and the index value V2 are equal, i.e., if none of the first to fourth cases above apply, the processor 210 calculates a priority detection result based on the reliability. Specifically, as shown in Fig. 10, in the fifth case where the reliability C1 is greater than the reliability C2, the processor 210 determines the detection result of the partial defect FD1 as the priority detection result. In the sixth case where the reliability C1 is equal to or less than the reliability C2, the processor 210 determines the detection result of the partial defect FD2 as the priority detection result.

[0096] In S260B, the processor 210 determines the reliability and type of defect included in the priority detection result as the final detection result.

[0097] If the defect types in the two detection results are the same (S250B: NO), in S265B, processor 210 calculates the average value of the reliability of the two detection results as the reliability of the detection result of the defect spanning the two partial images. In the example of Fig. 4, the average value of the reliability C1 of the detection result of partial defect FD1 and the reliability C2 of the detection result of partial defect FD2 is calculated as the reliability of the detection result of defect FD.

[0098] In S270B, the processor 210 integrates the two defect masks by taking the logical sum of the defect masks of the two partial images. In the example of Fig. 4, the processor 210 generates one mask MSt by taking the logical sum of the mask MS1 of the partial defect FD1 and the mask MS2 of the partial defect FD2.

[0099] In S275B, the processor 210 uses the newly generated mask MSt to correct the detection result image DI (FIG. 4(E)) generated in S220. Specifically, the processor 210 places the mask MSt generated in S265 in the detection result image DI, replacing the mask MS1 of the partial defect FD1 and the mask MS2 of the partial defect FD2. Furthermore, the processor 210 places an image indicating a bounding box BBt (rectangular area) surrounding the mask MSt in the detection result image DI. As a result, the corrected detection result image DIt shown in FIG. 9(A) is generated.

[0100] In S280, similar to S280 in FIG. 5, processor 210 displays a screen showing the detection results (also referred to as a result display screen) on display unit 240. FIG. 9B shows an example of a result display screen in the second embodiment. The result display screen W2 in FIG. 9B is an example of a screen displayed when a defect is detected (YES in S230 in FIG. 8) and the defect straddles two partial images (YES in S240 in FIG. 8). The result display screen W2 in FIG. 8B includes a message MG1 notifying that a defect has been detected, an inspection image IMs, and a corrected detection result image DIt superimposed on the inspection image IMs. The result display screen W2 also includes text TXt indicating the type of the detected defect, the reliability of the detection, and the size of the defect (e.g., the length of a linear scratch).

[0101] 8 is executed, the defect type information included in the text TXt is information based on the defect type information included in the priority detection result, and the detection reliability included in the text TXt is the reliability included in the priority detection result. When S265B of Fig. 8 is executed, the defect type information included in the text TXt is information based on the defect type information included in the detection results of partial defect FD1 and partial defect FD2, and the detection reliability included in the text TXt is the average value calculated in S265B. The size of the defect included in the text TXt is calculated based on, for example, the size (number of pixels in the vertical or horizontal direction) of the mask MSt or the bounding box BBt included in the detection result image DIt.

[0102] The result display screen displayed when no defect is detected in the inspection image IMs (NO in S230 in FIG. 8), and the result display screen displayed when a defect is detected (YES in S230 in FIG. 8) and the defect does not straddle two partial images (NO in S240 in FIG. 8) are the same as those in the first embodiment.

[0103] According to the second embodiment described above, the processor 210 acquires a plurality of partial images PI including the partial image PIi and the partial image PIi+1 showing the cloth 700 (S110-S130 in FIG. 3, FIG. 4(C)). The processor 210 executes a detection process including a first process for detecting defects using the partial image PIi and a first process for detecting defects using the partial image PIi+1 (S210 in FIG. 8). When certain conditions are met, including that partial defect FD1 is detected in the first process using partial image PIi and partial defect FD2 is detected in the second process using partial image PIi (YES in S240 of FIG. 8), processor 210 uses the detection results of the first process using partial image PIi (e.g., mask MS1, reliability C1, defect type information) and the detection results of partial image PIi+1 (e.g., mask MS2, reliability C2, defect type information) to generate detection results (e.g., mask MSt, reliability, defect type information) for the portion of cloth 700 that includes first portion p1 and second portion p2 (FIG. 4(B)) (S250B-S275B of FIG. 8, FIGS. 9(A) and (B)).

[0104] When a defect is detected in both adjacent partial images PIi and PIi+1, a defect FD that straddles partial images PIi and PIi+1 may be detected as partial defect FD1 and partial defect FD2, as shown in the example of Fig. 4. In this case, because only a portion of the defect FD is included in both partial image PIi and partial image PIi+1, the first process using partial image PIi and the second process using partial image PIi+1 may not be able to correctly generate information indicating the defect area (mask or bounding box B) or defect type information. For example, the determination result of the defect type included in the detection result of the first process may differ from the determination result of the defect type included in the detection result of the second process. According to this embodiment, when specific conditions are met, including that partial defect FD1 is detected in a first process using partial image PIi and partial defect FD2 is detected in a second process using partial image PIi+1, the detection result of the first process (e.g., the detection result of partial defect FD1) and the detection result of the second process (e.g., the detection result of partial defect FD2) are used to generate a detection result of a portion including first portion p1 and second portion p2 (FIGS. 4(B) and 9(A)) of the cloth 700. As a result, a defect FD spanning from the first portion p1 to the second portion p2 of the cloth 700 can be appropriately detected using the multiple partial images PIi and PIi+1.

[0105] Specifically, if the types of defects in the two detection results are different (YES in S250B of FIG. 8), the processor 210 determines a priority detection result from the detection result of the first process (e.g., the detection result of partial defect FD1) and the detection result of the second process (e.g., the detection result of partial defect FD2) (S250 of FIG. 8). The processor 210 determines the reliability and defect type information of the priority detection result as the detection result of the defect FD that straddles the two partial images (S260B of FIG. 8).

[0106] Specifically, as described with reference to FIG. 10 , if defect area S1 > defect area S2 and reliability C1 > reliability C2, processor 210 determines the detection result of partial defect FD1 as the priority detection result. That is, if the defect area of ​​the detection result of the first process is larger than the defect area of ​​the detection result of the second process and the reliability of the detection result of the first process is higher than the detection result of the second process, processor 210 determines the detection result of the first process as the priority detection result. As a result, an appropriate priority detection result can be determined based on the area and reliability of the defect area of ​​the detection results of the two partial images. If one detection result detects a larger defect area than the other detection result and detects the defect area with higher reliability than the other detection result, it is considered that one detection result has a higher reliability than the other detection result after taking into account a larger portion of the defect. Therefore, in this case, one detection result is considered to be sufficiently more reliable than the other detection result. In this way, in this embodiment, the more reliable detection result of the partial defect FD1 and the detection result of the partial defect FD2 can be determined as the final detection result of the defect FD consisting of the partial defect FD1 and the partial defect FD2. Therefore, the detection accuracy of the defect type, etc. of the defect FD can be improved.

[0107] 10, in a case where neither the first case (defect area S1 > defect area S2 and reliability C1 > reliability C2) nor the second case (defect area S2 > defect area S1 and reliability C2 > reliability C1) applies, processor 210 determines the detection result of partial defect FD1 as the priority detection result if reliability C1 * defect area S1 > reliability C2 * defect area S2. That is, in this case, if the product of defect area S1 and reliability C1 of the detection result of the first process is greater than the product of defect area S2 and reliability C2 of the detection result of the second process, processor 210 determines the detection result of the first process as the priority detection result. As a result, even in a case where neither the first case nor the second case applies, the defect areas S1 and S2 and the reliabilities C1 and C2 can be used to determine an appropriate priority detection result.

[0108] Furthermore, in this embodiment, if the defect types in the two detection results are the same (NO in S250B of FIG. 8), processor 210 calculates the average of the reliability C1 of the detection result of the first process (detection result of partial defect FD1) and the reliability C2 of the detection result of the second process (detection result of partial defect FD2) as the reliability of the detection result of defect FD (S265B of FIG. 8). As a result, if the defect types in the two detection results are the same, it is possible to appropriately calculate the reliability of the detection of defect FD that straddles two partial images.

[0109] Furthermore, in this embodiment, the processor 210 identifies a mask MSt obtained by taking the logical sum of a mask MS1 representing the detection result of the first process and a mask MS2 representing the detection result of the second process as the mask representing the detection result of the defect FD (S270B in FIG. 8). As a result, the area in which the defect FD is located that straddles the two partial images can be appropriately identified.

[0110] Furthermore, in this embodiment, the processor 210 determines the reliability and defect type information in the priority detection result as the detection result of the defect FD (S260B in FIG. 8). As a result, it is possible to appropriately determine the defect type and detection reliability of the defect FD that straddles two partial images.

[0111] C. Third Example In the third embodiment, the content of the defect detection process differs from that in the first embodiment. Other configurations of the third embodiment are the same as those in the first embodiment. FIG. 11 is a flowchart of the defect detection process of the third embodiment. Note that in the flowchart of FIG. 11, steps that are the same as those in the flowchart of FIG. 5 are assigned the same reference numerals as in FIG. 5, and steps that differ from those in the flowchart of FIG. 5 are assigned the suffix "C" to the reference numeral. Below, we will explain the parts of the defect detection process of FIG. 11 that are different from the defect detection process of FIG. 5.

[0112] As shown in Fig. 11, in the defect detection process of the third embodiment, S250C-S270C in Fig. 11 are executed instead of S250-S270 in Fig. 5. That is, in the third embodiment, when a detected defect straddles two partial images, the process executed is different from that in the first embodiment.

[0113] In S250C, the processor 210 determines whether the defect area S1 of the partial defect FD1 is greater than the defect area S2 of the partial defect FD2, and whether the reliability C1 included in the detection result of the partial defect FD1 is greater than the reliability C2 included in the detection result of the partial defect FD2.

[0114] If the defect area S1 is larger than the defect area S2 and the reliability C1 is larger than the reliability C2 (S250C: YES), the processor 210 proceeds to S270C. If the defect area S1 is equal to or smaller than the defect area S2 or the reliability C1 is equal to or smaller than the reliability C2 (S250C: NO), the processor 210 determines in S255C whether the defect area S2 is larger than the defect area S1 and the reliability C2 is larger than the reliability C1.

[0115] If the defect area S2 is larger than the defect area S1 and the reliability C2 is larger than the reliability C1 (S255C: YES), the processor 210 proceeds to S270C. If the defect area S2 is equal to or smaller than the defect area S1 or the reliability C2 is equal to or smaller than the reliability C1 (S255C: NO), the processor 210 proceeds to S260C.

[0116] In S260C, processor 210 executes S250-S270 of Fig. 5. That is, as described in the first embodiment, processor 210 generates a new partial image N I including a defect FD that straddles two partial images (S250 of Fig. 5), inputs the new partial image N I to the object detection model MD, and detects the defect FD included in the partial image N I (S260 of Fig. 5). Then, processor 210 corrects the detection result image D I generated in S220 using a mask image N M including a mask MS f of the defect FD included in the detection result (S270 of Fig. 5).

[0117] In S270C, processor 210 executes S250B-S275B of FIG. 8. That is, as described in the second embodiment, when the defect types in the defect detection results of the two partial images are different (S250B: YES in FIG. 8), processor 210 determines a prioritized detection result from the two detection results (S255B in FIG. 8), and determines the reliability and defect type of the prioritized detection result as the final detection result (S260B in FIG. 8). Note that in S255B, as shown in FIG. 10, when defect area S1 is larger than defect area S2 and reliability C1 is larger than reliability C2 (S250C: YES), the detection result of partial defect FD1 in partial image PIi is determined as the prioritized detection result. Also, when defect area S2 is larger than defect area S1 and reliability C2 is larger than reliability C1 (S255C: YES), the detection result of partial defect FD2 in partial image PIi+1 is determined as the prioritized detection result. If the defect types in the defect detection results of the two partial images are the same (S250B: NO in FIG. 8), processor 210 calculates the average value of the reliability of the two detection results as the final reliability (S265B in FIG. 8). Then, processor 210 generates a sample mask MSt by taking the logical sum of the masks of defects FD1 and FD2 in the two partial images (S270B in FIG. 8), and uses this mask MSt to correct the detection result image DI generated in S220 (S275B in FIG. 8).

[0118] According to the third embodiment described above, the processor 210 performs the same processing as in the second embodiment in the first case where the defect area S1>the defect area S2 and the reliability C1>the reliability C2 (YES in S250C of FIG. 11) and the second case where the defect area S2>the defect area S1 and the reliability C2>the reliability C1 (YES in S255C of FIG. 11). That is, in the first case, the processor 210 determines the detection result of the partial defect FD1 as the prioritized detection result (S265C of FIG. 11, S255B of FIG. 8, FIG. 10), and in the second case, determines the detection result of the partial defect FD2 as the prioritized detection result (S265C of FIG. 11, S255B of FIG. 8, FIG. 10). Then, in the case where the result is different from the first case and the second case (NO in S255C of FIG. 11), the processor 210 performs the same processing as in the first embodiment without determining the prioritized detection result. That is, in this case, the processor 210 generates a partial image NI that includes the portion of the partial image PIi that includes the partial defect FD1 and the portion of the partial image PIi+1 that includes the partial defect FD2 (S250 in FIG. 5), and detects the defect FD using the new partial image NI (S260 in FIG. 5).

[0119] In the first case, the detection result of defect FD1 in partial image PIi is considered to be sufficiently reliable, and in the second case, the detection result of defect FD2 in partial image PIi+1 is considered to be sufficiently reliable. For this reason, in the first and second cases, it is preferable to use the priority detection result as the final detection result. When the result is different from either the first or second case, it is considered that the detection results of partial defect FD1 and partial defect FD2 are less reliable than in the first or second case. For this reason, in this case, it is considered preferable to perform detection again using partial image NI including partial defect FD1 and partial defect FD2, i.e., partial image NI including the entire defect FD. As described above, in this embodiment, appropriate processing can be used depending on the detection results of partial defect FD1 and partial defect FD2, thereby achieving efficient and reliable defect detection processing.

[0120] D. Fourth Example In the fourth embodiment, the content of the defect detection process differs from that of the first embodiment. Other configurations of the fourth embodiment are similar to those of the first embodiment. FIG. 12 is a flowchart of the defect detection process of the fourth embodiment. In the flowchart of FIG. 12, steps that are the same as those in the flowchart of FIG. 5 are given the same reference numerals as in FIG. 5, and steps that differ from those in the flowchart of FIG. 5 are given the suffix "D" to the reference numeral. The following describes the parts of the defect detection process of FIG. 12 that are different from the defect detection process of FIG. 5.

[0121] As shown in Fig. 12, in the defect detection process of the fourth embodiment, steps S250D to S275D of Fig. 12 are executed instead of steps S250 to S270 of Fig. 5. That is, in the fourth embodiment, when a detected defect straddles two partial images, the process executed differs from that of the first embodiment. In the fourth embodiment, when a detected defect straddles two partial images (S240: YES), the process of S250D and the process of S255D are started in parallel.

[0122] In S250D, processor 210 executes S250-S260 in Fig. 5. That is, as described in the first embodiment, processor 210 generates a new partial image N I including a defect FD that straddles two partial images (S250 in Fig. 5), and inputs the new partial image N I to object detection model MD to detect defect FD included in partial image N I (S260 in Fig. 5).

[0123] In S255D, processor 210 executes S250B-S275B of Fig. 8. That is, as described in the second embodiment, if the defect types in the defect detection results of the two partial images are different (S250B: YES in Fig. 8), processor 210 determines a prioritized detection result from the two detection results (S255B in Fig. 8), and determines the reliability and defect type of the prioritized detection result as the final detection result (S260B in Fig. 8). If the defect types in the defect detection results of the two partial images are the same (S250B: NO in Fig. 8), processor 210 calculates the average value of the reliability of the two detection results as the final reliability (S265B in Fig. 8). Then, the processor 210 generates a sample mask MSt by taking the logical sum of the masks of defects FD1 and FD2 in the two partial images (S270B in FIG. 8), and uses the mask MSt to correct the detection result image DI generated in S220 to generate a corrected detection result image DIt (S275B in FIG. 8).

[0124] The processing of S250D includes processing using the object detection model MD, which is a machine learning model (S260 in FIG. 5). For this reason, the processing time of S250D is longer than the processing time of S255D, and therefore the processing of S255D ends before the processing of S250D. When the processing of S255D ends, in parallel with the processing of S250D that is currently being performed, the processor 210 displays a result display screen on the display unit 240 in S265D using the detection result image DIt generated in the processing of S255D. Since the processing of S255D is the same as that of the second embodiment, the screen displayed on the display unit 240 is, for example, the result display screen W2 of the second embodiment (FIG. 9(B)).

[0125] When the processing of S250D ends after the result display screen W2 is displayed on the display unit 240, the processor 210 determines whether the detection result generated in the processing of S255D is different from the detection result generated in the processing of S250D. For example, if at least one of the type of defect FD and the detection certainty based on the detection result generated in the processing of S255D is different from at least one of the type of defect FD and the detection certainty based on the detection result generated in the processing of S250D, the two detection results are determined to be different. Alternatively, for example, the detection certainty may not be taken into consideration, and the two detection results may be determined to be different if the type of defect FD based on the detection result generated in the processing of S255D is different from the type of defect FD based on the detection result generated in the processing of S250D.

[0126] If the detection result generated in the processing of S255D and the detection result generated in the processing of S250D differ (S270D: YES), in S275D, processor 210 corrects the detection result image DI generated in S220 using the mask image newly generated in S250D of Fig. 12, similar to S270 of Fig. 5. That is, similar to the first embodiment, a corrected detection result image DIf of Fig. 6(C) is generated, which indicates the detection result generated by inputting the new partial image NI to the object detection model MD.

[0127] In S280D, processor 210 displays a result display screen showing the detection results on display unit 240. If the defect is a defect that straddles two partial images (YES in S240), processor 210 uses detection result image DIf to generate a result display screen showing the detection results of the processing of S250D, for example, result display screen W1 of FIG. 7, and displays it on display unit 240. In this case, in S265D, result display screen W2 (FIG. 9(B)) showing the detection results of the processing of S255D has already been displayed on display unit 240. For this reason, processor 210 displays result display screen W1 on display unit 240, for example, instead of result display screen W2. Note that if the defect is not a defect that straddles two partial images (NO in S240), processor 210 uses detection result image DI generated in S220 to display a result display screen (not shown) showing the detection results of S210 on display unit 240.

[0128] According to the fourth embodiment described above, when a defect FD straddles two partial images (YES in S240), the processor 210 generates a new partial image N I including the defect FD straddling two partial images, as in the first embodiment, and inputs the new partial image N I to the object detection model MD to detect the defect FD included in the partial image N I (S250D in FIG. 12 ). Furthermore, in parallel, the processor 210 generates a detection result for the defect FD straddling two partial images using the detection results for the two partial defects FD1 and FD2, as in the second embodiment (S255D in FIG. 12 ). Then, before completing the process of S250D in FIG. 12 , the processor 210 displays the detection result obtained by the process of S255D in FIG. 12 on the display unit 240 (S265D in FIG. 12 ). After completing the process of S250D, the processor 210 displays the detection result obtained by the process of S250D on the display unit 240 instead of the detection result obtained by the process of S255D in FIG. 12 .

[0129] The method of the first embodiment, i.e., a method of generating a new partial image NI including a defect FD that straddles two partial images and inputting the new partial image NI into the object detection model MD to detect the defect FD included in the partial image NI, takes into account the entire defect FD, but requires re-executing processing using the object detection model MD. For this reason, the method of the first embodiment can accurately determine the type of defect FD, etc., but tends to require a long processing time. The method of the second embodiment, i.e., a method of generating a defect FD detection result using the detection results of partial defects FD1 and FD2, is based on a detection result obtained by considering only the partial defects, but does not require re-executing processing using the object detection model MD. For this reason, the method of the second embodiment may be less accurate in determining the type of defect FD, etc., than the method of the first embodiment, but requires a shorter processing time than the method of the first embodiment. According to this embodiment, the detection result of the defect FD can be displayed quickly, and if the previously displayed detection result is incorrect, the correct detection result can be displayed subsequently. As a result, it is possible to achieve both rapid display of detection results and accurate display of detection results.

[0130] E. Variations (1) Fig. 13 is a flowchart of a defect detection process according to a modified example. In the flowchart of Fig. 13, processing of S245E is added between S240 and S250. Other configurations of the flowchart of Fig. 13 are the same as those of the flowchart of Fig. 5.

[0131] In S245E, processor 210 determines whether the types of the two defects in the defect detection results for the two partial images are different. In the example of Fig. 4, it determines whether the type of partial defect FD1 in the detection result for partial image PIi is different from the type of partial defect FD2 in the detection result for partial image PIi+1.

[0132] If the types of the two defects are different (S245E: YES), processor 210 executes the processes of S250-S270. If the types of the two defects are the same (S245E: NO), processor 210 does not execute the processes of S250-S270.

[0133] In the first embodiment of FIG. 5, the specific condition for determining whether to execute the processes of S250-S270 is only the above-mentioned condition 1-3 for determining whether the detected defect is a defect that straddles two partial images (S240). In contrast, in the modified example of FIG. 13, the specific condition includes, in addition to conditions 1-3, that the defect types differ between the detection results of partial images PIi and PIi+1. Thus, in this embodiment, if the defect types are the same between the detection results of partial images PIi and PIi+1, the detection results of partial images PIi and PIi+1 are used as is. As a result, the processing load of the defect detection process can be further reduced.

[0134] The condition of S245E can be modified in various ways. For example, in each of the above embodiments, the condition of S245E may be that the reliability differs between the two detection results, or that the masks differ between the two detection results. For example, if the entire defect FD is included in the overlapping area IAi between the partial images PIi and PIi+1, the detection results of the partial images PIi and PIi+1 are considered to be substantially the same. In such a case, it is considered unnecessary to execute the processes of S250-S270.

[0135] (2) In each of the above embodiments, the process of detecting defects contained in each partial image PI using each partial image PI is a process using an object detection model MD. The defect detection method is not limited to this, and other methods may be used. For example, defect detection may be performed based on an anomaly detection mechanism using a machine learning model called PaDiM (a Patch Distribution Modeling Framework for Anomaly Detection and Localization). Furthermore, defect detection may be performed based on a method using an autoencoder, which is a machine learning model including an encoder and a decoder. In this method, for example, the partial image PI is input to the autoencoder, and an output image corresponding to the partial image PI is generated. Then, the partial image PI is compared with the output image to detect defects contained in the partial image PI.

[0136] Furthermore, the defect detection method may be a method that does not use a machine learning model. In this case, defects contained in the partial image PI are detected by analyzing the partial image PI using an analysis process that does not use a machine learning model. For example, defects contained in the partial image PI may be detected by searching the partial image PI for defects of a predetermined shape, such as linear scratches, using pattern matching.

[0137] (3) In S250 of Fig. 5, the processor 210 generates a new partial image NI by cutting out a portion including a defect FD consisting of partial defect FD1 in partial image PIi and partial defect FD2 in partial image PIi+1 from an inspection image IMs obtained by previously combining captured images IM1-IM4 (Fig. 4(B)). Without being limited to this, for example, the processor 210 may generate the partial image NI including the defect FD by combining partial images PIi and PIi+1 (Fig. 4(C)). Alternatively, the processor 210 may generate the partial image NI by cutting out a portion including the defect FD from a captured image IM3 including the defect FD.

[0138] In addition, if the entire defect FD is larger than the input size of the object detection model MD, the processor 210 may generate a partial image NI to input to the object detection model MD by reducing a partial image including the entire defect FD to the input size.

[0139] Furthermore, the partial image N1 used in S260 of FIG. 5 may be generated before the defect detection process (S140 of FIG. 5 and FIG. 3). For example, between S130 and S140 of FIG. 3, the processor 210 may generate and store in the non-volatile storage device 230 a plurality of other partial images N1 including overlapping regions between two adjacent partial images among the plurality of partial images P1 of FIG. 4C. For example, a partial image N1 including an overlapping region IA1 between partial images P11 and P12 of FIG. 4C in the center in the first direction Dx is generated. Similarly, a partial image N1 including an overlapping region IAj between partial images P1j and P1j+1 in the center in the first direction Dx is generated (j is an integer greater than or equal to 2 and less than or equal to (k-1)). That is, for k partial images P1, (k-1) other partial images N1 are generated in advance. These (k-1) separate partial images N1 may be generated by combining two adjacent partial images P1 of the k partial images P1, or may be cut out from the inspection images M1. The processor 210 may identify one partial image N1 that includes the entire defect FD determined to straddle two partial images P1 from the (k-1) separate partial images N1, and perform the process of S260 in FIG. 5 using the identified partial image N1.

[0140] (4) In each of the above embodiments, the processor 210 divides the inspection image IMs obtained by previously combining the captured images IM1-IM4 to generate multiple partial images PI having the input size of the object detection model MD. Alternatively, the size of the captured images IM1-IM4 may be matched to the input size of the object detection model MD in advance. In this case, the processor 210 may input each of the captured images IM1-IM4 to the object detection model MD and detect defects in each captured image.

[0141] (5) In each of the above embodiments, when the inspection image IMs is divided to generate a plurality of partial images PI, an overlapping area (e.g., IA1, IAi in FIG. 4C) is provided between adjacent partial images PI. Alternatively, there may be no overlapping area between adjacent partial images PI. In this case, among the above-described conditions 1-3 for determining whether a detected defect is a defect that straddles two partial images, condition 3 may be that the mask MS1 of partial image PIi and the mask MS2 of partial image PIi+1 are adjacent in the detection result image DI.

[0142] (6) In the above embodiment, the partial image NI is generated so that the defect FD is located in the center (FIG. 6A). However, the defect FD may be located at the edge of the partial image NI.

[0143] (7) In the second embodiment described above, a priority detection result is determined from two detection results of partial defects FD1 and FD2 based on both the defect areas S1 and S2 and the reliability C1 and C2 (FIG. 10). Alternatively, for example, processor 210 may use only defect areas S1 and S2 to determine the detection result with the larger defect area as the priority detection result. Alternatively, processor 210 may use only reliability C1 and C2 to determine the detection result with the larger reliability as the priority detection result.

[0144] (8) In the second embodiment, both the reliability of the priority detection result and the type of defect are used as the final detection result of the defect FD. Alternatively, only the type of defect in the priority detection result may be used as the final detection result of the defect FD. In this case, the reliability of the final detection result of the defect FD may be, for example, the average value of the reliability of the two detection results of the partial defects FD1 and FD2.

[0145] (9) The object detection model MD used in the special detection process of the above embodiment is a machine learning model called RTMDet that realizes instance segmentation, but may be another machine learning model. For example, the object detection model MD may be a machine learning model that realizes other instance segmentation, such as Mask R-CNN, or a machine learning model that realizes semantic segmentation, such as YOLO (You only look once).

[0146] Furthermore, the object detection model MD in the above embodiment is trained to determine multiple types of defects. Alternatively, the object detection model MD may be trained to detect only one type of defect. In other words, the object detection model MD may be trained to determine only the presence or absence of a defect, without determining the type of defect.

[0147] (10) In the above embodiment, the object to be inspected is cloth 700, but it may be another object. The other object may be, for example, a sheet-like object, such as a metal foil, a resin film, or paper. The object to be inspected may also be an object having a shape other than a sheet, such as various products or parts thereof, such as automobiles or electrical appliances. Depending on the object to be inspected, the defects to be detected may take various forms.

[0148] (11) The total number of digital cameras used to photograph the cloth 700 is not limited to four and may be any number greater than or equal to one. Furthermore, the device used to photograph the cloth 700 may include a line sensor as an image sensor instead of an area sensor such as a digital camera. In either case, the processor 210 uses the target images acquired using these devices to detect defects in the cloth 700. For example, if there is one digital camera, the captured image generated by the single digital camera may be used as the inspection image IMs as is.

[0149] (12) In the above embodiment, the processor 210 may cause the GPU 260 to execute various calculations. For example, the processor 210 may cause the GPU 260 to execute some or all of the calculations based on the object detection model MD. Note that the GPU 260 may be omitted.

[0150] (13) Data processing device 200 in Fig. 1 may be a device of a type different from a personal computer (e.g., a digital camera, a scanner, or a smartphone). Furthermore, multiple devices (e.g., computers) that can communicate with each other via a network may share some of the data processing functions of the data processing device and collectively provide the data processing functions (a system including these devices corresponds to a data processing device).

[0151] (14) In each of the above embodiments, a part of the hardware configuration may be replaced with software, and conversely, a part or all of the software configuration may be replaced with hardware. For example, the processing by the object detection model MD (FIG. 2) may be performed by a dedicated hardware circuit such as an Application Specific Integrated Circuit (ASIC).

[0152] Furthermore, when some or all of the functions of the present disclosure are realized by a computer program, the program can be provided in a form stored on a computer-readable recording medium (e.g., a non-transitory recording medium). The program can be used in a state stored on the same or a different recording medium (computer-readable recording medium) from when it was provided. The "computer-readable recording medium" is not limited to portable recording media such as memory cards and CD-ROMs, but can also include internal storage devices within a computer, such as various ROMs, and external storage devices connected to a computer, such as a hard disk drive.

[0153] The above-described examples and modifications can be combined as appropriate. The above-described examples and modifications are provided to facilitate understanding of the present disclosure and are not intended to limit the present invention. The present invention may be modified or improved without departing from the spirit thereof, and the present invention includes equivalents thereof. [Explanation of symbols]

[0154] 1000... inspection system, 10... inspection device, 111-114... digital camera, 120... rotary encoder, 130... light source, 200... data processing device, 210... processor, 215... storage device, 220... volatile storage device, 230... non-volatile storage device, 240... display unit, 250... operation unit, 270... communication interface, 31, 32... core material, 700... cloth, 710... upstream roll, 720... downstream roll, 900... conveying device, 950... conveying mechanism, 980... operation unit, 990... control unit, DI, DIf, DIt... detection result image, Df... conveying direction, FD... defect, FD1, FD2... partial defect, IM1-IM4... captured image, IMs... inspection image, MD... object detection model, MI1-MIk... mask image, PG... computer program, PI... partial image

Claims

1. A computer program comprising: a first acquisition function that acquires a first object image and a second object image showing an object, the first object image and the second object image being images generated using an image sensor, the first object image including an image of a first portion of the object and not including an image of a second portion adjacent to the first portion of the object, and the second object image including an image of the second portion and not including at least a portion of the image of the first portion; a first detection function that executes a first process of detecting defects in the object shown in the first object image using the first object image, and a second process of detecting defects in the object shown in the second object image using the second object image; a second acquisition function that acquires a third object image including a portion of the first object image that includes the first defect and a portion of the second object image that includes the second defect when a specific condition is satisfied, the third object image including a portion of the first object image that includes the first defect and a portion of the second object image that includes the second defect; a second detection function that executes a third process using the third object image to detect defects in the object shown in the third object image when the specific condition is satisfied; A computer program that enables a computer to realize the above.

2. 2. The computer program of claim 1, The computer program, wherein the first process, the second process, and the third process include processes using a trained machine learning model.

3. 3. A computer program according to claim 2, comprising: the detection results of the first process and the second process include at least one of area information indicating an area of ​​a defect, a reliability that is an index value indicating the reliability of detection, and type information indicating the type of the defect; The specific condition includes that at least one of the area information, the reliability, and the type information is different between the detection result of the first process and the detection result of the second process.

4. 3. A computer program according to claim 2, comprising: The second acquisition function acquires the third target image arranged so that the first defect and the second defect are located at the center.

5. 3. A computer program according to claim 2, comprising: a computer program, wherein the machine learning model used in the first process and the machine learning model used in the second process are the same model;

6. 2. The computer program of claim 1, The specific condition includes a condition indicating that the first defect and the second defect constitute one continuous defect.

7. A computer program comprising: a first acquisition function that acquires a first object image and a second object image showing an object, the first object image and the second object image being images generated using an image sensor, the first object image including an image of a first portion of the object and not including an image of a second portion adjacent to the first portion of the object, and the second object image including an image of the second portion and not including at least a portion of the image of the first portion; a first detection function that executes a first process that uses the first object image to detect defects in the object shown in the first object image, and a second process that uses the second object image to detect defects in the object shown in the second object image, wherein the detection results of the first process and the second process include information indicating at least one of a detected defect area and a reliability of detection of the defect area; a result generation function that, when a specific condition is satisfied, including that a first defect is detected in the first process and a second defect is detected in the second process, generates the detection result of a specific portion including the first portion and the second portion of the object using the detection result of the first process and the detection result of the second process; A computer program comprising:

8. 8. A computer program according to claim 7, comprising: the specific condition includes that at least a part of the detection result of the first process and a part of the detection result of the second process are different from each other; The result generation function comprises: determining a priority detection result between the detection result of the first processing and the detection result of the second processing that satisfies at least one of the following conditions: that the defective area is detected in a larger area than the other detection result; and that the defective area is detected with a higher reliability than the other detection result; A computer program product that generates a detection result of the specific part by determining at least a part of the priority detection result as a detection result of the specific part.

9. 8. A computer program according to claim 7, comprising: the specific condition includes that the type of defect detected in the first process is the same as the type of defect detected in the second process; The result generation function executes a generation process including a process of calculating the average value of the reliability of the detection result of the first process and the reliability of the detection result of the second process as the reliability of the detection result of the specific part, thereby generating a detection result of the specific part.

10. A computer program according to any one of claims 7 to 9, The result generation function comprises: A computer program that generates a detection result of the specific part by executing a generation process that includes a process of identifying an area obtained by taking the logical OR of the defective area of ​​the detection result of the first process and the defective area of ​​the detection result of the second process as a defective area of ​​the detection result of the specific part.

11. 9. A computer program according to claim 8, comprising: the detection results of the first process and the second process include information indicating at least one of the defect area, the reliability, and the type of the detected defect; A computer program in which the result generation function generates a detection result for the specific part by determining at least one of the defect area, the reliability, and the defect type in the priority detection result as the detection result for the specific part.

12. 9. A computer program according to claim 8, comprising: The result generation function comprises: A computer program that determines the detection result of the first processing as the priority detection result when the defect area of ​​the detection result of the first processing is larger than the defect area of ​​the detection result of the second processing and the reliability of the detection result of the first processing is higher than the detection result of the second processing.

13. 9. A computer program according to claim 8, comprising: The result generation function comprises: A computer program that determines the detection result of the first processing as the priority detection result when the product of the area of ​​the defective area and the reliability of the detection result of the first processing is greater than the product of the area of ​​the defective area and the reliability of the detection result of the second processing.

14. 9. A computer program according to claim 8, comprising: The result generation function comprises: determining the detection result of the first processing as the priority detection result in a first case where the defect area of ​​the detection result of the first processing is larger than the defect area of ​​the detection result of the second processing and the reliability of the detection result of the first processing is higher than the reliability of the detection result of the second processing; determining the detection result of the second processing as the priority detection result in a second case where the defect area of ​​the detection result of the second processing is larger than the defect area of ​​the detection result of the first processing and the reliability of the detection result of the second processing is higher than the reliability of the detection result of the first processing; If the result is different from either the first case or the second case, the priority detection result is not determined; The computer program further comprises: a second acquisition function that acquires a third object image including a portion of the first object image that includes the first defect and a portion of the second object image that includes the second defect when the specific condition is satisfied and the condition is different from the first case and the second case; a second detection function that executes a third process using the third object image to detect defects in the object shown in the third object image when the specific condition is satisfied and the condition is different from either the first case or the second case; A computer program that enables a computer to realize the above.

15. 8. The computer program of claim 7, further comprising: a second acquisition function that acquires, when the specific condition is satisfied, a third object image including a portion of the first object image that includes the first defect and a portion of the second object image that includes the second defect; a second detection function that executes a third process using the third object image to detect defects in the object shown in the third object image when the specific condition is satisfied; a display control function that displays the detection result of the specific portion generated by the result generating function on a display unit before the third processing by the second detection function is completed, and displays the detection result of the third processing on the display unit instead of the detection result of the specific portion after the third processing by the second detection function is completed; A computer program that enables a computer to realize the above.

16. 8. A computer program according to claim 7, comprising: The computer program, wherein the first process and the second process include a process using a trained machine learning model.

17. 8. A computer program according to claim 7, comprising: The specific condition includes a condition indicating that the first defect and the second defect constitute one continuous defect.

18. 1. A data processing device, comprising: a first acquisition unit that acquires a first object image and a second object image showing an object, the first object image and the second object image being images generated using an image sensor, the first object image including an image of a first portion of the object and not including an image of a second portion adjacent to the first portion of the object, and the second object image including an image of the second portion and not including at least a portion of the image of the first portion; a first detection unit that executes a first process of detecting defects in the object shown in the first object image using the first object image, and a second process of detecting defects in the object shown in the second object image using the second object image; a second acquisition unit that acquires a third object image including a portion of the first object image that includes the first defect and a portion of the second object image that includes the second defect when a specific condition is satisfied, the third object image including a portion of the first object image that includes the first defect and a portion of the second object image that includes the second defect; a second detection unit that executes a third process using the third object image to detect defects in the object shown in the third object image when the specific condition is satisfied; A data processing device comprising:

19. 1. A data processing device, comprising: a first acquisition unit that acquires a first object image and a second object image showing an object, the first object image and the second object image being images generated using an image sensor, the first object image including an image of a first portion of the object and not including an image of a second portion adjacent to the first portion of the object, and the second object image including an image of the second portion and not including at least a portion of the image of the first portion; a first detection unit that executes a first process that uses the first object image to detect defects in the object shown in the first object image, and a second process that uses the second object image to detect defects in the object shown in the second object image, wherein detection results of the first process and the second process include information indicating at least one of a detected defect area and a reliability of detection of the defect area; a result generation unit that generates the detection result of a specific portion including the first portion and the second portion of the object using the detection result of the first processing and the detection result of the second processing when a specific condition is satisfied, the specific condition including that a first defect is detected in the first processing and that a second defect is detected in the second processing; A data processing device comprising:

Images