Inspection of sheet goods using deep learning

Abstract

An inspection system includes an inspection device having at least one image capture device. The image capture device captures image data of a sheet material component passing through the inspection device. A processing unit of the inspection device provides the image data representing the sheet material component to a plurality of neural networks, where each of the neural networks is trained to identify a corresponding defect in the sheet material component and outputs data indicative of a presence of the corresponding defect. The processing unit determines a quality class of the sheet material component based on the data indicative of the presence of the corresponding defect output by each corresponding neural network. The processing unit can also output the quality class of the sheet material component to a sorter, which can sort the sheet material component based on the quality class.

Description

Background Technology

[0001] Sheet components, which serve as parts for various manufactured articles, are typically made from various types of films (e.g., transparent polyester films). The manufacturing processes used to prepare these films generally involve producing films in the form of long, continuous sheets (referred to as webs). A web itself is typically a material having a fixed width in one direction (“transverse direction”) and a predetermined or indefinite length in the orthogonal direction (“coaxial direction”). In the various manufacturing processes used to prepare and handle webs, the web is conveyed along a longitudinal axis extending parallel to its length dimension and perpendicular to its width dimension.

[0002] Optical films are one example of manufactured films and are applied to a wide variety of consumer products. For example, sheet components derived from optical films may be located in screen assemblies of electronic devices such as mobile phones, televisions, laptops, desktop computers, or tablets. Manufacturing facilities may produce multiple similar sheet components (e.g., optical film sheet components) intended for use in specific consumer products. Typically, it is necessary to inspect the sheet components produced by the manufacturing facility for quality defects so that sheet components identified as defective can be discarded. In some examples, the sheet components are inspected by employees of the manufacturing facility. Additionally or alternatively, image processing techniques configured to automatically identify defects may be used to inspect the sheet components. Summary of the Invention

[0003] Generally, this disclosure describes techniques for inspecting sheet components for defects. More specifically, this disclosure describes exemplary techniques for detecting defects within a sheet component based on image data of the sheet component.

[0004] As described herein, as a component moves through an inspection apparatus, one or more image capture devices capture image data of the sheet component. A processing unit of the inspection apparatus receives the image data, preprocesses the image data, and feeds the image data to a set of convolutional neural networks (CNNs), where each CNN can be uniquely trained to detect different types of defects in the component. The output of each CNN may include data indicating the presence of defects detected by the CNN in the image data, and if present, may include the location of the defects spatially related to the sheet component. A classifier of the inspection apparatus is configured to receive output data from each CNN and classify the sheet component into quality categories based, for example, defects detected by the CNN based on the image data. In some aspects, the inspection apparatus may be configured with data defining one or more rules, each of which specifies a threshold level for each type of defect allowed for various quality categories. The classifier may determine which quality category to assign a given sheet component to by applying the rules to the output of the set of CNNs based on the different types of defects detected by the CNNs. In some aspects, the set of rules may be defined to classify the sheet component based on various combinations and / or correlations of the number of defects detected by the CNNs in the image data for different types of defects. In some respects, the inspection apparatus may provide a multi-level machine learning architecture in which a second set of one or more neural networks are trained to assign quality categories to sheet parts based on the outputs of a first set of CNNs, the outputs of which indicate the number of defects in the sheet parts for various different types of defects.

[0005] In the various examples described herein, the inspection apparatus can take one or more actions based on determinations made by a trained CNN. For example, quality categorization can be used to determine whether a sheet component is acceptable, defective, or requires rework. Furthermore, quality categorization can be used to sort components to suit different customers, some requiring very high quality while others are willing to accept slightly lower quality. Quality categorization can also be used to sort sheet components into bins corresponding to their quality categories.

[0006] The technology disclosed herein provides at least one technical advantage. For example, the inspection apparatus is capable of detecting defects with low contrast and / or low frequency relative to sheet components, defects that are undetectable in current sheet article inspection systems. A practical application of the technology described herein is an inspection apparatus capable of classifying low-contrast defects and low-frequency defects in sheet components.

[0007] In one embodiment, a system for determining the quality category of each of a plurality of sheet components produced by a manufacturing facility includes: an inspection device including at least one image capturing device configured to capture image data representing the sheet components among the plurality of sheet components; and a processing unit having one or more processors that execute instructions causing the processing unit to: provide the image data representing the sheet component to a first set of a plurality of neural networks, each of which is trained to identify a corresponding defect in the sheet component and output data indicating the presence of the corresponding defect; determine data indicating the quality category of the sheet component based on the data indicating the presence of the corresponding defect output by each corresponding neural network; and output the data indicating the quality category of the sheet component.

[0008] In another embodiment, the inspection apparatus for determining the quality category of a web material produced by a manufacturing facility includes: at least one image capturing device configured to capture image data representing the web material; a position controller configured to obtain position data along the web material; and a processing unit configured to: provide the image data representing the web material to each of a plurality of neural networks, each of the neural networks being trained to identify a corresponding defect in the sheet component and output data indicating the presence of the corresponding defect, and output the data indicating the presence of the corresponding defect and position data associated with the data indicating the presence of the corresponding defect.

[0009] In another embodiment, the method includes: receiving image data representing sheet components among a plurality of sheet components passed through an inspection device from at least one image capturing device; providing the image data representing the sheet component to a plurality of neural networks, each of which is trained to identify a corresponding defect in the sheet component based on the image data representing the sheet component and output data indicating the presence of the corresponding defect; determining data indicating a quality category of the sheet component based on the data indicating the presence of the corresponding defect output by each corresponding neural network; and outputting the data indicating the quality category of the sheet component.

[0010] Details of at least one example of this disclosure are set forth in the accompanying drawings and the following description. Other features, objectives, and advantages of this disclosure will become apparent from the specification, drawings, and claims. Attached Figure Description

[0011] Figure 1This is a block diagram illustrating a system for manufacturing multiple sheet components and for performing defect imaging and detection on the multiple sheet components according to at least one exemplary technique described in this disclosure.

[0012] Figure 2 This illustrates at least one exemplary technology according to the present disclosure. Figure 1 A block diagram showing additional details of the system.

[0013] Figure 3 This illustrates at least one exemplary technology according to the present disclosure. Figure 1 A block diagram of the top perspective view of an exemplary imaging unit of the system.

[0014] Figure 4 This is a block diagram illustrating an exemplary processing unit according to at least one exemplary technology described in this disclosure.

[0015] Figures 5A to 5E This is a block diagram illustrating an exemplary configuration of a machine learning component of a sheet component inspection system according to at least one exemplary technique described in this disclosure.

[0016] Figure 6A It is a block diagram of a training environment including a machine learning engine for generating a machine learning model that associates sheet component defects with process variables in at least one exemplary technique described in this disclosure.

[0017] Figure 6B It is based on at least one exemplary technique described in this disclosure, which uses Figure 6A A flowchart of the sheet component manufacturing environment using machine learning models to determine the root cause of defects in sheet components.

[0018] Figure 7 This is a flowchart illustrating exemplary operation of a method for determining the quality of a sheet component according to at least one exemplary technique described in this disclosure.

[0019] Figure 8A and Figure 8B An exemplary sheet component defect that can be identified according to at least one exemplary technique described in this disclosure is shown.

[0020] Figure 9 This is a block diagram illustrating an exemplary implementation of a production line in an exemplary embodiment of a web fabrication plant. Detailed Implementation

[0021] Systems and techniques for automatically determining the quality of sheet components formed by a manufacturing facility are described, enabling actions (e.g., sorting) to be taken based on quality for multiple sheet components, even in high-speed production environments. Accuracy is critical when determining the quality of multiple sheet components, as incorrect quality determination can lead to the rejection of high-quality sheet components or the shipment of low-quality sheet components for consumption. Therefore, a well-performing inspection system with the advantages of the techniques discussed herein can improve the yield and efficiency of a manufacturing facility and enhance the quality of sheet components produced by that facility. The techniques described herein enable efficient and accurate determination of the quality category of sheet components.

[0022] Figure 1 This is a block diagram illustrating a system for manufacturing multiple sheet components and for performing defect imaging and inspection on the multiple sheet components according to at least one exemplary technique described in this disclosure. Figure 1 In the example shown, system 100 includes input 101, manufacturing process 102, sheet component 104, inspection device 105, imaging unit 108, one or more image capture devices 110A-110C (collectively referred to as "image capture device 110"), compartment 112, processing unit 120, and user interface 122.

[0023] like Figure 1 The manufacturing process 102 shown receives various inputs 101 (e.g., materials, energy, people, and machines) and produces an output comprising multiple sheet components (e.g., sheet component 104). Manufacturing process 102 is not limited to any particular type or form of manufacturing and illustrates any type of manufacturing process capable of operating to produce sheet components. In some examples, input 101 includes a long, continuous sheet of a rolled article product (e.g., a polyester film (e.g., an optical film)). Manufacturing process 102 may include dividing the long, continuous sheet into portions to form sheet components 104. For example, sheet component 104 may include sheets cut from the long, continuous sheet. In some examples, sheet component 104 includes sheets of a long, continuous sheet cut to the shape of an object (e.g., a mobile device, laptop computer, desktop computer, television, or window). Once produced by manufacturing process 102, sheet component 104 can be applied to the surface of the corresponding object.

[0024] Sheet components 104 may be substantially similar in shape, material composition, and thickness, such that each of the plurality of sheet components appears similar. In some embodiments, each sheet component 104 comprises a single layer of transparent or translucent material, or may comprise multiple layers of material. Sheet components 104 may comprise transparent or translucent materials designed to provide a specific level of transmittance (typically through the thickness dimension of sheet component 104) for light of a specific wavelength or a specific wavelength range. Sheet components 104 may have various requirements related to the flatness of the top and / or bottom surfaces of these sheet components and / or various requirements related to the absence of defects.

[0025] During manufacturing process 102, sheet component 104 may develop various defects. In some examples, defects include particles, abrasions, scratches, dents, streaks, or indentations. The presence of defects can determine the quality of the sheet component in sheet component 104. Some defects are minimal in size and severity and do not significantly affect the quality of the sheet component. However, other defects may be more severe and may adversely affect the quality of the sheet component. Thresholds can be established for each type of defect. If the number of defects for a certain type exceeds a major threshold level for that type of defect, the sheet component can be classified as defective. Furthermore, if a certain type of defect or a combination of certain types of defects is detected in the sheet component, the sheet component can be classified as defective. In some cases, a sheet component may be classified as "reworkable" if the number of defects of a particular type detected in the sheet component is greater than a second threshold level for said type of defect but less than a first threshold level for detected defects, or if a certain type of defect or combination of defects is detected in the sheet component where such defect or combination of defects does not render the sheet component unusable. Additionally or alternatively, a sheet component may be classified as acceptable if, for each type of defect, a defect level less than the second threshold level is detected in the sheet component, or if a certain type of defect or combination of defects is absent. Therefore, sheet components classified as requiring rework may have a higher quality level than defective sheet components and a lower quality level than acceptable sheet components.

[0026] System 100 can manufacture and inspect sheet components 104 in an assembly line. In other words, after sheet components 104 are formed in manufacturing process 102, these sheet components can travel through inspection device 105. Subsequently, sheet components 104 can be sorted into bin 112. Sheet components 104 can continuously circulate through system 100 such that as a newly inspected sheet component leaves inspection device 105 and enters bin 112, another sheet component enters inspection device 105. In some examples, a moving belt (e.g., a conveyor belt) continuously transports sheet components 104 from the end of manufacturing process 102 through inspection device 105 to bin 112.

[0027] Inspection device 105 may include imaging unit 108. In some examples, inspection device 105 includes an entry area (not shown) for sheet components 104 to enter inspection device 105. In some examples, the entry area of ​​inspection device 105 automatically receives sheet components from a collection of sheet components 104. For example, the entry area of ​​inspection device 105 may include a portion of a moving belt that transports sheet components 104 from manufacturing process 102, enabling sheet components 104 to enter inspection device 105. In other examples, sheet components 104 may be assembled into a stack after being formed by manufacturing process 102, and the individual sheet components of the stack of sheet components may be automatically and continuously assigned to the entry area of ​​inspection device 105. Additionally or alternatively, inspection device 105 may include a selection member (not shown) configured to select an individual sheet component from the stack of sheet components 104 and place that individual sheet component onto the entry area of ​​inspection device 105. In other examples, the equipment operator may manually place the sheet component 104 on the entry area.

[0028] Imaging unit 108 may include one or more image capture devices 110. Image capture device 110 may be a camera or other component configured to capture image data representing the sheet component 104 within imaging unit 108. In other words, image data captures a visual representation of the environment, such as the sheet component 104 within imaging unit 108. Image capture device 110 may include components capable of capturing image data, such as a video recorder, infrared camera, CCD (electrically coupled device) array, or laser scanner.

[0029] In some examples, the image capture device 110 may be a conventional imaging device capable of reading some or all of the moving sheet component and providing output as a digital data stream. The image capture device 110 may capture images corresponding to the sheet component in the sheet component 104. The image capture device 110 may be a camera that directly provides a digital data stream or an analog camera with an additional analog-to-digital converter. Alternatively, the image capture device 110 may include other sensors, such as, for example, laser scanners and line scan cameras.

[0030] In some examples, there may be a single image capture device (e.g., image capture device 110A) for capturing image data. In some examples, there may be multiple image capture devices (e.g., image capture devices 110A-110C), each capturing image data of the sheet component. In some aspects, the camera may capture image data of the sheet component from different angles or different perspectives. In such examples, the acquisition computer ( Figure 1 (Not shown) or processing unit 120 may create composite image data from image data of each image capture device in the image capture device 110. As used herein, image data may include image data captured from a single image capture device and composite image data of sheet components captured from multiple image capture devices.

[0031] After the imaging unit 108 captures an image of the sheet component, it can output image data to the processing unit 120. The processing unit 120 can be configured to evaluate the quality of the sheet component. For example, the processing unit 120 may include one or more inference engines 114 that can use a machine learning model 116 to detect and classify defects in the sheet component using the image data. The machine learning model 116 may define multiple layers of a convolutional neural network (CNN), where each CNN can be trained to detect different categories of defects based on the image data of the sheet component and output defect data indicating the defects detected in the image data. The processing unit 120 can use the defect data to determine data indicating a quality category for the sheet component. For example, the processing unit 120 may determine data indicating a quality category for the sheet component, where the quality category indicates whether the sheet component is acceptable, defective, or requires rework. The processing unit 120 may determine other quality categories besides acceptable, defective, or requiring rework, or alternatives to acceptable, defective, or requiring rework.

[0032] In some examples, user interface 122 allows a user to control system 100. User interface 122 may include any combination of a display screen, touch screen, buttons, speaker input, or speaker output. In some examples, user interface 122 is configured to power on or off any combination of components of system 100 and to provide configuration information to processing unit 120.

[0033] The inspection device 105 can receive information indicating a determined quality category of the sheet component and perform one or more operations based on the information indicating the quality category. In some aspects, in response to receiving the information indicating the quality category, the inspection device 105 can sort the sheet component into one of the bins in bin 112 based on the data indicating the quality category.

[0034] Bin 112 may be configured to receive sheet components classified into at least one quality category. In some aspects, three quality categories may exist, such as a qualified category (also referred to as the "good" category), a defective category (also referred to as the "bad" category), and a rework category as determined by processing unit 120. In some examples, bin 112 includes at least one bin corresponding to each quality category. For example, bin 112 may include a "good" bin, a "bad" bin, and a "rework" bin. The output area (not shown) of inspection device 105 may route the classified sheet components to the corresponding bins. Sheet components classified as defective by processing unit 120 are sorted into the bad bin. Defective sheet components may be discarded, recycled, or reused to manufacture another product. Additionally, in some examples, bin 112 includes a good bin, which is configured to receive sheet components classified as qualified. Sheet components sorted into the good bin may be shipped from the manufacturing facility, sold, and consumed. Bin 112 may also include a rework bin. Sheet parts sorted into the rework bin may have defects of a lower degree than those sorted into the bad bin and a higher degree than those sorted into the good bin. After being sorted into the rework bin, the sheet parts can be repaired (e.g., reworked) to remove defects. After rework, the processing unit 120 can classify the reworked sheet parts as having acceptable quality. Therefore, rework classification allows the system 100 to salvage sheet parts that would otherwise be classified as defective.

[0035] In some examples, the output area of ​​inspection device 105 automatically routes sorted sheet components to the appropriate bins. Additionally or alternatively, system 100 may indicate the sorting of sheet components, and the device operator may manually sort the sheet components into the appropriate bins. In examples where the output area automatically sorts sheet components into the appropriate bins 112, inspection device 105 may include a set of diverging tracks at the output area, where each of these diverging tracks leads to one of the multiple bins 112.

[0036] Figure 2 This illustrates at least one exemplary technology according to the present disclosure. Figure 1 A block diagram of additional details of System 100. Figure 2 The image shows an imaging unit 108, an image capture device 110, a moving belt 111, an acquisition computer 214, and a processing unit 120.

[0037] The image capturing device 110 can be arranged to inspect sheet components that continuously advance past the image capturing device. In such cases... Figure 2 In an exemplary embodiment of the inspection system 100 shown, sheet component 104 ( Figure 1At least one sheet component is positioned on the moving belt 111. The image capturing device 110 can be positioned close to the moving belt 111 carrying the sheet component, so that the image capturing device 110 can capture a reference image of each sheet component advancing through the imaging unit 108. Although in Figure 2 Three image capture devices 110 are shown, but the number of image capture devices that may be included in system 100 is not limited to a specific number of devices, and there may be fewer or more than three devices.

[0038] During an imaging process that can be performed using system 100, the moving belt 111 may advance in a direction generally indicated by arrow 117. In some embodiments, the moving belt 111 may be manufactured from a sheet component (not shown) by a manufacturing process (such as manufacturing process 102). Figure 1 This provides support and propels these sheet components. In other embodiments, such as Figure 2 The imaging of the sheet component is performed at a point in time after the sheet component has been manufactured and stored. In some embodiments, the sheet component may be in the form of a single sheet component having both a predefined width and length, and is advanced through an imaging unit 108 including an image capturing device 110, such that the image capturing device 110 can capture at least one reference image of the sheet component as it passes through the imaging unit 108.

[0039] Image capturing device 110 is not limited to any particular type of image capturing device and can be a conventional imaging device capable of imaging a sheet component and providing an output (such as a digital data stream of image data) in the form of an electronic signal as the moving belt 111 advances past image capturing device 110. In some exemplary embodiments, one or more image capturing devices in image capturing device 110 may be line scan cameras. In other exemplary embodiments, image capturing device 110 may be area scan cameras. In other exemplary embodiments, one or more image capturing devices in image capturing device 110 may be sensitive to the visible spectrum, UV spectrum, IR spectrum, or combinations thereof. In some exemplary embodiments having more than one image capturing device to capture image data of the sheet component, each image capturing device in image capturing device 110 is of the same type. In other exemplary embodiments having more than one image capturing device to capture image data of the sheet component, at least one image capturing device is of a different type of image capturing device compared to another image capturing device present in system 100.

[0040] As described above, there may be one image capture device for capturing image data of the sheet component, or there may be a set of image capture devices for capturing image data of the sheet component. Figure 2In the example shown, image capture device 110A, along with optional image capture devices 110B and 110C, captures image data of the sheet component. In some exemplary embodiments, image data of multiple sheet components may be captured substantially in parallel. For example, multiple sets of multiple image capture devices may exist, each set for acquiring image data of the sheet component as it passes through imaging unit 108.

[0041] Image capture device 110 may provide an electrical output signal representing a sensed image of a sheet component to acquisition computer 214. Acquisition computer 214 may be coupled to processing unit 120 and provide output representing image data captured by image capture device 110 to processing unit 120. In other embodiments, image capture device 110 may provide digital data streams and / or analog signals representing images captured by image capture device 110 directly to computing device, such as processing unit 120, for further processing by processing circuitry included in processing unit 120. Each set of image capture devices 110 may have its own acquisition computer 214, or multiple sets of image capture devices may be associated with the same acquisition computer.

[0042] The processing circuitry of processing unit 120 can process an image stream including image data provided from acquisition computer 214 or, alternatively, directly from image capture device 110, to generate an image of a sheet component advancing on moving belt 111 through imaging unit 108. Processing unit 120 can use multiple convolutional neural networks (CNNs) to analyze the image data, where each CNN is configured to determine whether a specific defect, such as dents, wrinkles, bubbles, ripples, or other low-contrast and / or low-frequency defects, is detected in the image data of the sheet component. For example, a low-contrast defect can be a defect whose intensity value varies within the normal range of product intensity variations. That is, when comparing a defective area with a normal product area, there is no substantial difference in intensity variation. In such examples, the defective area may only be identified as a noticeable pattern within the normal intensity variation. Furthermore, low-frequency defects may be relatively large relative to the pixel imaging resolution of the image. For example, a typical imaging resolution may be 5µm to 50µm per pixel. The size of a low-frequency defect may be 5mm or larger. Processing unit 120 can process the output of each CNN to determine the quality characteristics of the sheet component.

[0043] User interface (e.g.) Figure 1 The user interface 122 can be coupled to the processing unit 120 and can be used to provide a graphical display indicating the analysis results of multiple sets of reference images. For example, the user interface can indicate the determined quality of each sheet component advancing through the imaging unit 108.

[0044] Figure 3 This illustrates at least one exemplary technology according to the present disclosure. Figure 1 A block diagram of the top perspective view of the imaging unit 108 of the system. (See diagram below.) Figure 3 As shown, the perspective view of the imaging unit 108 may include a moving belt 111 and sheet components 304A-304C (collectively referred to as "sheet components 304").

[0045] Available from Figure 1 and Figure 2 The image capture device 110 shows the sheet component 304 at an angle. In other words, Figure 3 The sheet component 304 is shown in a bird's-eye view. The moving belt 111 can advance in the direction indicated by arrow 308, thereby allowing the sheet component 304 to pass through the image capture area 306. The image capture area 306 can correspond to... Figure 1 The image capturing device 110 is defined, and its field of view is represented. For example, the image capturing region 306 may represent the area captured by the image capturing device 110. Thus, each sheet component in the sheet component 304 passes through the image capturing region 306, and the image capturing device 110 captures image data corresponding to each sheet component that advances through the imaging unit 108 on the moving belt 111. Figure 3 In the example shown, one image capture area 306 corresponds to a single group of one or more image capture devices. As described above, in some exemplary embodiments, multiple groups of image capture devices 110 may exist to process sheet components substantially in parallel. In such exemplary embodiments, multiple image capture areas 306 may exist, each corresponding to a group of image capture devices 110.

[0046] Figure 4 This is a block diagram illustrating an exemplary processing unit 400 according to at least one exemplary technology described in this disclosure. The processing unit 400 may be... Figure 1 Examples or alternative implementations of the processing unit 120 of system 100. Figure 4 The architecture of the processing unit 400 shown is illustrated for illustrative purposes only. The processing unit 400 should not be limited to the illustrated exemplary architecture. In other examples, the processing unit 400 can be configured in various ways. Figure 4 In the example shown, processing unit 400 includes a defect detection unit 410 configured to detect one or more types of defects in the sheet component based on analysis of image data of the sheet component. Defect detection unit 410 may include one or more inference engines 412 configured to process the image data of the sheet component using a machine learning model 414 and provide defect detection data as output. Processing unit 400 also includes a quality assessment unit 420 configured to assess the quality of the sheet component based on defects detected by inference engine 410 (if any).

[0047] In some aspects, the machine learning model 414 may include multiple CNNs trained to detect different types of defects. The inference engine 412 may receive image data of the sheet component and pass that image data through multiple CNNs. The output of each CNN may be data indicating the presence of that type of defect detected by the CNN in the image data. The quality assessment unit 420 may receive data indicating the presence of each type of defect detected by the corresponding CNN in the image data and apply weights from weight 418 to the data for each defect type to produce data indicating a quality category for the sheet component. As described above, in some aspects, the quality category may include acceptable, defective, or requiring rework.

[0048] The processing unit 400 may be implemented as any suitable computing system (e.g., at least one server computer, workstation, mainframe, appliance, cloud computing system, and / or other computing system) capable of performing the operations and / or functions described in at least one aspect of this disclosure. In some examples, the processing unit 400 is electrically coupled to... Figure 1 The inspection device 105. In other examples, the processing unit 400 represents a cloud computing system, server cluster, and / or server group (or part thereof) configured to be connected to the system 100 via a wired or wireless connection. In other examples, the processing unit 400 may represent or be implemented through at least one virtualized computing instance (e.g., a virtual machine or container) of a data center, cloud computing system, server cluster, and / or server group. In some examples, the processing unit 400 includes at least one computing device, each computing device having memory and at least one processor.

[0049] like Figure 4 As illustrated in the example, processing unit 400 includes processing circuitry 402, at least one interface 404, and at least one storage unit 406. Defect detection unit 410, inference engine 412, and quality assessment unit 420 may be implemented as program instructions and / or data stored in storage unit 406 and executable by processing circuitry 402. Storage unit 406 may store machine learning models 414 and weights 418 associated with the outputs of these machine learning models. Storage unit 406 of processing unit 400 may also store an operating system (not shown) executable by processing circuitry 402 to control the operation of components of processing unit 400. Components, units, or modules of processing unit 400 may be coupled using communication channels (physical, communicative, and / or operational) for inter-component communication. In some examples, these communication channels include system buses, network connections, inter-process communication data structures, or any other method for transmitting data.

[0050] In one example, processing circuitry 402 may include at least one processor configured to implement functions and / or processing instructions for execution within processing unit 400. For example, processing circuitry 402 may be capable of processing instructions stored in storage unit 406. Processing circuitry 402 may include, for example, a microprocessor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or equivalent discrete or integrated logic circuitry, or any combination of the foregoing means or circuitry.

[0051] Multiple instances of processing circuitry 402 may exist within processing unit 400 to facilitate parallel processing of inspection operations. These instances may be of the same type, such as a multiprocessor system or a multi-core processor. Alternatively, the instances may be of different types, such as a multi-core processor with multiple associated graphics processing units (GPUs).

[0052] Processing unit 400 may communicate with external systems via at least one network using interface 404. In some examples, interface 404 includes an electrical interface configured to electrically couple processing unit 400 to inspection device 105. In other examples, interface 404 may be a network interface (e.g., an Ethernet interface, an optical transceiver, a radio frequency (RF) transceiver, Wi-Fi, or via a wireless technology using the trade name "Bluetooth," a telephone interface, or any other type of device capable of sending and receiving information). In some examples, processing unit 400 utilizes interface 404 to communicate with external systems (e.g., Figure 1 The inspection device 105) communicates wirelessly.

[0053] Storage unit 406 may be configured to store information within processing unit 400 during operation. Storage unit 406 may include a computer-readable storage medium or a computer-readable storage device. In some examples, storage unit 406 includes at least short-term memory or long-term memory. Storage unit 406 may include, for example, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), magnetic hard disk, optical disk, flash memory, magnetic disk, optical disk, flash memory, or electrically programmable memory (EPROM) or electrically erasable programmable memory (EEPROM). In some examples, storage unit 406 is used to store program instructions for execution by processing circuitry 402. Storage unit 406 may be used by software or applications running on processing unit 400 to temporarily store information during program execution.

[0054] Figures 5A to 5E This is a block diagram illustrating an exemplary configuration of a machine learning component of a sheet component inspection system according to at least one exemplary technique described in this disclosure. In some aspects, the machine learning component may be implemented as a defect detection unit 410. Figure 4 Part of ). Figure 5A The configuration of CNN504A-504N (collectively referred to as CNN 504) is shown, where each individual CNN is trained to detect different defects in a sheet component. For example, CNN 504A can be trained to detect dent defects in a sheet component. Figure 8A An exemplary sheet component with dent defects is shown. CNN 504B can be trained to detect wave defects in the sheet component. Figure 8B An exemplary sheet component with wave defects is shown. Similarly, other CNNs 504C-504N can be trained to detect defects such as wrinkles, bubbles, etc. One or more inference engines 412 can apply CNN 504 to the sheet component image data 502 to detect defects. In some aspects, one inference engine 412 may exist for each of the CNNs 504. The following is relative to Figure 6A Further exemplary details are described regarding training CNN 504 using labeled sheet component image data.

[0055] In some respects, the hyperparameters for each CNN can be tuned for the defects to be detected by the CNN. The hyperparameters for a CNN can be tuned for accuracy and / or processing efficiency relative to the defects to be detected by the CNN. For example, dent defects can be larger and / or easier to detect than wrinkle defects. The hyperparameters for a CNN trained to detect dent defects can be tuned for dent detection, and therefore can detect dents more efficiently or faster than a CNN using untuned hyperparameters.

[0056] exist Figure 5AIn the exemplary configuration shown, sheet component image data 502 is provided as input to each of the CNNs 504A-504N. Each CNN 504 processes the sheet component image data 502 to determine whether a defect detected by the corresponding CNN is detected in the sheet component image data. In some aspects, the CNNs may execute in parallel with each other. For example, each of the CNNs 504 may be processed on a different processor, processor core, or GPU. The output of each CNN of the CNNs 504 may be data indicating whether a defect detected by the CNN is present in the image data 502. In some aspects, the data may be a score with a value between zero and one, representing the probability that a defect is present in the sheet component based on the analysis of the sheet component image data 502. In some aspects, in addition to, or instead of, the probability value, a value indicating the severity of the defect may be output. The output of each CNN may be provided to a classifier 506. The classifier 506 may determine a quality category based on the score. The quality category may be provided to a sorter or other device to indicate how the sheet component should be handled. In some exemplary embodiments, if the quality category determined by classifier 506 indicates that the sheet component is of acceptable quality, the sheet component may be sorted into the "Good" bin 508. In some exemplary embodiments, if the quality category indicates that the sheet component is defective, the sheet component may be sorted into the "Bad" bin 510. In some exemplary embodiments, if the quality category indicates that the sheet component is unacceptable quality, the sheet component may be sorted into the "Rework" bin 510, but the detected defects may allow the sheet component to be reworked to an acceptable form or to be reworked for a different purpose.

[0057] In some respects, classifier 506 can apply different thresholds to each score. In this respect, if the score exceeds the maximum threshold for a particular defect, classifier 506 can classify the sheet part as defective. If the score is less than the minimum threshold, classifier 506 can classify the sheet part as acceptable. If the score is between the minimum and maximum thresholds, classifier 506 can classify the sheet part as reworkable. Scores can be weighted according to the importance of that type of defect.

[0058] In some respects, classifier 506 may perform a logical OR on each score produced by the CNN. For example, if at least one score for a defect indicates a defective classification, classifier 506 may classify the sheet part as defective. If none of the scores for a defect indicate a defective classification, but at least one score indicates a rework classification, classifier 506 may classify the sheet part as reworkable. If none of the scores indicate either defective or reworkable, classifier 506 may classify the sheet part as acceptable.

[0059] In some respects, classifier 506 can classify sheet parts based on a combination of scores output by CNN 504. For example, none of the individual scores may indicate a defect, but a combination of two or more scores may allow classifier 506 to classify the sheet part as defective or reworkable. In some respects, classifier 506 may use a set of one or more rules that can be applied to the scores to determine a quality category.

[0060] In some respects, the classifier 506 can implement machine learning techniques, such as neural networks, to determine the quality category for sheet parts. The neural network can be trained to determine the quality category based on different combinations of scores.

[0061] When determining the quality category for a sheet component, weights can be assigned to each score based on the importance of each defect. For example, a dent in a sheet component can be considered a more serious defect than a wave in the sheet component. The output of the CNN 504 that detects dents can be assigned a higher weight than the output of the CNN that detects waves in the sheet component. Classifier 506 can determine the quality category for the sheet component based on the weighted scores received from each of the CNNs 504.

[0062] Quality categories can be provided to sorters or other devices to indicate how sheet components should be handled. In some exemplary embodiments, if the quality category determined by classifier 506 indicates that the sheet component is of acceptable quality, the sheet component can be sorted into the "Good" bin 508. In some exemplary embodiments, if the quality category indicates that the sheet component is defective, the sheet component can be sorted into the "Bad" bin 510. In some exemplary embodiments, if the quality category indicates that the sheet component is of unacceptable quality, the sheet component can be sorted into the "Rework" bin 512, but the detected defects may allow the sheet component to be reworked into an acceptable form or to be reworked for a different purpose.

[0063] Figure 5B The configuration of CNN 504A-504N is shown, as discussed above. Figure 5AIn this system, each individual CNN is trained to detect different sheet component defects. Additionally, the outlier detection CNN 514 can be trained on the sheet component to learn how typical sheet components (even those with some degree of defect) are presented in image data. Sheet component image data 502 can be fed to each of the CNNs 504, and further, to the outlier detection CNN 514. The output of the outlier detection CNN 514 can be used to detect from the sheet component image data 502 that the sheet component differs significantly from the typical sheet component processed by system 100. In some respects, if the outlier detection CNN 514 detects such a sheet component, the sheet component can be assigned a rework quality category. This can result in the sheet component being sent to the rework bin 512 for manual inspection. Such processing can be desirable because it can be used to discover new types of defects that might not be detected using CNN 504. New CNNs can be trained to detect this CNN and added to CNN 504.

[0064] Figure 5C The configuration of CNNs 504A-504N is shown, where each individual CNN is trained to detect defects in different sheet components. In this example, the output of CNN 504 can be a subset of the complete image data in which defects were detected. Figure 5C In the example shown, the sheet component image data 502 includes image data of two defects 518 and 520. Defects 518 and 520 can be the same type of defect, or they can be different types of defects. One or more of the CNNs 504 can be trained such that the CNN can detect the location of defects, but cannot determine the severity of defects based on the image data. The CNN can output a subset of the image data, which includes the region around the suspected defect location. In this example, the CNN 504 can detect the presence and location of different types of defects, and in this example, the classifier 506 can be a CNN trained to determine the severity associated with the detected defects. Figure 5C In the example shown, image data subset 522 corresponds to image data including defect 518, and image data subset 524 corresponds to image data including defect 520 found in sheet component image data 502. CNNs 504B and 504D can detect the presence of defects 518 and 520, respectively. The image data subsets can be provided to classifier 506, which can be trained to determine the severity of defects based on the defect image data subsets.

[0065] Figure 5D The configuration of the machine learning model is shown, including a machine learning model trained with sheet component metadata according to the techniques of this disclosure. Figure 5DIn the example shown, each of CNNs 504A-504N is trained to detect different sheet component defects based on image data (or a subset of image data) of the sheet component. The outputs of CNNs 504A-504N (e.g., data indicating the presence or absence of defects detected by each CNN) can be collected into CNN feature data 532. Sheet component metadata 526 can also be used as input to the inspection system. Sheet component metadata 526 may include data describing aspects of sheet component manufacturing. Examples of such metadata include source data, time data, temperature data, location data, process setting data, and measurement data.

[0066] Source data may include data indicating the origin of the sheet component. Examples of such data include manufacturing line identifiers, warehouse identifiers, etc. Other source data may include the conditions or quality of the source, such as whether the warehouse is temperature and / or humidity controlled. Time data may include the time taken to prepare the sheet component. Temperature data may include the temperature at which the sheet component is prepared or the temperature at which the web of the sheet component is prepared. Location data may include the location within the web of the sheet component being produced. Process setup data may include data values ​​associated with the sheet component manufacturing process, such as oven temperature, tension values, speed values, etc. Measurement data may include data such as the ambient temperature and / or humidity at which the sheet component is prepared or the web of the sheet component is prepared. Measurement data may also include data generated from non-destructive testing applied to the sheet component or the web of the sheet component.

[0067] The sheet component metadata 526 may optionally be processed by a data filtering unit 528. The data filtering unit 528 may cleanse the data and perform feature extraction. For example, the data filtering unit 528 may normalize data values, filter unused data, corrupted data, and / or incomplete data, etc. To produce the filtered sheet component metadata 526' (or sheet component metadata 526 if no filtering is performed), it can be fed as input to a fully connected neural network layer 534 along with CNN feature data 532. The output of the fully connected layer 534 may represent data indicating the presence of any defects detected by the CNN 504 in the image data, based on both the image data 502 and the sheet component metadata 526. In some aspects, the output of the fully connected layer 534 may be fed as input to a softmax function 536. The softmax function 536 may convert the vector output of the fully connected layer 534 into a probability distribution 538 indicating the probability that the CNN 504 detects each defect in the sheet component based on the image data.

[0068] The sheet component metadata 526, 526', combined with CNN feature data 532, can improve defect detection in the inspection system. Temporal metadata can be useful in determining the presence of a defect when it is known that a particular defect is more likely to occur during a specific time period. For example, a particular defect may be more likely to occur during nighttime production.

[0069] When producing sheet parts from specific locations on a web, location metadata can be used to determine the presence of defects when they are more likely to occur. For example, a particular defect may be more likely to occur at the center of the web, so a sheet part produced from the center location may be more likely to have that particular defect.

[0070] Process setting metadata can be useful in identifying defects when they are more likely to occur based on one or more process settings. For example, defects are more likely to occur when a process for manufacturing sheet parts is performed at higher oven temperatures.

[0071] Measurement metadata can be useful in identifying defects when they are more likely to occur based on measurement data. Such measurement data may include data obtained from external sensors (e.g., sensors not belonging to specific components of the apparatus used to manufacture the sheet part) and / or data from non-destructive quality measurements. For example, lamination-related defects are more likely to occur when the ambient air temperature during the lamination operation used to form the sheet part exceeds a threshold. Ambient air temperature data may be obtained from external sensors during or approximately during the manufacture of the sheet part and used during the inspection process of the sheet part.

[0072] As an example of nondestructive quality measurement, measurement data obtained during air permeability testing of nonwoven materials in sheet components can be used to determine whether perforation defects exist in sheet components, such as medical filters.

[0073] Figure 5E The configuration of the machine learning model is shown, including a machine learning model 548 trained with sheet component metadata according to the techniques of this disclosure. Figure 5EIn the example shown, machine learning engine 546 may receive sheet component metadata history 542 and sheet component yield history 544 during the training phase in training environment 540. Machine learning engine 546 may use sheet component metadata history 542 and sheet component yield history 544 to generate machine learning model 548. Sheet component metadata history 542 may include metadata values ​​acquired over a period of time. As discussed above, metadata may include source data, time data, temperature data, location data, process setting data, and measurement data. Sheet component yield history 544 for the same time period is available. Once trained, machine learning model 548 can be used to determine the probability of sheet component defects based on sheet component metadata. The trained machine learning model 548 may be provided to production environment 550 including inspection system 100. The output defect prediction of machine learning model 548 may be combined with defect predictions provided by each of CNNs 504A-504N via weighting unit 506 to determine data indicative of sheet component quality characteristics. As discussed above, data indicating the quality of sheet components can be used to perform operations such as sorting sheet components into good bins 508, bad bins 510, and rework bins 512. Adding the output of a machine learning model 548 based on sheet component metadata to the output of a CNN 504A-504N based on image data of sheet components can produce a more accurate determination of sheet component quality.

[0074] Figures 5A to 5E The techniques described above can be incorporated into various specific implementations. For example, specific implementations may include... Figure 5B Outlier detection CNN 514 and Figure 5C The image cropper 540 is shown. Furthermore, relative to... Figures 5A to 5C Any of the techniques shown can be combined as relative to Figure 5D and Figure 5E Metadata 526 of the sheet components discussed. Furthermore, the aforementioned techniques can be combined with conventional inspection techniques. For example, conventional inspection techniques can be used to detect defects that are easier to detect (e.g., defects exhibiting high contrast in image data) and sort the sheet components accordingly. The techniques described herein can then be applied to sheet components inspected using conventional inspection techniques.

[0075] Figure 6AThis is a block diagram of a training environment 602 including a machine learning engine 614 for generating a machine learning model (defect root cause model 616) that associates sheet component defects with process variables in at least one exemplary technique described herein. The exemplary training environment 602 will be used to illustrate the training of a CNN for detecting defects using images of source components, and to illustrate the training of a machine learning model that can be used to determine or indicate the root cause of defects. The training environment 602 may include an inference engine 612, CNNs 604A-604N (collectively, “CNN 604”), and the machine learning engine 614. Each of the CNNs 604A-604N can be trained to detect different types of defects in manufactured components using source data 603. In some aspects, source data 603 may be a rich set of image data of a set of components with different defects. Images in source data 603 may be labeled as images, where the label may represent a ground truth relative to the type of defect shown by the corresponding image (if any). Images in source data 603 may be selected such that a number of images exist for each type of defect. For example, the component can be a sheet component, and source data 603 can be a rich set of image data associated with sheet components having known defects of different types. A machine learning engine can be used to train each of the CNNs 604 to detect different types of defects using labeled images. Once trained, CNN 604 can be deployed in a production environment to identify defects in images of the component. In such an example, once trained, CNN 604 can correspond to CNN504 in Figure 5.

[0076] In some aspects, the component may be a web, and source data 603 may include image data associated with portions of one or more webs. In such an example, CNN 604 may be trained individually to detect different types of defects in the web or portions of the web.

[0077] As described above, training environment 602 can be used to train defect root cause model 616 to identify potential root causes of defects. After CNN 604 has been trained as described above, inference engine 612 can apply CNN 604 to source data 603 to detect different types of defects. In this example, source data 603 can be labeled with both the type of defect and the part manufacturing time. The output of CNN 604 can be stored in defect history 606, which represents the history of defects over a period of time.

[0078] Process variable 608 may be metadata describing aspects of the processing or manufacturing conditions when manufacturing a part associated with source data 603. Examples of such metadata, as described above, include time data, temperature data, location data, process setting data, and measurement data associated with the manufacturing conditions when manufacturing a part associated with time source data 603. Process variable 608 may be stored in process variable history 610.

[0079] Machine learning engine 614 can analyze defect history 606 and process variable history 610 to determine, if any, the correlations between defects detected in the part and process variables acquired during part manufacturing. For example, machine learning engine 614 can apply time series analysis to defect history 606 and process variables 610 to determine the correlations between defects detected in the part and process variables acquired during part manufacturing. The learned correlations can be maintained in defect root cause model 616. Once trained, defect root cause model 616 can be used in various ways, such as... Figure 6B Used in the production environment described herein.

[0080] Figure 6B This is a block diagram of a production environment 620 that uses a trained defect root cause model 616 to determine the root cause of defects in sheet components according to at least one exemplary technique described in this disclosure. The production environment 620 may include a manufacturing process 630 for producing the manufactured components. For example, manufacturing process 630 may correspond to manufacturing component 102 for producing sheet components. Figure 1 Manufacturing process 630 can produce web components that can be used to prepare sheet components.

[0081] Inspection devices, such as inspection device 105 ( Figure 1 ) can be sent to processing units such as processing unit 120 ( Figure 1 Source data 605 is provided that is associated with the component. Source data 605 may correspond to sheet component image data 502. Figures 5A to 5E (This can be the source data 605, or it can be image data of the fabric. The inference engine 622 of the processing unit can transmit source data 605 via CNN 604 to detect defects in the component based on the source data 605.)

[0082] The manufacturing process 102 of a component may generate process variables 608 or otherwise associate them. As described above, process variables 608 may be time data, temperature data, location data, process setting data, and measurement data associated with manufacturing conditions when the component is being manufactured, as associated with source data 605. Inference engine 622 may use a defect root cause model 616 trained as discussed above to analyze source data 605 and process variables 608. For example, inference engine 622 may provide the output of CNN 604 indicating whether one or more defects are detected based on source data 605. Inference engine 622 may use defect root cause model 616 and process variables 608 when manufacturing the component to infer the root cause 624 of the detected defects. Processing unit may determine one or more feedback actions 626 designed to mitigate defects in the manufactured component, thereby potentially improving the yield of the manufactured component. Feedback action 626 may alert the operator to potential root causes. This alert may include a set of one or more recommended process adjustments that may be provided to the operator. In some respects, feedback operation 626 can be a modification of a processing variable that can be transmitted to manufacturing process 630 to automatically modify the manufacturing process.

[0083] The following non-limiting examples illustrate operation in a training environment 602 and a production environment 620 relative to the fabrication of the sheet component. One of the CNNs 604 detects particulate defects in image data of the sheet component. Such particles may be caused by human handling during upstream manufacturing processes. A machine learning engine 614 correlates temporal data associated with the defects to reveal spikes in the particulate defects at each shift change. As an example, this can be traced back to the opening of a cleanroom door.

[0084] One of the CNNs 604 can detect distortion defects that may occur along the length or width of the sheet component. A root cause model can be used to infer the underlying cause of the distortion defect, which is due to tension variations during winding and unwinding during the slitting process used to manufacture the sheet component. Feedback operation 626 can be transmitted to the manufacturing process to increase winding tension when the inspection system identifies wrinkles in the transverse direction and decrease tension when it identifies wrinkles in the longitudinal direction.

[0085] One of the CNNs 604 can detect dents (e.g., localized depressions) in sheet components. The defect root cause model 616 can be trained to associate dents with surface debris in the web processing rolls during coating or slitting processes that lead to recurring dents. A feedback action 626 can trigger an alert to clean the rolls when the inspection system identifies recurring dents. Based on the size, frequency, and / or location of the dents in the component, the system can be trained to identify the specific rolls that caused the dents.

[0086] The following non-limiting examples illustrate operation in a training environment 602 and a production environment 620 relative to the fabrication of the sheet web. One of the CNNs 604 can be trained to detect thin spots in the coating of the web. A defect root cause model 616 can be trained to determine the correlation with blockages in the coating or contaminants dragging on the web based on the characteristics of the thin spots. Feedback operation 626 for this defect root cause can generate a warning based on the stripe type caused by the thin spots and classified by the defect root cause model 616. Corrective actions, including cleaning the mold or removing debris from the manufacturing line, can be taken based on the root cause and the warning generated from the root cause.

[0087] As an example, one of the CNNs 604 can be trained to detect “machine direction line (MDL)” defects in a film. In various examples, MDL defects are defects in the film surface that extend in the longitudinal (longitudinal axis) direction and typically have extremely small dimensions in the transverse direction, such as in the range of 10 to 100 nanometers. This degree of transverse variation in the film surface is difficult to detect using known film inspection techniques. An exemplary detection technique entitled “Machine Direction Line Film Inspection” is described in Patent Cooperation Treaty application WO / 2018 / 045280, which is incorporated herein by reference. MD lines can be detected as thin spots in the web film (indicating the diameter gauge is configured in a low position). In such examples, a defect root cause model 616 can be trained to determine that the root cause 624 of the MD line defect is a problem in the extrusion die. Feedback operation 626 can be similar to the feedback operation for stripe defects, i.e., generating a warning based on the stripe type caused by the MD line classified by the defect root cause model 616. Based on the root cause and the warnings generated by the root cause, corrective actions can be taken, including cleaning the mold or removing debris from the manufacturing line.

[0088] As another example, one of the CNNs 604 can be trained in training environment 602 to detect scratches in the web based on input image data. Such defects can be small directional scratches on the film coating. The root causes of scratch defects can be mismatched roll speeds, dragging due to bearing wear, contaminants on the roll, etc. The root cause defect model 616 can be trained to distinguish the type of scratch (small, long, longitudinal, diagonal, etc.) and / or the grouping, density, or location of scratches, and associate the root cause with the type of scratch.

[0089] As another example, CNN 604 can be trained to detect defects by combining an inspection device that uses ultraviolet light to excite one or more layers of a fluorescent agent in a web. In some examples, the fluorescent agent emits light when excited by ultraviolet light of a different wavelength (such as light within the visible spectrum). After the web is exposed to ultraviolet light, an image capture device captures one or more images of the web, and a computational system analyzes the images to determine whether the web includes defects. If a portion of the image is darker than the surrounding portion, this may indicate that a portion of the web does not have sufficient material for impregnating the layer of fluorescent agent, indicating the presence of a potential defect in said area of ​​the web. Exemplary detection techniques are described in U.S. Provisional Patent Application 62 / 932,717 entitled “Ultraviolet Light-Based Inspection for Determining Coating Defects in Manufactured Webs Using Fluorescent Agents,” which is incorporated herein by reference.

[0090] As another example, a CNN 604 can be trained to detect defects and classify sheet articles based on application-specific parameters that can be used to influence the conversion selection process. Exemplary detection techniques are described in U.S. Patent 8,935,104 entitled “Application-Specific Repeated Defect Detection in Web Manufacturing Processes,” which is incorporated herein by reference.

[0091] Figure 7 Flowchart 700 illustrates the use of an inspection device, such as inspection device 105, according to at least one exemplary technology described in this disclosure. Figure 1 This is an exemplary operation of a method for determining the quality of sheet components. (Relative to...) Figure 1 System 100 describes exemplary operations. However, the exemplary operations of flowchart 700 should not be construed as being limited to system 100. The exemplary operations of flowchart 700 can be performed by any element or group of elements configured to perform the operations shown in flowchart 700.

[0092] According to the exemplary operation of flowchart 700, the input area of ​​inspection device 105 receives sheet components (702) from a plurality of sheet components 104. In some examples, to receive a sheet component, inspection device 105 selects a sheet component from a stack of sheet components and transfers the sheet component to the input area. To select a sheet component, inspection device 105 may include a selection member configured to remove a sheet component from the stack of sheet components and place the sheet component on the input area. Additionally or alternatively, to receive a sheet component, system 100 may be configured to receive a sheet component from an operator, who manually places the sheet component on the input area. In some examples, the input area of ​​inspection device 105 includes the starting point of a moving belt configured to transfer the sheet component from an endpoint of manufacturing process 102 through imaging unit 108.

[0093] Imaging unit 108 can detect a sheet component and capture image data (704) of the sheet component using one or more image capturing devices. In some examples, imaging unit 108 includes a lamp, wherein the sheet component travels between the lamp and image capturing device 110. Thus, in some such examples, image capturing device 110 can detect that the sheet component partially blocks the light as it passes between the lamp and image capturing device 110, and image capturing device 110 can thus determine that the sheet component is located within imaging unit 108. In response to detecting that the sheet component is located within imaging unit, at least one image capturing device 110 captures an image of the sheet component. In some examples, a single image capturing device 110 can be used to capture an image of the sheet component. In other examples, multiple image capturing devices 110 can capture images of the sheet component. Multiple images can be combined into a composite image.

[0094] The inspection device 105 outputs image data of the captured images to multiple CNNs 504 (706) implemented by the defect detection unit 410 of the processing unit 400. Each of the CNNs 504 can be pre-trained to detect different sheet component defects in the image data of the sheet component. The output of each CNN can be data indicating whether a sheet component defect detected by the CNN exists in the sheet component, based on the image data provided to the CNN. In some aspects, the data indicating whether a sheet component defect detected by the CNN exists in the sheet component can be a probability value based on the image data indicating the probability that a defect detected by the CNN exists in the sheet component.

[0095] The quality unit 420 of the processing unit 400 may receive the output of the CNN 504 and use the output to determine the quality (708) of the sheet component corresponding to the image data. In some examples, the quality unit 420 may apply different weights to the output of each of the CNN 504 based on the importance of the defects detected by the CNN. The weighted outputs may be evaluated individually or in combination to determine data indicating the quality associated with the sheet component.

[0096] Quality unit 420 can sort sheet components (710) based on data indicating the quality of the sheet components. In some examples, sheet components can be classified into at least one of a qualified category, a defective category, or a rework category. In some examples, processing unit 400 can sort sheet components into a warehouse based on the category.

[0097] Figure 8A and Figure 8B An exemplary sheet component defect that can be identified according to at least one exemplary technique described in this disclosure is shown. Figure 8A An exemplary sheet component 802 with a dent defect 804 is shown, which can be detected using the techniques disclosed herein. The dent defect 804 may be a low-contrast defect, and thus may be undetectable using conventional image processing techniques with existing systems.

[0098] Figure 8B An exemplary sheet component 810 with wavy defects is shown, which can be detected using the techniques described herein. As... Figure 8A Similar to the dent defect 804 shown, the waves in the sheet component 810 can be of low contrast and / or low frequency, and therefore may not be detected using conventional image processing techniques with existing systems.

[0099] The techniques described herein are primarily discussed in the context of inspecting defects in sheet components. These techniques can be readily applied to other inspection systems. For example, a CNN can be trained to detect defects in the web material used to manufacture sheet components or other products, as will be discussed in the following references. Figure 9 Let's have a discussion.

[0100] Figure 9 This is a block diagram illustrating an exemplary embodiment of a production line in an exemplary embodiment of a web fabrication plant 902. In this exemplary embodiment, a section of web 920 is positioned between two support rollers 922, 924. Image acquisition devices 110A–110N (“Image Acquisition Device 110”) are positioned close to the continuously moving web 920. Image acquisition device 110 can scan a continuous portion of the continuously moving web 920 to obtain image data. Acquisition computer 214 collects image data from image acquisition device 110 and transmits the image data to processing unit 120 for analysis.

[0101] The image acquisition device 110 can be a conventional imaging device capable of reading continuous portions of the moving webpage 920 and providing output as a data stream. For example... Figure 2 As shown, the imaging device 110 can be a camera that directly provides a digital data stream or an analog camera with an additional analog-to-digital converter. Other sensors, such as laser scanners, can be used as image capture devices. Examples of suitable devices for acquiring images include line scan cameras, such as the Dalsa (Waterloo, Ontario, Canada) or E2V (Chelmsford, United Kingdom).

[0102] Images can be optionally acquired using optical components that assist in image acquisition. These components can be part of the camera or separate from it. The optical components utilize reflected, transmitted, or refracted light during the imaging process. Reflected light, for example, is often suitable for detecting defects caused by deformation of the fabric surface, such as surface scratches.

[0103] Position controller 930 controls position mark reader 929 to collect roll information and position information from web 920. For example, the position mark controller may include one or more optical photographic sensors for reading position marks, such as barcodes, reference marks, or other position marks, from web 920. Additionally, position controller 930 may receive position signals from one or more high-precision encoders engaged with web 920 and / or rolls 922, 924. Position controller 930 determines position information for each detected position mark based on the position signals. For example, position controller 930 may generate position information for locating each detected position mark within a coordinate system applied to the production line. Alternatively, processing unit 120 may place each detected position mark in the coordinate system based on position data received from position controller 930. In this case, the position data provided by position controller 930 may represent the distance between each position mark along the length of web 920. In either case, position controller 930 transmits roll information and position information to processing unit 120.

[0104] Processing unit 120 processes the image stream from acquisition computer 214. Processing unit 120 uses a CNN 904 to process the image data, which is trained to detect defects in the web 920 based on the image data in the image stream. Similar to CNN 504 discussed above, each CNN can be trained to detect defects of different types than those of other CNNs. Each CNN can operate in parallel with each other using one or more inference engines 114. The output of each CNN can be data indicating whether the type of defect detected by the CNN is present in the web 920. In some aspects, the output can be a probability value of the defect's presence, as well as the severity of the defect. The output of each CNN can be evaluated in a manner similar to that discussed above with respect to the sheet component. For example, one or more rules can be applied to the output to determine how to handle the presence of defects with respect to the web 920.

[0105] Based on the position data generated by the position controller 930, the processing unit 120 determines the spatial location of each detected defect within the production line coordinate system. That is, based on the position data from the position controller 930, the processing unit 120 determines the x, y, and possible z positions of each anomaly within the coordinate system used in the current production line. For example, a coordinate system can be defined such that the x-dimensional represents the lateral distance of the web 920, the y-dimensional represents the distance along the length of the web, and the z-dimensional represents the height of the web, which can be based on the number of coatings, the material, or other layers previously applied to the web. Furthermore, the origin of the x, y, z coordinate system can be defined at a physical location within the production line, and the origin is typically associated with the initial feed position of the web 920. The coordinate system defined for the current production line may not (and typically is not) be the same coordinate system applied to any previous or subsequent processes on the web 920.

[0106] In any case, processing unit 120 records the spatial position of each detected defect relative to the production line coordinate system in database 932; this information is referred to herein as local defect information. Other process variables may also be stored in database 932. That is, processing unit 120 stores local defect information of web 920 within database 932, including roll information of web 920 and position information of each detected defect. Local defect information generated for the current production line can then be spatially aligned with local defect information generated by other production lines for the same web. Further details regarding local defect information alignment can be found in U.S. Patent 7,542,821, filed July 26, 2007, entitled “MULTI-UNIT PROCESS SPATIAL SYNCHRONIZATION OF IMAGE INSPECTION SYSTEMS,” which is incorporated herein by reference.

[0107] In some respects, defect information can be stored in database 932. Time series analysis of defect information can be applied to the defect information and used to determine trends or patterns in a series of defects.

[0108] The techniques described in this disclosure can be implemented, at least in part, in hardware, software, firmware, or any combination thereof. For example, aspects of the techniques can be implemented within at least one processor, which includes at least one microprocessor, DSP, ASIC, FPGA, and / or any other equivalent integrated or discrete logic circuitry, and any combination of such components. The terms "processor" or "processing circuitry" can generally refer to any one of the aforementioned logic circuits, alone or in combination with other logic circuitry, or any other equivalent circuitry. A control unit including hardware can also perform at least one of the techniques of this disclosure.

[0109] Such hardware, software, and firmware may be implemented within the same device or in different devices to support the various operations and functions described in this disclosure. Furthermore, any of the described units, modules, or components may be implemented together or separately as discrete but interoperable logical devices. Describing different features as modules or units is intended to highlight different functional aspects and does not necessarily imply that such modules or units must be implemented by separate hardware or software components. Rather, the functionality associated with at least one module and / or unit may be performed by separate hardware or software components, or integrated into common or separate hardware or software components.

[0110] The techniques described in this disclosure can also be embodied or encoded in a computer-readable medium containing instructions, such as a non-transitory computer-readable medium or a computer-readable storage medium. Instructions embedded or encoded in a computer-readable medium can cause a programmable processor or other processor to perform the method (e.g., when executing the instructions). Computer-readable storage media may include: RAM, read-only memory (ROM), programmable read-only memory (PROM), EPROM, EEPROM, flash memory, hard disk, CD-ROM, floppy disk, magnetic tape cassette, magnetic media, optical media, or other computer-readable storage media. The term "computer-readable storage medium" refers to a physical storage medium, not a signal or carrier wave, although the term "computer-readable medium" may also include transient media, such as signals, in addition to physical storage media.

Claims

1. A system for determining the quality category of each of a plurality of sheet components produced by a manufacturing facility, the system comprising: The inspection apparatus includes at least one image capturing device configured to capture image data representing the sheet components among the plurality of sheet components; A processing unit having one or more processors, the one or more processors executing instructions that cause the processing unit to: The image data representing the sheet component is provided to a first group of multiple neural networks, each of which is trained to identify a corresponding defect in the sheet component and output data indicating the presence of the corresponding defect. Based on the data indicating the presence of the corresponding defect output by each corresponding neural network, data indicating the quality category of the sheet component is determined, and Output the data indicating the quality category of the sheet component; as well as An outlier detection neural network, wherein, in response to determining, based on the output of the outlier detection neural network, the image data indicating an outlier defect of the sheet component, the data indicating the quality category of the sheet component is set to a rework category.

2. The system of claim 1, wherein the processing unit is configured to evaluate a set of one or more rules, the set of one or more rules using the data, output by each corresponding neural network, indicating the presence of the corresponding defect, to determine the quality category of the sheet component.

3. The system of claim 1, wherein the processing unit is configured to provide the data, output by each corresponding neural network, indicating the presence of the corresponding defect, to a second neural network to determine the quality category of the sheet component.

4. The system of claim 1, further comprising a sorter configured to: Receive the data indicating the quality category of the sheet component; and The sheet components are sorted based on the data indicating the quality category of the sheet components.

5. The system of claim 4, wherein the data indicating the quality category of the sheet component includes a qualified category, a defective category, or a rework category.

6. The system of claim 1, wherein the neural network of the plurality of neural networks outputs one or more image portions, each image portion containing image data representing a potential defect, and wherein a classifier is trained to identify a corresponding defect in the sheet component based on the one or more portions of the image data representing the potential defect.

7. The system of claim 1, further comprising a machine learning model trained to determine the probability of a defect based on metadata associated with the manufacturing of the sheet component, wherein the processing unit is further configured to determine the quality category based on the probability of the defect in the metadata.

8. The system of claim 7, wherein the metadata includes one or more of source data, time data, temperature data, location data, process setting data, and sensor measurement data associated with the manufacturing of the sheet component.

9. The system of claim 1, further comprising an inference engine configured to receive data indicating the presence of the corresponding defect from the output of each corresponding neural network and one or more process variables, and configured to determine the root cause of the corresponding defect based on the data indicating the presence of the corresponding defect from the output of each corresponding neural network and the process variables.

10. The system of claim 9, wherein the processing unit is further configured to output feedback operation based on the root cause of the corresponding defect.

11. The system of claim 1, wherein the neural network comprises a convolutional neural network.

12. The system of claim 1, wherein at least one of the neural networks is trained to detect one of particles, wear, scratches, dents, bubbles, stripes, or indentations.

13. The system of claim 1, wherein the sheet component comprises an optical film.

14. An inspection apparatus for determining the quality category of web material produced by a manufacturing facility, the inspection apparatus comprising: At least one image capturing device, the at least one image capturing device being configured to capture image data representing the web material; A position controller configured to acquire position data along the web material; as well as Processing unit, the processing unit being configured to: The image data representing the web material is provided to each of a plurality of neural networks, each of which is trained to identify a corresponding defect in the web material and output data indicating the presence of the corresponding defect. Based on the data indicating the presence of the corresponding defect output by each corresponding neural network, data indicating the quality category of the web material is determined. In response to the output of the outlier detection neural network determining that the image data representing the web material indicates an outlier defect, the data indicating the quality category of the web material is set to the rework category, and The output includes the data indicating the presence of the corresponding defect and the location data associated with the data indicating the presence of the corresponding defect.

15. A method for determining the quality category of each of a plurality of sheet components produced by a manufacturing facility, the method comprising: Receive image data representing sheet components among the plurality of sheet components inspected by at least one image capturing device; The image data representing the sheet component is provided to a plurality of neural networks, each of which is trained to identify a corresponding defect in the sheet component based on the image data representing the sheet component, and output data indicating the presence of the corresponding defect. Based on the data indicating the presence of the corresponding defect output by each corresponding neural network, data indicating the quality category for the sheet component is determined. In response to the output of the outlier detection neural network determining that the image data representing the sheet component indicates an outlier defect, the data indicating the quality category of the sheet component is set to the rework category, and Output the data indicating the quality category of the sheet component.

16. The method of claim 15, wherein determining the data indicating the quality category of the sheet component comprises evaluating a set of one or more rules, the set of one or more rules using the data, output by each corresponding neural network, indicating the presence of the corresponding defect.

17. The method of claim 15, wherein determining the data indicating the quality category of the sheet component comprises providing the data to a second neural network indicating the presence of the corresponding defect as output by each corresponding neural network.

18. A computer-readable medium having instructions stored thereon, the instructions being executable by one or more processors to cause the one or more processors to: Receive image data representing sheet components among a plurality of sheet components inspected by at least one image capturing device; The image data representing the sheet component is provided to a plurality of neural networks, each of which is trained to identify a corresponding defect in the sheet component based on the image data representing the sheet component, and output data indicating the presence of the corresponding defect. Based on the data indicating the presence of the corresponding defect output by each corresponding neural network, data indicating the quality category for the sheet component is determined. In response to the output of the outlier detection neural network determining that the image data representing the sheet component indicates an outlier defect, the data indicating the quality category of the sheet component is set to the rework category, and Output the data indicating the quality category of the sheet component.

19. The computer-readable medium of claim 18, wherein the instructions for determining the data indicating the quality category of the sheet component include instructions for evaluating a set of one or more rules, the set of one or more rules using the data, output by each corresponding neural network, indicating the presence of the corresponding defect.

Images