Method and apparatus for debugging artificial intelligence-based inspection system

Abstract

The described implementations provide a technique for debugging AI-based inspection systems. In particular, the described implementations provide a tool with instant feedback for a field engineer to perform tasks including data collection, and training, testing, and deployment of AI algorithms in the field. The described tasks are performed in the same physical environment with the same sensor settings that are in turn used by the field devices on which the AI algorithms are deployed. The deployment cycle is significantly reduced. The technique leverages the recognition that inspection tasks in industrial settings are repetitive in nature and performed in a controlled environment, whereby the AI algorithms do not need to generalize beyond the specific use case.

Description

Technical Field

[0001] This disclosure relates to the debugging of artificial intelligence (AI) systems in the art. The described embodiments are particularly suitable for debugging AI systems capable of performing repetitive inspection tasks in industrial environments. Background Technology

[0002] AI algorithms are typically trained on massive amounts of data with the aim of generalizing to a wide range of use cases. For example, an AI-based object detection algorithm for detecting cars in images can be trained with a large amount of data related to different types of cars produced by different car manufacturers, with the goal of training a model that can be used in many general use cases. Similarly, AI-based algorithms for product inspection can be trained to target a wide range of products and features. Typically, the training and tuning of such AI algorithms is an iterative process; it is time-consuming and involves various resources in terms of personnel, software, and hardware. Summary of the Invention

[0003] In short, the various aspects of this disclosure relate to the on-site commissioning and deployment of AI-based inspection systems, with a focus on shortening workflow / process cycles to provide immediate feedback for retraining and tuning AI algorithms to achieve optimal performance for specific use cases.

[0004] According to a first aspect of this disclosure, a method for debugging an AI-based inspection system is provided. The method includes (i) receiving sensor data captured by a sensor positioned in a physical environment to capture sensor data relating to individual items in a sequence of similar items by a debugging computer; (ii) collecting multiple data samples by the debugging computer, wherein collecting each data sample includes providing an operator interface to a field operator to access and assign classification labels to the sensor data relating to the individual item, the collected data samples corresponding to fixed settings of the sensors; (iii) training an AI algorithm using the collected data samples by the debugging computer to configure the AI ​​algorithm to predict classification labels based on input sensor data; and (iv) testing the trained AI algorithm by the debugging computer by providing real-time sensor data captured by a sensor positioned in a physical environment with fixed settings. Steps (ii) through (iv) are performed iteratively using different fixed settings of the sensors to determine the final setting of the sensor that achieves defined success criteria during testing. The method also includes (v) deploying the iteratively trained and tested AI algorithm to a field device coupled to a sensor configured with the final settings to provide sensor data as input to the deployed AI algorithm.

[0005] Other aspects of this disclosure implement the features of the above-described method in computer program products and computing devices.

[0006] Additional technical features and benefits can be achieved through the technology disclosed herein. Embodiments and aspects of this disclosure are described in detail herein and are considered part of the claimed subject matter. For a better understanding, refer to the detailed description and accompanying drawings. Attached Figure Description

[0007] The foregoing and other aspects of this disclosure are best understood when read in conjunction with the accompanying drawings, in accordance with the following detailed description. For ease of identification of any element or action discussed, the most significant digit or digit in the reference numerals refers to the figure number in which the element or action is first introduced.

[0008] Figure 1 is a schematic diagram illustrating an example of a known processing flow for deploying an AI system in the field.

[0009] Figure 2 This is a schematic diagram illustrating a method for field commissioning and deployment of an AI-based inspection system according to an exemplary embodiment of the present disclosure.

[0010] Figure 3 An exemplary computing environment is shown that can implement aspects of this disclosure. Detailed Implementation

[0011] Various techniques related to the system and method will now be described with reference to the accompanying drawings, wherein the same reference numerals always denote the same elements. The drawings discussed below and the various embodiments used to describe the principles of this disclosure in this patent document are for illustrative purposes only and should not be construed as limiting the scope of this disclosure in any way. Those skilled in the art will understand that the principles of this disclosure can be implemented in any suitably arranged apparatus. It should be understood that functions described as being performed by certain system elements can be performed by multiple elements. Similarly, for example, elements can be configured to perform functions described as being performed by multiple elements. Many of the innovative teachings of this application will be described with reference to exemplary, non-limiting embodiments.

[0012] It is recognized that many artificial intelligence (AI) algorithms cannot be used out of the box and must be retrained and tuned for specific use cases using data from real-world scenarios. For example, an AI algorithm might include a neural network-based object detection method that uses cameras to visually detect and locate defects on products on a production line or assembly line. Typically, the algorithm starts with a pre-trained neural network. The subsequent retraining process might involve collecting images of specific products and labeling the identified defects on the images, then providing those labeled images as training data to retrain the neural network. The retrained neural network is then tested on a test dataset. Based on the results obtained from the trials (e.g., success or inferred quality), the training or data collection process is tuned to improve performance until the test results meet defined requirements. After successful retraining and testing, the AI ​​algorithm is deployed for field operations.

[0013] Figure 1 illustrates an example of a known process 100 for deploying an AI system in the field. The described process 100 includes: data collection by a field engineer (box 102), comprising data collected from multiple use cases (e.g., multiple fields, multiple machines, etc.); data labeling by a domain expert (box 104); data management by a data engineer (box 106), which includes, for example, transferring data to the cloud; training an AI algorithm, such as a neural network, by a data scientist (box 108), which is typically performed in a cloud computing environment; and deploying the AI ​​algorithm back to the field (box 110) to obtain feedback from the field engineer (box 112). Based on this feedback (box 112), multiple iterations of retraining can be performed until the AI ​​algorithm is ready to be deployed in the field.

[0014] The process described above provides a systematic approach to a wide range of use cases, with well-defined roles and responsibilities. The basic idea here is that each role has its own expertise and can therefore fully contribute to achieving optimal results. For example, it is convenient to store data and train neural networks in the cloud because the cloud can hold massive amounts of data and provide abundant computing resources.

[0015] However, in many applications, the above process can cause practical problems. First, the process involves long iterative loops across multiple (sometimes disconnected) modules. In typical deployment use cases, many iterations may be required on this long loop to reach a usable solution. In this case, the above process significantly increases time and cost. Second, the above process does not provide immediate feedback on test results. Preferably, field engineers want to use the results in-situ to determine how to adjust the sensors and how to recollect tag data. In the above process, field engineers may have to wait for the entire loop to complete to determine the next step, which prevents the task from being completed. Furthermore, any discrepancies in the setup at each step can lead to problems in obtaining successful inferences from the AI ​​algorithm. For example, if the data used to train the AI ​​algorithm has been converted to another format in the cloud with some information loss, this may result in some discrepancies with the actual data obtained from the sensors, causing the AI ​​algorithm to fail. Other potential obstacles include cross-organizational data privacy and sharing, cybersecurity, etc.

[0016] In summary, the long loops in the aforementioned process increase costs, lack immediate feedback, and can lead to data inconsistencies, all of which can result in suboptimal settings for deploying AI algorithms in the field. Addressing these suboptimal settings while still delivering good results is a challenging research topic. For example, ongoing research focuses on how to train AI algorithms or models on one type of data and ensure they can generalize to other types. However, as described above, these approaches still have a low level of readiness for practical implementation.

[0017] Embodiments of this disclosure relate to a technique for field-deploying AI-based testing systems, focusing on shortening process cycles by providing field feedback to adjust and retrain AI algorithms, with the goal of achieving optimal performance for specific use cases. In one aspect, the proposed solution can be embodied in a software suite comprising tools for field engineers to manage the entire AI pipeline, including data collection, training, testing, and conditioning within a single commissioning device (computer). In implementations, the commissioning device can be located on-site, providing a “local deployment” solution for implementing the AI ​​pipeline. This is particularly useful in many situations where field engineers cannot access the internet or other external networks during deployment, or cannot share data outside the facility, due to limitations on communication or data sharing.

[0018] The proposed solution is based on the understanding that in some real-world applications, particularly in manufacturing, many machine learning inspection tasks are repetitive on the same type of products and are performed in controlled environments. For example, on production lines or assembly lines, "good" products are almost identical, "defects" are similar or identical, products are positioned or oriented the same way on the line, and inspection stations on the same line have almost identical controlled lighting conditions at all times, and so on. In traditional methods (as shown in Figure 1), the training process of AI models typically attempts to generalize, for example, to avoid overfitting. This requires a large amount of data and may further require data augmentation and other training techniques. The goal of the training process is for the AI ​​model to be usable in many general cases, not just those close to the training scenario, once trained. However, due to one or more of the reasons mentioned above, in repetitive inspection tasks, especially in manufacturing, it may not be necessary to generalize the model training process beyond the specific use cases where the AI ​​model will be deployed. As a result, the size of the training dataset can be significantly reduced, and the deployment cycle can be significantly shortened. The proposed solution leverages the above findings by training the AI ​​model in the same location using a “single-box” debugging device and utilizing the same environment and sensor setup as where the AI ​​model is tested and ultimately deployed to achieve a successful AI solution that meets cost and time constraints.

[0019] Figure 2 A method for field commissioning and deployment of an AI-based inspection system, according to an example implementation, is shown. As mentioned above, the method is implemented via a commissioning computer, which... Figure 2 The debug computer is schematically represented by dashed box 202. It can be a PC (desktop computer, laptop computer) or any other computing device having at least one processor and a memory storing instructions, which, when executed by the processor, causes the computing device to perform an AI pipeline. For illustration, the functional blocks of the AI ​​pipeline executed via the debug computer are located within dashed box 202.

[0020] While the described embodiments illustrate a specific application of the proposed solution, namely, fault detection and classification in a production / assembly line, the general teachings of this disclosure can be extended to other factory tasks, such as identifying the presence or location of objects, detecting the physical characteristics of objects, etc.

[0021] During the commissioning and deployment phase, the commissioning computer 202 may be located within the site of the industrial facility where the AI-based inspection system is to be deployed. This industrial facility may include a production facility comprising a physical environment 204 within which sensors 206 are located to capture sensor data relating to individual items in a sequence of similar items to be inspected. In implementations, the physical environment 204 may include a production line or assembly line. The term "similar item" refers to an item with the same design but with the potential for defects or malfunctions, for example, arising from the production or assembly process.

[0022] Sensor 206 is connected to field device 208, where deployed AI algorithms are executed. Field device 208 may include a field controller, such as a programmable logic controller, industrial PC, industrial edge computer, etc. In one embodiment, field device 208 may include one or more neural processing unit (NPU) modules. The NPU includes dedicated edge AI hardware that can be custom-designed to run neural networks in a computationally efficient manner.

[0023] For the training and testing process, the debugging computer 202 uses sensor data from the same sensor 206 that will be fed to the field device 208 during actual operation.

[0024] Inspection tasks on the production / assembly line (e.g., detecting faults or defects) can be performed based on visible features, acoustic anomalies, or changes in vibration characteristics. In the described example, sensor 206 includes a vision camera (e.g., a gigabit Ethernet camera) mounted at a preferred location on the production / assembly line and configured to send a video stream to commissioning computer 202 (during commissioning) and field device 208 (after deployment). In other embodiments, sensor 206 may include an acoustic sensor, such as a microphone, configured to send an audio stream to commissioning computer 202 (during commissioning) and field device 208 (after deployment). In other embodiments, sensor 206 may include a vibration sensor, such as a piezoelectric sensor. In further embodiments, multiple sensors of the same or different types may be used. In one example, sensor 206 may be configured to support multicasting of sensor data to commissioning computer 202 and field device 208. Alternatively, after commissioning and deployment, the sensor cable may be disconnected from commissioning computer 202 and inserted into field device 208.

[0025] In summary, the AI ​​pipeline solution executed via the debugging computer 202 includes iteratively performing the following steps: data collection (including data tagging) based on sensor data captured by sensors 206 located within the physical environment 204 to be tested; training an AI model (algorithm) using the collected data samples; optionally converting the AI ​​model to a format supported by the field device 208; testing the trained AI model by providing the trained (and, if applicable, converted) AI model with real-time sensor data captured by sensors 206 located within the same physical environment 204; and retraining the AI ​​model based on the test results. Each iteration or loop is performed using different fixed settings of the sensors 206. That is, the settings of the sensors 206 remain unchanged during the execution of a single iteration or loop. This iterative process is used to determine the final fixed settings of the sensors 206, for which defined success criteria are implemented during testing. Each fixed setting of the sensors 206 can be defined by a combination of internal sensor settings and / or the arrangement of the sensors 206 relative to the physical environment 204. After successful training and testing, the AI ​​model is deployed to the field device 208. Following this point, field device 208 is connected to sensor 206, which is configured with a final fixed setup, to provide sensor data as input to the deployed AI model, for example, for performing product inspection / fault classification.

[0026] The data collection step involves collecting multiple data samples by providing an operator interface to field operators (e.g., field engineers) to access sensor data related to individual items in the production / assembly line and assign them classification labels. The collected data samples correspond to fixed settings of sensor 206.

[0027] Still referencing Figure 2In the described embodiment, the operator interface of the commissioning computer 202 includes a sensor data viewer 210 that displays real-time sensor data to the field operator. In this example, the sensor data viewer 210 may include a camera viewer that displays a real-time video stream representing the dynamic physical environment to be inspected (e.g., a sequence of items moving along a production line / assembly line). Based on the real-time sensor data, the field operator can evaluate images relevant to individual items and perform sensor setting adjustments 212 such that the collected sensor data is satisfactory in terms of quality and relevance. In this embodiment, sensor 206 includes a vision camera. In this case, the adjustable settings of camera 206 include internal settings of camera 206, such as focus (lens), brightness, contrast, saturation, shutter speed, ISO, aperture, etc., and / or the arrangement of camera 206 relative to the physical environment 204, such as lighting, camera placement, viewing angle (mount bracket and angle), etc. These settings can be adjusted for specific items on a particular production / assembly line so that the collected images reveal the most relevant features in the same physical environment. In another embodiment, where sensor 206 includes a microphone, the adjustable microphone settings may include internal microphone settings such as gain, sampling rate, frequency filters, etc., and / or the microphone's arrangement relative to the physical environment, such as the microphone's mounting location and orientation. After a satisfactory sensor setting adjustment 212 is determined by the field operator, the settings of sensor 206 remain unchanged during specific cycles of the data collection, training, and testing process.

[0028] The operator interface of the debugging computer 202 may also include a data tagging user interface (UI) 214 for providing real-time sensor data to the field operator so that they can tag the data after the sensor settings are fixed. In the described embodiment, the data tagging UI 214 may include a frame grabber for capturing still images from a real-time video stream. To tag the captured images, the data tagging UI 214 enables the field operator to locate identified faults or defects by drawing bounding boxes around the region of interest in the captured images. The field operator can use domain expertise to assign fault classification labels to the captured images of the project. Fault classification labels may be binary (e.g., good for fault) or include multiple labels (e.g., category 0 – indicating the project is good or defect-free; category 1 – indicating sub-parts A and B are misaligned; category 2 – indicating sub-part C is not fully inserted into sub-part D; etc.). The data tagging UI 214 enables the field operator to enter tags associated with the captured images in a standard format (such as text, html, etc.). Each captured image with its bounding box and corresponding tag is saved as a data sample 216 with a tag. In this example, the labeled data sample 216 may have one or more labels associated with one or more bounding boxes in the captured image. As mentioned above, training an AI model for a specific use case involving the repeated testing of similar projects in a controlled environment requires a significantly reduced number of data samples. For example, a few hundred data samples may be sufficient, as long as they cover most lighting variations in a factory, which is minimal in an indoor environment. Therefore, the labeled data sample 216 obtained by the data collection process can be stored directly on the local storage medium of the debug computer 202, thereby eliminating the need to transmit data outside the device (i.e., no cloud connection is required).

[0029] Once a sufficiently large amount of labeled data samples 216 have been collected, the debug computer 202 provides the field operator with the option to train an AI model (typically a neural network) via a training pipeline 218. During the debug phase, a pre-trained AI model is used, where iterative training and testing are performed via the debug computer 202 to adjust the pre-trained AI model according to a specific sensor 206 and a specific physical environment 204. The result of the training process is a trained AI model 220, configured to predict classification labels from input sensor data from sensor 206. The debug computer 202 can provide a training monitoring interface, allowing the operator to monitor the training process and determine when to stop it. For example, the training process can be stopped when the total loss function drops below a defined threshold. In implementations, the debug computer 202 may be equipped with sufficient computing resources, such as one or more graphics processing units (GPUs), for efficiently training the neural network. In an exemplary assembly line use case, an advanced GPU might require approximately 1-2 hours to train a neural network (e.g., Tiny-YOLO) to detect product defects using approximately 200-300 labeled data samples with reasonable accuracy.

[0030] In some implementations, the trained AI model can be converted (box 222) to a format compatible with field device 208 before testing. For example, if field device 208 includes an NPU chip, the trained AI model can be converted to a supported NPU format (e.g., by Intel). TM The manufactured NPU chip and OpenVino TM (Framework compatibility). Model conversion improves the reliability of model testing by ensuring that the AI ​​model is tested using the debugging computer 202 in the same format that is subsequently executed by the field device 208.

[0031] The trained (and transformed) AI model can be tested by launching the AI ​​model's inference engine on debug computer 202 and providing it with real-time sensor data captured by sensor 206 located within the same physical environment 204, along with the same fixed settings as during data collection (box 224). The test result is considered satisfactory if the inferences generated by the AI ​​model regarding the input image (e.g., predicted bounding boxes with classification labels) are consistent with those of the field operator / domain expert. One or more success criteria can be defined to determine whether the AI ​​model needs to be retrained. Examples of success criteria may include, for example, success rate (percentage of correct inferences), confidence level of inference, etc. The test results can be monitored for a predetermined number of samples (e.g., 10-50 samples) to determine if retraining is necessary. If the defined success criteria are not met, subsequent iterations of data collection, training, and testing are performed with different sensor settings, as explained in conjunction with box 212. Additionally, the retraining process may include modifying the data labeling method, such as how bounding boxes are drawn on the image to label the data samples. After iteratively retraining the AI ​​model, the final fixed settings for sensor 206 are determined, which produce satisfactory test results based on defined success criteria. Immediate and on-site feedback enables field operators to retrain and tune the AI ​​model to provide a usable solution within a very short timeframe (e.g., within the same day).

[0032] Once the test results are satisfactory (i.e., the success criteria are met), the AI ​​model is deployed to field device 208. At this point, the commissioning computer 202 can be disconnected from this setup. After deployment, the configuration of sensor 206 remains at the final settings determined during the commissioning phase. The trained AI model is intended to be deployed in the same physical environment as during the commissioning phase, with the same environmental settings (e.g., lighting). To deploy to different production lines (e.g., with different lighting), the commissioning computer 202 can be used to train another AI model for that specific environment. Assuming that the model can be quickly retrained and redeployed, this is still a more efficient solution than the long loop shown in Figure 1.

[0033] Figure 3 An exemplary computing environment is shown, including a computing system 302 in which various aspects of this disclosure can be implemented. For example, the computing system 302 can be used to implement a debug computer 202. Computers and computing environments such as the computing system 302 and the computing environment 300 are known to those skilled in the art and are therefore briefly described herein.

[0034] like Figure 3As shown, the computing system 302 may include a communication mechanism such as a system bus 304 or other communication mechanisms for transmitting information within the computing system 302. The computing system 302 also includes one or more processors 306 connected to the system bus 304 for processing information. The processors 306 may include one or more central processing units (CPUs), graphics processing units (GPUs), or any other processor known in the art.

[0035] The computing system 302 also includes a system memory 308 connected to the system bus 304 for storing information and instructions to be executed by the processor 306. The system memory 308 may include computer-readable storage media in the form of volatile and / or non-volatile memory, such as read-only memory (ROM) 310 and / or random access memory (RAM) 312. The system memory RAM 312 may include other dynamic storage devices (e.g., dynamic RAM, static RAM, and synchronous DRAM). The system memory ROM 310 may include other static storage devices (e.g., programmable ROM, erasable PROM, and electrically erasable PROM). Additionally, the system memory 308 can be used to store temporary variables or other intermediate information during instruction execution by the processor 306. The basic input / output system 314 (BIOS) 302, which contains basic routines for transferring information between elements, may be stored in the system memory ROM 310. The system memory RAM 312 may contain data and / or program modules that are immediately accessible by the processor 306 and / or currently being operated by the processor 306. The system memory 308 may also include, for example, an operating system 316, an application program 318, other program modules 320, and program data 322.

[0036] The computing system 302 also includes a disk controller 324 coupled to the system bus 304 to control one or more storage devices, such as magnetic hard disks 326 and removable media drives 328 (e.g., floppy disk drives, optical disk drives, tape drives, and / or solid-state drives). Storage devices can be added to the computing system 302 using appropriate device interfaces, such as Small Computer System Interface (SCSI), Integrated Device Electronics (IDE), Universal Serial Bus (USB), or FireWire.

[0037] The computing system 302 may also include a display controller 330 connected to the system bus 304 to control a display 332, such as a cathode ray tube (CRT) or liquid crystal display (LCD), for displaying information to a computer user. The computing system 302 includes a user input interface 334 and one or more input devices, such as a keyboard 336 and a clicking device 338, for interacting with the computer user and providing information to one or more processors 306. The clicking device 338 may be, for example, a mouse, light pen, trackball, or pointer, for transmitting directional information and command selections to one or more processors 306 and for controlling cursor movement on the display 332. The display 332 may provide a touchscreen interface that allows input to supplement or replace the communication of directional information and command selections by the clicking device 338.

[0038] The computing system 302 also includes an I / O adapter 346 connected to the system bus 304 to connect the computing system 302 to a controllable physical device, such as a robot. Figure 3 In the example shown, I / O adapter 346 is connected to one or more sensors 348, such as a camera, microphone, vibration sensor, or a combination thereof.

[0039] The computing system 302 may perform some or all of the processing steps of embodiments of the present disclosure in response to one or more processors 306 executing one or more sequences of one or more instructions contained in memory such as system memory 308. Such instructions may be read into system memory 308 from another computer-readable storage medium (magnetic hard disk 326 or removable media drive 328). Magnetic hard disk 326 may contain one or more data memories and data files used in embodiments of the present disclosure. The contents of the data memories and data files may be encrypted to enhance security. Processor 306 may also be used in a multiprocessing device to execute one or more sequences of instructions contained in system memory 308. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions. Therefore, the embodiments are not limited to any particular combination of hardware circuitry and software.

[0040] The computing system 302 may include at least one computer-readable storage medium or memory for storing instructions programmed according to embodiments of the present disclosure and for including data structures, tables, records, or other data described herein. As used herein, the term "computer-readable storage medium" refers to any medium that participates in providing instructions to one or more processors 306 for execution. Computer-readable storage media may take many forms, including but not limited to non-transient, non-volatile, volatile, and transmission media. Non-limiting examples of non-volatile media include optical disks, solid-state drives, magnetic disks, and magneto-optical disks, such as magnetic hard disks 326 or removable media drives 328. Non-limiting examples of volatile media include dynamic memory, such as system memory 308. Non-limiting examples of transmission media include coaxial cables, copper wires, and optical fibers, including conductors constituting system bus 304. Transmission media may also take the form of acoustic or optical waves, such as acoustic or optical waves generated during radio wave and infrared data communication.

[0041] The computing environment 300 may also include a computing system 302 operating in a networked environment using logical connections to one or more remote computers, such as a remote computing device 344. The remote computing device 344 may be a personal computer (laptop or desktop), mobile device, server, router, network PC, peer-to-peer device, or other public network node, and typically includes many or all of the elements described above with respect to the computing system 302. When used in a networked environment, the computing system 302 may include a modem 342 for establishing communication over a network 340, such as the Internet. The modem 342 may be connected to the system bus 304 via a network interface 345 or via another suitable mechanism.

[0042] Network 340 can be any network or system commonly known in the art, including the Internet, intranet, local area network (LAN), wide area network (WAN), metropolitan area network (MAN), direct connection or a series of connections, cellular telephone network, or any other network or medium capable of facilitating communication between computing system 302 and other computers (e.g., remote computing device 344). Network 340 can be wired, wireless, or a combination thereof. Wired connections can be implemented using Ethernet, Universal Serial Bus (USB), RJ-6, or any other wired connection known in the art. Wireless connections can be implemented using Wi-Fi, WiMAX and Bluetooth, infrared, cellular networks, satellite, or any other wireless connection method known in the art. Furthermore, multiple networks can operate independently or communicate with each other to facilitate communication within network 340.

[0043] Embodiments of this disclosure can be implemented using any combination of hardware and software. Furthermore, embodiments of this disclosure can be included in an article of manufacture (e.g., one or more computer program products) having, for example, a non-transient computer-readable storage medium. The computer-readable storage medium contains computer-readable program instructions, for example, mechanisms for providing and facilitating embodiments of this disclosure. This article of manufacture can be included as part of a computer system or sold separately.

[0044] Computer-readable storage media may include tangible devices that can hold and store instructions for use by an instruction execution device. Computer-readable storage media may be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. The computer-readable program instructions described herein can be downloaded from the computer-readable storage media to a suitable computing / processing device, or downloaded via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network) to an external computer or external storage device.

[0045] The systems and processes illustrated in the accompanying drawings are not unique. Other systems, processes, and menus can be derived from the principles of this disclosure to achieve the same purpose. Although this disclosure has been described with reference to specific embodiments, it should be understood that the embodiments and variations shown and described herein are for illustrative purposes only. Modifications to the present design can be made by those skilled in the art without departing from the scope of this disclosure.

Claims

1. A method for debugging an artificial intelligence (AI)-based inspection system, comprising: (i) Sensor data captured by sensors positioned within the physical environment is received by a debugging computer to capture sensor data relating to individual items in a sequence of similar items. (ii) The debugging computer collects multiple data samples, wherein collecting each data sample includes providing an operator interface for the field operator to access sensor data related to a single project and assigning classification labels to it, and the collected data samples correspond to fixed settings of the sensors. (iii) The AI ​​algorithm is trained by the debug computer using the collected data samples to configure the AI ​​algorithm to predict classification labels based on input sensor data. (iv) The debug computer tests the trained AI algorithm by feeding it real-time sensor data captured by the sensors, which are positioned within the physical environment and have the fixed configuration. Steps (ii) to (iv) are performed iteratively using different fixed settings of the sensor to determine the final sensor settings that achieve the defined success criteria during the test. (v) Deploy the iteratively trained and tested AI algorithm to a field device that is connected to a sensor configured with the final setup to provide sensor data as input to the deployed AI algorithm.

2. The method according to claim 1, wherein, The physical environment includes production lines or assembly lines.

3. The method according to any one of claims 1 and 2, wherein, The AI ​​algorithm is pre-trained before being trained by the debugging computer, wherein the iterative training and testing are performed via the debugging computer to adjust the sensors and the physical environment to fit the pre-trained AI algorithm.

4. The method according to any one of claims 1 to 2, wherein, Each fixed configuration of the sensor is defined by a combination of internal sensor configuration and / or the arrangement of the sensor relative to the physical environment.

5. The method according to any one of claims 1 to 2, wherein, The sensor includes a vision camera that can transmit captured sensor data as a video stream to the debugging computer and the field equipment.

6. The method according to claim 5, wherein, The operator interface includes a camera viewer configured to display field sensor data to the field operator, and the method further includes the field operator adjusting the current settings of the sensors based on the displayed field sensor data before collecting the data samples.

7. The method according to claim 5, wherein, The operator interface includes a frame grabber that captures still images from the video stream, and the operator interface is configured to enable the field operator to draw bounding boxes around regions of interest in the captured images and mark the captured images.

8. The method according to any one of claims 1 to 2, wherein, The sensor includes an acoustic sensor capable of transmitting captured sensor data as an audio stream to the debugging computer and the field equipment.

9. The method according to any one of claims 1 to 2, wherein, The sensor includes a vibration sensor.

10. The method according to any one of claims 1 to 2, comprising storing the collected data samples in a local storage medium of the debugging computer.

11. The method according to any one of claims 1 to 2, wherein, The training and testing of the AI ​​algorithm are performed via the graphics processing unit (GPU) of the debugging computer.

12. The method according to any one of claims 1 to 2, further comprising converting the trained AI algorithm into a format compatible with the field device by the debugging computer, such that the debugging computer performs tests according to the converted format of the AI ​​algorithm.

13. The method according to any one of claims 1 to 2, wherein, The field device includes a neural processing unit (NPU).

14. A non-transient computer-readable storage medium comprising instructions that, when processed by a computing device, configure the computing device to perform the method according to any one of claims 1 to 13.

15. A computing device for debugging an artificial intelligence (AI)-based inspection system, comprising: processor; as well as A memory for storing instructions that, when executed by the processor, cause the computing device to: (i) Receive sensor data captured by sensors positioned within a physical environment to capture sensor data relating to individual items in a sequence of similar items. (ii) Collecting multiple data samples, wherein collecting each data sample includes providing an operator interface for field operators to access sensor data for each individual item and assigning classification labels to it, the collected data samples corresponding to fixed settings of the sensors. (iii) Using collected data samples to train an AI algorithm to configure the AI ​​algorithm to predict classification labels based on input sensor data. (iv) The trained AI algorithm is tested by feeding it real-time sensor data captured by the sensor, which has the fixed configuration and is positioned within the physical environment. Steps (ii) to (iv) are performed iteratively using different fixed settings of the sensor to determine the final sensor settings that achieve the defined success criteria during the test. (v) Deploy the AI ​​algorithm, which has been iteratively trained and tested, to a field device connected to the sensor configured with the final setup, so as to provide sensor data as input to the deployed AI algorithm.

Images