structural detection

Abstract

The present disclosure provides a method for detecting structures. The method can include receiving inspection image data characterizing a region of interest of an object being inspected. The region of interest can include one or more structures of the object. The method can also include determining the structures within the region of interest using a computer vision algorithm for luminosity properties of pixel data in the inspection image data. The structures can be determined using a prediction model trained to determine image filter parameter values of an image filter of the computer vision algorithm based on applying an optimization technique using training image data and annotation data. An indication of the structures can be provided, e.g., for display or storage in a memory. Systems and computer readable media implementing the method are also provided.

Description

BACKGROUND

[0001] Object inspection can be performed using image data to assess structures present in the object and the quality or defects of the object. Rapid and accurate characterization of the structures, quality, and defects of an object can reduce operational costs and improve object quality and / or reliability in manufacturing and industrial operations. Computer-implemented inspection systems configured to process inspection image data can reduce inspection time and further improve inspection quality.

[0002] Machine learning is an application of artificial intelligence that automates the development of analytical models through the use of algorithms that iteratively learn patterns from data without explicit indication of the data patterns. Machine learning can enable the construction of a training model or algorithm that can accurately learn from data and can be deployed for data-driven predictions or decisions. SUMMARY

[0003] In an aspect, a method for detecting structures in inspection image data is provided. In one embodiment, the method can include receiving, by a data processor, first image data acquired during an inspection of an object. The first image data can characterize a region of interest of the object. The region of interest can contain one or more structures of the object. The method can further include determining, using the data processor and a computer vision algorithm, at least one structure of the one or more structures within the region of interest. The computer vision algorithm can include a plurality of image filters configured to filter the first image data with respect to luminosity characteristics of a plurality of pixels of the first image data. Each image filter of the plurality of image filters can include one or more image filter parameters. The method can further include providing an indication of the at least one structure in second image data.

[0004] In another embodiment, determining the at least one structure within the region of interest can further include receiving, using the data processor, training image data of the object. The training image data can include at least one first annotation that identifies at least one structure of the object present in at least one region of interest in the training image data. Determining the at least one structure within the region of interest can further include determining, using the data processor and a prediction model, at least one set of image filter parameter values associated with at least one image filter parameter of the plurality of image filters. The prediction model can be trained to receive the training image data including the at least one first annotation and determine the at least one set of image filter parameter values for each image filter parameter of the plurality of image filters based on applying at least one optimization technique using the training image data and the at least one first annotation. Determining the at least one structure within the region of interest can further include updating the at least one image filter of the computer vision algorithm based on the at least one set of image filter parameter values.

[0005] In another embodiment, the at least one optimization technique can include a grid search technique, a random search technique, a Levenberg-Marquardt technique, a gradient search technique, a Bayesian optimization technique, or a combination of two or more optimization techniques. In another embodiment, the prediction model can be trained to determine the at least one set of image filter parameter values for at least one of an average profile image filter parameter, a line tracking image filter parameter, or a stop criterion image filter parameter.

[0006] In another embodiment, in response to providing the indication of the at least one structure in the second image data, the method can further include modifying the indication in the second image data with the at least one annotation and including the second image data in the training image data. The second image data can include an annotation of the at least one structure. In response to providing the indication of the at least one structure in the second image data, the method can further include characterizing the region of interest of the object. The characterizing can include characterizing a defect in the region of interest or characterizing the region of interest as defective. In response to providing the indication of the at least one structure in the second image data, the method can further include determining the at least one set of image filter parameter values based on training the prediction model using the training image data including the second image data and the annotation of the at least one structure.

[0007] In another embodiment, the computer vision algorithm can be configured to generate profile data associated with a portion of the plurality of pixels. The portion can correspond to the at least one structure present within the region of interest and the profile data can include at least one of photometric data, tip detection data, particle detection data, or void detection data. In another embodiment, the first image data can include two-dimensional computed tomography image data, three-dimensional computed tomography image data, or x-ray radiographic data.

[0008] In another embodiment, the object can be a lithium-ion battery, a nickel-metal hydride battery, or a nickel-cadmium battery. In another embodiment, the region of interest can include at least one of a cathode of the battery, an anode of the battery, a casing of the battery, a volume between the cathode of the battery and the anode of the battery, a centerline of the cathode of the battery, a centerline of the anode of the battery, and a contour of a shape of the battery.

[0009] In another embodiment, the method can include determining, using the data processor, at least one of a position of the object within the region of interest or an alignment of the one or more portions of the first image data. The one or more portions can include the region of interest characterizing the object. The method can further include providing an indication of the position of the object within the region of interest or the alignment of the one or more portions of the first image data.

[0010] A non-transitory computer program product (i.e., physically embodied computer program product) storing instructions that, when executed by one or more data processors of one or more computing systems, cause at least one data processor to perform operations herein is also described. Similarly, a computer system is also described that can include one or more data processors and a memory coupled to the one or more data processors. The memory can temporarily or permanently store instructions that cause at least one processor to perform one or more of the operations described herein. Additionally, methods can be implemented by one or more data processors either within a single computing system or distributed among multiple computing systems. Such computing systems can be connected by one or more connections, which can be either direct connections or indirect connections such as can be found through a network (e.g., the Internet, a wide area network, a local area network, a wide area network, a wired network, etc.), and can exchange data and / or commands or other instructions through the one or more connections, in addition to other possible data and / or commands or other instructions. The one or more connections can each be one or more of a number of connections including wireless connections, electrical connections, optical connections, and the like. BRIEF DESCRIPTION OF DRAWINGS

[0011] These and other features will be more readily understood from the following detailed description, taken in conjunction with the accompanying drawings, in which:

[0012] Figure 1 An example embodiment of a system for detecting structures in examination image data in accordance with the subject matter described herein is shown;

[0013] Figure 2 Components of one or more apparatuses and / or one or more systems in accordance with the subject matter described herein are shown; Figure 1

[0014] Figure 3 is a flowchart showing one example embodiment of a method for detecting structures in examination image data in accordance with the subject matter described herein;​

[0015] Figure 4 is a flowchart illustrating one exemplary embodiment of a method for determining at least one structure within a region of interest included in examination image data according to the subject matter described herein;

[0016] Figure 5 is an image of examination image data for detecting a structure therein according to the subject matter described herein;

[0017] Figure 6 is an image of examination image data including an indication of a structure provided according to the subject matter described herein;

[0018] Figure 7 is an image of examination image data of a battery including an indication of a cathode determined according to the subject matter described herein;

[0019] Figures 8A-8C is an image of examination image data of a battery for detecting a cathode therein according to the subject matter described herein;

[0020] Figure 8D is a plot of cathode luminance data according to the subject matter described herein;

[0021] Figures 9A-9C is an image of examination image data of a battery for detecting an anode therein according to the subject matter described herein; and

[0022] Figure 9D is a plot of cathode luminance data according to the subject matter described herein.

[0023] It should be noted that the figures shown are not necessarily drawn to scale. The figures are merely intended to depict typical aspects of the subject matter disclosed herein, and therefore should not be considered to limit the scope of the disclosure. DETAILED DESCRIPTION

[0024] Executable object inspections can be performed to assess the quality of an object, such as defects that can be included on or within the object. Inspection image data can be evaluated to determine structures on or within the object and defects of the structures. Computer vision algorithms can be employed to process the inspection image data and determine aspects of the inspected object or structures thereof. Computer vision algorithms (CVAs) can implement image filters that can be used to characterize the inspection image data. Typically, CVAs include algorithms that are configured to implement the image filters as formulas using source code. Various parameters of the image filters can be fixed or pre-computed to determine one or more structures in the image data to which the CVA is applied. For more precise and accurate structure detection, the parameters of the CVA must be manually tuned or adjusted by a user, which can be time consuming and require specialized human resources. The additional time can be multiplied because the parameters can cross-influence each other. Additionally, additional tuning time can be required because parameter value determinations can be influenced by the type of input and can depend on the object type and quality of the inspection image data. This problem can require additional time and resources when tuning the CVA for a new object because existing CVAs can need to be retrained.

[0025] Structure detection using CVAs implemented as prediction models in machine learning processes utilizing deep learning multi-layer artificial neural networks can require large amounts of training data. Each instance of the training data must be accurately manually labeled. This approach can also be very time consuming and can require additional variations of labeled training data to adequately train the prediction model for new object types or inspection types.

[0026] Accordingly, there is a need for an efficient and less time consuming way to accurately perform structure detection using inspection image data by a CVA. Rather than implementing the CVA as a prediction model in a machine learning process, the systems, methods, and computer readable media provided herein can implement a prediction model trained to produce one or more image filter parameter values for the CVA. This approach can reduce training time, reduce the need for experienced personnel, and can be more fully automated than a CVA implemented entirely in a machine learning process. For example, a smaller amount of inspection image data can be used to train the prediction model described herein, and the inspection image data used for training can include a greater variety of structure types, regions of interest, and qualities. Further, the training data can be labeled by less skilled users.

[0027] Another benefit is that the predictive model configured to determine image filter parameter values for the CVA can include an automatic parameter discovery routine that can estimate parameter values for various parameters associated with a plurality of image filters implemented by the CVA. The automatic parameter discovery routine can automatically vary the parameters, compare training data labels to observed structures separately for different structure types, and can select optimal parameter values. The automatic parameter discovery routine can operate iteratively. Once found, the optimal parameter values can be saved and automatically applied during examination of other structures, other objects, or in other regions of interest associated with the same structure or object. Such automatic operation can increase examination time without requiring human resources.

[0028] Once the predictive model has been trained and deployed, the predicted image filter parameter values can be applied to the CVA for performing analysis of new examination image data, and structures can be determined in the new examination image data. The structures and indications of the structures can be provided to a user via a display. The user can review the indications, and in some embodiments, the user can adjust the indications such that the annotated examination image data can be provided back to the machine learning process as training data to produce new image filter parameter values based on the refined examination image data. Advantageously, the feedback of the refined examination image data enables faster retraining and more accurate generation of image filter parameter values that can be used for examination of new objects or structures.

[0029] Figure 1 FIG. 1 is a schematic diagram illustrating an exemplary architecture of a system 100 for detecting structures in examination image data and providing indications of the detected structures. The system 100 includes a client 105, a database 110, and a server 115 that can be communicatively coupled through a network.

[0030] As Figure 1As shown, the system 100 includes clients 105, e.g., clients 105A-105C. For example, the client 105A can include a laptop computer configured with a web browser to display 2D or 3D inspection data acquired during an inspection of an object, such as a battery. The client 105B can include training image data used in a machine learning process to train a prediction model configured to generate image filter parameters for a computer vision algorithm based on annotations in the training image data. The computer vision algorithm can be configured to receive inspection image data and determine an indication of a structure that can be present within a region of interest of the inspection image data. The client 105C can include optimized image filters, optimized image filter parameters, and optimized image filter parameter values that can be further used in a machine learning process to train a prediction model or can be used to improve the operation of a computer vision algorithm to detect a structure of an object being inspected that is captured in inspection image data. The inspection image data, training data, annotation data, image filter data, image filter parameter data, image filter parameter value data, and data associated with one or more optimization techniques can be stored in a database, e.g., database 110, and can be transmitted to a prediction server 115.

[0031] As further shown in Figure 1 The inspection image data can be transmitted from the clients 105 and / or from the database 110 to the prediction server 115, as further shown in

[0032] In some embodiments, the inspection image data includes prediction data 125 that can be transmitted as input to the prediction server 115 to a computer vision algorithm that can be configured with image filter parameter values produced by a trained prediction model 145 that was trained using the training input 120 in a machine learning process.

[0033] The examination image data can include computed tomography (CT) data associated with the object being examined. In some embodiments, the examination image data can include X-ray data. The object can include a structure found within a region of interest of the examination image data. For example, in the case of a battery, the examination image data can include a structure such as an electrode, which can include a cathode, an anode, or other similar portion of the battery. In some embodiments, the examination image data can include one or more slices of image data of the object being examined. In some embodiments, the structure can be a surface structure of the object being examined. In some embodiments, the structure can include a linear structure. The examination image data can include two-dimensional (2D) examination image data, three-dimensional (3D) examination image data, and four-dimensional (4D) examination image data.

[0034] The examination image data can include one or more irregularities of the structure. For example, the examination image data can include a void, a hole, or a particle of the structure. The irregularity can be, for example, a crack or a deformation in the structure. The void can be, for example, a bubble. The hole can be, for example, a dent, a depression, or a surface anomaly of the structure. The particle can be, for example, a piece of a first material included in a second material of the structure.

[0035] The client device 105 can be coupled to the prediction server 115 via a network. The network can include, for example, any one or more of a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network, the Internet, and the like. Further, the network can include, but is not limited to, any one or more of the following network topologies, including a bus network, a star network, a ring network, a mesh network, a star-bus network, tree or hierarchical network, and the like.

[0036] As Figure 1As shown, system 100 includes a prediction server 115 to receive inspected image data and generate image filter parameter values ​​150 for use in a computer vision algorithm 155, which can be applied to the inspected image data to determine indications 160 of structure in objects included in the inspected image data. In summary, prediction server 115 plays a role in the training aspect of a machine learning process by receiving inspected image data as training input and generating a trained model 140 for predicting image filter parameter values ​​used in computer vision algorithm 155 to determine the structure and structural indications 160 of objects included in the prediction data 125. Prediction server 115 includes a feature selector 130 for selecting a subset of features from the inspected image data during the training aspect of the machine learning process. The prediction server 115 also includes a model trainer 135 that uses a selected machine learning algorithm to process a selected subset of features as input and generate a new training model 140 that can be subsequently used outside the machine learning process to predict image filter parameter values ​​150 that are used in a computer vision algorithm 155 to detect and determine structures 160 in the examined image data received as prediction data 125.

[0037] like Figure 1 As shown, prediction server 115 includes feature selector 130. During the training aspect of the machine learning process, feature selector 130 receives examination image data and selects a subset of features from the examination image data. This subset of features is used as training input to train a selected machine learning algorithm to produce image filter parameter values ​​associated with one or more image filter parameters used in computer vision algorithm 155. For each selected subset of features in training input 120, the selected machine learning algorithm can be trained to predict image filter parameter values ​​associated with the feature subset, for which the selected machine learning algorithm is trained. The trained machine learning algorithm can then be output as a new trained model (e.g., training model 140), which can then be subsequently applied to examination image data (e.g., prediction data input 125) to determine the structure and structural indication 160 corresponding to objects included in the examination image data 125.

[0038] During the machine learning process, the feature selector 130 provides the selected subset of features to the model trainer 135 as input to a machine learning algorithm to generate one or more trained models in a machine learning process utilizing a deep learning multi-layer artificial neuron network. A variety of machine learning algorithms can be selected for use, including algorithms such as: support vector regression, ordinary least squares regression (OLSR), linear regression, logistic regression, stepwise regression, multivariate adaptive regression splines (MARS), locally estimated scatterplot smoothing (LOESS), sequential regression, Poisson regression, random forest quantile regression, Bayesian linear regression, neural network regression, decision forest regression, boosted decision tree regression, artificial neural networks (ANN), Bayesian statistics, case-based reasoning, Gaussian process regression, inductive logic programming, learning automata, learning vector quantization, informal fuzzy networks, conditional random fields, genetic algorithms (GA), information theory, support vector machines (SVM), average one-dependence estimators (AODE), group method of data handling (GMDH), instance-based learning, negative case learning, and maximum information gain tree (MIST).

[0039] The prediction server 115 also includes a model trainer 135. In some embodiments, the model trainer 135 can be included in the prediction server 115. In other embodiments, the model trainer 135 can be located remotely from the prediction server 115. During the training aspect of the machine learning process, the model trainer 135 receives training input including the selected subset of features from the feature selector 130 and iteratively applies the subset of features to a previously selected machine learning algorithm to evaluate the performance of the algorithm. As the machine learning algorithm processes the training input, the model trainer 135 learns patterns in the training input that map machine learning algorithm variables to target output data (e.g., predicted action sequences) and generates a trained model that captures these relationships. For example, as shown in Figure 1 The model trainer 135 outputs a trained model 140. As further shown in Figure 1 The output trained model 140 can be a trained prediction model 145, as further shown in

[0040] As further shown in Figure 1As further shown, the prediction server 115 includes a trained prediction model 145. The trained prediction model 145 is a model or algorithm generated as a result of model training performed during a training aspect of a machine learning process. Once trained, the trained prediction model 145 can operate outside of the machine learning process to receive inspection image data as prediction data 125 and determine image filter parameter values 150 that can be applied to a computer vision algorithm 155. The computer vision algorithm 155 can be configured to generate indications of structures 160 and provide them for visualization on a display of the client 105, such as the client 105C. The indications of structures 160 can also be provided for data storage, such as to the database 110. For example, the trained prediction model 145 outputs indications of electrode structures of a battery in a region of interest of the inspection image data received by the prediction server 115 as prediction data 125 from the client 105A.

[0041] The structure indications 160 can include various data or graphical indications associated with structures of an object being inspected. In some embodiments, the object can be an electrical component, such as a battery. For example, the object can be a lithium-ion battery, a nickel-metal hydride battery, or a nickel-cadmium battery. The indications 160 can include one or more structures, portions of structures, regions of interest, defects, locations, or alignments of structures. In some embodiments, the regions of interest can include a cathode of a battery, an anode of a battery, a casing of a battery, a volume between a cathode of a battery and an anode of a battery, a centerline of a cathode of a battery, a centerline of an anode of a battery, and an outline of a shape of a battery. A variety of non-limiting objects and structures included in objects are contemplated for inspection using the subject matter described herein. In some embodiments, the indications 160 can be provided as overlay data placed over image data that includes at least a portion of a structure and an object. In some embodiments, the indications 160 can include alerts, notifications, measurement data, annotation data, or graphical cues that identify structures.

[0042] Referring now to Figure 2 , a schematic diagram of an apparatus 200 is shown. In some embodiments, the apparatus 200 can correspond to the client apparatus 105 described with respect to Figure 1 In some embodiments, the apparatus 200 can correspond to the server 115 described with respect to Figure 1 In some embodiments, the system 100 shown in Figure 1 may be configured in the apparatus 200. As shown in Figure 2 , the apparatus 200 includes a processor 204, a memory 206, a storage component 208, an input interface 210, an output interface 212, a communication interface 214, and a bus 202. As shown in Figure 2As shown, the device 200 includes a bus 202, a processor 204, a memory 206, a storage component 208, an input interface 210, an output interface 212, and a communication interface 214.

[0043] The bus 202 includes a component that permits communication among the components of the device 200. In some embodiments, the processor 204 is implemented in hardware, software, or a combination of hardware and software. In some examples, the processor 204 includes a processor (e.g., a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), etc.), a microphone, a digital signal processor (DSP), and / or any processing component that can be programmed to perform at least one function (e.g., a field-programmable gate array (FPGA), an application- specific integrated circuit (ASIC), etc.). The memory 206 includes a random access memory (RAM), a read only memory (ROM), and / or another type of dynamic or static storage (e.g., a flash memory, a magnetic storage, an optical storage, etc.) that stores data and / or instructions for use by the processor 204.

[0044] The storage component 208 stores data and / or software related to the operation and use of the device 200. In some examples, the storage component 208 includes a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optic disk, a solid state disk, etc.), a compact disc (CD), a digital versatile disc (DVD), a floppy disk, a cartridge, a magnetic tape, a CD-ROM, a RAM, a PROM, an EPROM, a FLASH-EPROM, NV-RAM, and / or another type of computer- readable medium, along with a corresponding drive.

[0045] The input interface 210 includes a component that permits the device 200 to receive information, such as via user input (e.g., a touch screen display, a keyboard, a keypad, a mouse, a button, a switch, a microphone, a camera, etc.). The output interface 212 includes a component that provides output information from the device 200 (e.g., a display, a speaker, one or more light-emitting diodes (LEDs), etc.).

[0046] In some embodiments, the communication interface 214 includes a transceiver-like component (e.g., a transceiver, a separate receiver and transmitter, etc.) that enables the device 200 to communicate with other devices, such as via a wired connection, a wireless connection, or a combination of wired and wireless connections. In some examples, the communication interface 214 enables the device 200 to receive information from another device and / or provide information to another device. In some examples, the communication interface 214 includes an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi® interface, a Bluetooth® interface, a Zigbee® interface, a near-field communication (NFC) interface, a Global Positioning System (GPS) interface, a

[0047] ​In some embodiments, the device 200 performs one or more processes described herein. The device 200 performs these processes based on the processor 204 executing software instructions stored by a computer-readable medium, such as the memory 205 and / or the storage component 208. A computer-readable medium (e.g., a non-transitory computer-readable medium) is defined herein as a non-transitory memory device. A non-transitory memory device includes memory space located inside of a single physical storage device or memory space spread across multiple physical storage devices.

[0048] In some embodiments, the software instructions are read into the memory 206 and / or the storage component 208 from another computer-readable medium or from another device via the communication interface 214. When executed, the software instructions stored in the memory 206 and / or the storage component 208 cause the processor 204 to perform one or more processes described herein. Additionally or alternatively, hardwired circuitry is used in place of or in combination with software instructions to perform one or more processes described herein. Thus, embodiments described herein are not limited to any specific combination of hardware circuitry and software, unless otherwise explicitly stated.

[0049] The memory 206 and / or the storage component 208 includes a data store or at least one data structure (e.g., a database, etc.). The device 200 is capable of receiving information from, storing information in, communicating information to, or searching information stored in the data store or at least one data structure in the memory 206 or the storage component 208. In some examples, the information includes network data, input data, output data, or any combination thereof.

[0050] In some embodiments, the device 200 is configured to execute software instructions stored in the memory 206 and / or a memory of another device (e.g., another device that is the same as or similar to the device 200). As used herein, the term “module” refers to at least one instruction stored in the memory 206 and / or a memory of another device that, when executed by the processor 204 and / or a processor of another device (e.g., another device that is the same as or similar to the device 200), causes the device 200 (e.g., at least one component of the device 200) to perform one or more processes described herein. In some embodiments, a module is implemented in software, firmware, hardware, etc.

[0051] Figure 2 The number and arrangement of components shown in FIG. 1 are provided as an example. In some embodiments, devices 200 can have additional components, can lack some of the components shown in FIG. 1, or can have a different arrangement of the components shown in FIG. 1. Figure 2The device 200 can include additional components, fewer components, different components, or differently arranged components than those shown. Additionally or alternatively, a set of components (e.g., one or more components) of the device 200 can perform one or more functions described as being performed by another component or set of components of the device 200.

[0052] Figure 3 is a flowchart illustrating an exemplary embodiment of a process 300 for detecting structures in inspection image data using a relative Figure 1 The system 100 shown and described can be used to detect structures in inspection image data. In certain aspects, embodiments of the process 300 can include more or fewer operations than those shown and can perform the operations in different orders than those shown. Figure 3 Figure 3 The operations shown and described can be performed in different orders than those shown.

[0053] In operation 305, the data processor can receive first image data acquired during an inspection of an object. The inspection can be performed using X-ray or computed tomography, and the object can be any object for which internal structures can be inspected. The first image data can include multi-dimensional inspection image data, such as 2D inspection image data, 3D inspection image data, 4D inspection image data, and the like. The inspection image data can include linear data, surface data, and volumetric data.

[0054] The first image data can characterize a region of interest of the object, and the region of interest can include one or more structures. For example, the object can be a battery, and the image data can include an electrode of the battery, such as an anode or a cathode. In some embodiments, the battery can include a lithium-ion battery, a nickel-metal hydride battery, or a nickel-cadmium battery. The region of interest can include a cathode of the battery, an anode of the battery, a casing of the battery, a volume between the cathode of the battery and the anode of the battery, a centerline of the cathode of the battery, a centerline of the anode of the battery, and an outline of a shape of the battery.

[0055] At operation 310, the data processor and the CVA can determine at least one structure of the one or more structures present within the region of interest. The CVA can be configured with a plurality of image filters that can filter the first image data with respect to luminosity characteristics of a plurality of pixels included in the first image data. Each image filter can include one or more image filter parameters.

[0056] ​The CVA can be configured to generate profile data that can be associated with a portion of the plurality of pixels included in the first image data. The portion can correspond to a structure present within the region of interest, such as a cathode or an anode of a battery being inspected. The profile data can include photometric data, tip detection data, particle detection data, or void detection data. For example, the photometric data can include grayscale data corresponding to an amount of light intensity (e.g., brightness) or light flux of the pixels. The tip detection data can include profile data derived from the plurality of pixel data and tip location data derived from the profile data. The particle detection data can include profile area data, particle area data, and mass center data. The void detection data can include profile area data, void area data, and mass center data.

[0057] At operation 315, the data processor can determine a location of the object within the region of interest. A location of the structure within the region of interest can also be determined. For example, the location of the object or structure can be determined relative to a coordinate value system corresponding to the pixel locations of the first image data.

[0058] At 320, the data processor can alternatively determine an alignment of one or more portions of the first image data. For example, an alignment of a terminal portion of an electrode relative to a housing of a battery including the electrode can be determined. The alignment can be determined relative to a coordinate value system associated with the pixel locations of the first image data.

[0059] At 325, the data processor can provide the location of the object determined at 315 or the alignment of the portions of the first image data determined at 320. The location or alignment can be provided via a display. In some embodiments, the location or alignment can be provided as a numerical value and stored in a memory or database.

[0060] At 330, the data processor can provide an indication of at least one structure in the second image data. The indication can include the structure, the region of interest, and the object. In some embodiments, the indication can include an overlay that labels the structure. The second image data can include multi-dimensional image data, such as 2D image data, 3D image data, 4D image data, etc. The second image data can include linear data, surface data, and volumetric data.

[0061] Figure 4 FIG. 4 is a flowchart illustrating an example embodiment of a process 400 for determining at least one structure within a region of interest included in first image data (e.g., inspection image data). The process 400 can correspond to the operation 310 described and can be performed using the system 100 as shown and described with respect to FIGS. 1-3. Figure 3 The process 400 can be performed in accordance with the operation 310 described and can be performed using the system 100 as shown and described with respect to FIGS. 1-3. Figure 1 In certain aspects, embodiments of the process 400 can include fewer steps, additional steps, or different steps than those shown in FIG. 4. Figure 4more or fewer operations are shown, and operations can be performed in an order other than the Figure 4 The process 400 can be configured in machine learning processes described herein to generate image filter parameter values based on training data that includes inspection image data. The generated image filter parameter values can be applied to a CVA, such as the CVA used in operation 310, to determine an indication of a structure within an inspected object for which the inspection image data has been received.

[0062] At 405, the data processor can receive training image data of an object. The training image data can include at least one first annotation that identifies at least one structure of the object present in at least one region of interest in the training image data. The at least one first annotation can be applied manually and can indicate the structure. For example, the annotation can identify a centerline of an electrode, a casing of a battery, an anode, a cathode, or a terminal of a battery or electrode. The training image data can also include or otherwise characterize a region of interest of the object, and the region of interest can include one or more structures. For example, the object can be a battery, and the image data can include an electrode of the battery, such as an anode or a cathode. In some embodiments, the battery can include a lithium-ion battery, a nickel-metal hydride battery, or a nickel-cadmium battery. The region of interest can include a cathode of the battery, an anode of the battery, a casing of the battery, a volume between the cathode of the battery and the anode of the battery, a centerline of the cathode of the battery, a centerline of the anode of the battery, and an outline of a shape of the battery.

[0063] At 410, the data processor and the prediction model can determine at least one set of image filter parameter values associated with at least one image filter parameter configured in a plurality of image filter parameters implemented in the CVA used in operation 310. The prediction model can be trained to receive trained image data including annotations corresponding to the trained image data and a structure within an object. The prediction model can be trained to determine the set of image filter parameter values based on applying an optimization technique using the training image data and the annotations. For example, the prediction model can be trained to determine image filter parameter values of parameters associated with an average outline image filter. The image filter parameters can include an average outline image filter, a line tracking image filter, a stopping criterion image filter, a 2D region structure image filter, a 2D center region structure image filter, a 3D surface patch image filter, a 3D center surface patch image filter, a 3D volume structure image filter, and a 3D center volume structure image filter.

[0064] In some embodiments, the set of image filter parameter values can be determined as a set of convolution filter parameters that can be configured by an optimization technique to maximize a convolution result when applying a convolution operation of the image filter to a plurality of pixels of the training image data. The set of image filter parameter values are determined, for example, to maximize the convolution result in a centerline of one of the cathode or anode structures and to minimize the convolution result near the centerline of one of the cathode or anode structures. The set of convolution filter parameters can include a set of one-dimensional parameter values or a two-dimensional array of parameter values.

[0065] In some embodiments, the prediction model can be trained to determine image filter parameter values for parameters associated with a mean profile image filter. The mean profile image filter can be used to determine a location of the anode centerline or the cathode centerline by applying a normalized cross correlation (NCC) operation or a mean square difference (MSD) operation to a profile derived from a plurality of pixels in the inspection image data and the mean profile data.

[0066] In some embodiments, the prediction model can be trained to determine image filter parameter values for parameters associated with a line tracking image filter. The line tracking image filter can be used to iteratively determine a plurality of pixels belonging to the cathode centerline or the anode centerline by applying a convolution image filter or a mean profile image filter to a next predicted location on the centerline and thus determine a next pixel of the plurality of pixels belonging to the cathode centerline or the anode centerline based on the filter output.

[0067] In some embodiments, the prediction model can be trained to determine image filter parameter values for parameters associated with a stopping criteria image filter. The stopping criteria image filter can be configured to use tip detection data to determine a tip location and thus stop the iterative operation of the line tracking image filter.

[0068] In some embodiments, the prediction model can be trained to determine image filter parameter values for parameters associated with a 2D region structure image filter. The 2D region structure image filter can be configured to determine a region of a 2D structure, such as a void or an inclusion, in 2D examination image data. In some embodiments, the prediction model can be trained to determine image filter parameter values for parameters associated with a 2D center region structure image filter. The 2D center region structure image filter can be configured to determine a center point location of a region of a 2D structure in 2D examination image data. In some embodiments, the prediction model can be trained to determine image filter parameter values for parameters associated with a 3D surface patch image filter. The 3D surface patch image filter can be configured to determine a surface portion that describes an interface between an object and air or between two different materials of an object. In some embodiments, the prediction model can be trained to determine image filter parameter values for parameters associated with a 3D center surface patch image filter. The 3D center surface patch image filter can be configured to determine a point on an interface between an object and air or between two different materials of an object. In some embodiments, the prediction model can be trained to determine image filter parameter values for parameters associated with a 3D volume structure image filter. The 3D volume structure image filter can be configured to determine a volume of a 3D structure, such as a void or an inclusion, in 3D examination image data. In some embodiments, the prediction model can be trained to determine image filter parameter values for parameters associated with a 3D center volume structure image filter. The 3D center volume structure image filter can be configured to determine a center point location of a volume of a 3D structure in 3D examination image data.

[0069] The optimization technique can include a grid search technique, a random search technique, a Levenberg-Marquardt technique, a gradient search technique, a Bayesian optimization technique, or a combination of two or more optimization techniques.

[0070] In response to providing the indication of the structure in the second image data at operation 330, the indication can be used in a feedback loop to refine the training image data. For example, the indication provided in the second image data can be modified with the at least one annotation. The modification can be provided with respect to the previously determined structure and with respect to the additional or new structure annotation. The second image data can then be included in the training image data. The second image data can include the annotation of the at least one structure. The region of interest of the object can then be characterized. For example, the characterization can include identifying or characterizing a defect in the region of interest. The defect can include a pit, a void, a burr, a crack, a particle, or a foreign substance, among others. In some embodiments, the characterization can include characterizing the region of interest as defective. For example, the region of interest can be characterized as defective based on the presence of the defect. The set of image filter parameters determined at operation 410 can be determined based on training a prediction model using the training image data including the second image data and the structure annotation.

[0071] At 415, at least one image filter of the CVA can be updated based on the set of image filter parameter values determined in operation 410. In this way, a feedback loop is created to improve the accuracy of the CVA based on the results of the trained prediction model.

[0072] Figure 5 is an image of inspection image data 500 for detecting a structure therein according to the subject matter described herein. The inspection image data 500 can include CT slice (e.g., 2D) data of an object including one or more structures. In Figure 5 In the illustrated embodiment, the object can include a battery, and the one or more structures of the object can include an anode and a cathode of the battery. As Figure 5 illustrated in the middle, the inspection image data can include a region of interest 505. Within the region of interest 505, there can be a cathode 510 and an anode 515. In some embodiments, the region of interest 505 or one of the structures can be defective. For example, as Figure 5 illustrated, an end 520 of the anode 515 can be bent and thus determined to be defective.

[0073] Figure 6 is an image of inspection image data 600 including an indication of a structure provided according to the subject matter described herein. The inspection image data 600 can correspond to Figure 5 the inspection image data 500 of FIG. 5, and can also include one or more indications of the structure present in the region of interest within the object. The indications can be generated by the CVA (e.g., CVA 155) described herein for which the image filter parameter values can be generated via a prediction model in a machine learning process. For example, as Figure 6As shown, the region of interest 605 may include an indicator 610 that identifies the center line of the cathode. Indicator 615 may identify the center line of the anode. Indicator 620 may indicate a defective anode.

[0074] Figure 7 It is an image of inspection image data 700 including the region of interest 705 of the battery 710, the inspection image data including an indication of the cathode determined according to the subject matter described herein. Figure 7 As shown, the region of interest 705 includes a portion of the battery 710. The battery 710 includes a cathode 715 and an anode 720. An indicator 725 is provided to identify the centerline of one of the cathodes. The image data 700 may also include a housing or casing 730 of the battery 710.

[0075] Figures 8A-8C These are images of inspection image data for detecting the cathode of a battery, based on the subject described in this article. (Example) Figures 8A-8C As shown, image data 800, 805, and 810 are provided and can correspond to relative to Figure 7 The described inspection image data 700. Inspection image data 800, 805, and 810 may correspond to contour data of the cathode included in each of the inspection image data 800, 805, and 810. The contour data may include photometric data acquired at one or more locations on the structure (e.g., the cathode). More specifically, the contour data may include grayscale pixel information sampled from the inspection image data along a line typically drawn on the cathode. In this case, the contour consists of grayscale pixel information drawn at locations on said line. For example, Figure 8A The image data 800 associated with the cathode is shown at a first location along the length of the cathode. Figure 8B The image data 805 associated with the cathode is shown at a second location along the length of the cathode. Figure 8C The image data associated with the cathode is shown at a third position along the length of the cathode.

[0076] Figure 8D This is a graph 815 showing the cathode profile data according to the subject matter described herein. The profile data included in graph 815 may correspond to various profiles determined with respect to the inspection image data 800, 805, and 810. Additionally, average profile data of the cathode may also be provided.

[0077] Figures 9A-9C These are images of inspection image data for detecting the anode in a battery, based on the subject described in this article. (Example) Figures 9A-9C As shown, image data 900, 905, and 910 are provided and can correspond to relative to Figure 7The inspection image data 700 is described. The inspection image data 900, 905, and 910 can correspond to photometric data of an anode included in each of the inspection image data 900, 905, and 910. The profile data can include photometric data acquired at one or more locations of a structure (e.g., an anode). More specifically, the profile data can include grayscale pixel information sampled from the inspection image data along a line typically drawn on the anode. In this case, the profile is composed of the grayscale pixel information plotted at the locations on the line. For example, Figure 9A The inspection image data 900 associated with the anode at a first location along the length of the cathode is shown. Figure 9B The inspection image data 905 associated with the anode at a second location along the length of the anode is shown. Figure 9C The inspection image data associated with the anode at a third location along the length of the anode is shown.

[0078] Figure 9D A graph 915 of anode photometric data in accordance with the subject matter described herein. The profile data included in the graph 915 can correspond to various profiles determined with respect to the inspection image data 900, 905, and 910. In addition, average profile data of the anode can also be provided.

[0079] As non-limiting examples, exemplary technical effects of the methods, systems, and computer readable media described herein include determining and generating image filter parameter values for computer vision algorithms to detect and provide indications of structures within objects being inspected. Based on training a prediction model using inspection image data and annotated inspection image data, the image filter parameter values can be generated by the prediction model in a machine learning process. The improved structure detection systems and methods provided herein can reduce inspection time, increase inspection accuracy, and enable detection of structures of new object types with minimal retraining time or manual input.

[0080] Certain example embodiments are described to provide a thorough understanding of the principles of structure, functionality, manufacture, and use of the systems, devices, and methods disclosed herein. One or more examples of these embodiments are illustrated in the accompanying drawings. One skilled in the art will understand that the systems, devices, and methods illustrated and hereinafter described are non-limiting example embodiments, and that the scope of the present invention is only limited by the claims. Features shown or described with respect to one example embodiment can be combined with features of other embodiments. Such modifications and variations are intended to fall within the scope of the present invention. Further, in the disclosure, like-named components of the embodiments generally have similar features, and thus one of ordinary skill in the art will understand that each feature of the one implementation can be, and often is, used in other implementations, and likewise, features of other implementations can be, and often are, used in the one implementation. Therefore, features of one implementation can be incorporated into other implementations without further description.

[0081] The subject matter described herein can be implemented in analog electronic circuits, digital electronic circuits, and / or computer software, firmware, or hardware, including the structural means disclosed in this specification and structural equivalents thereof, or in combinations of them. The subject matter described herein can be implemented as one or more computer program products, such as one or more computer programs tangibly embodied in information carrier (e.g., in a machine -readable storage device), or embodied in propagated signals, for execution by, or to control the operation of, data processing apparatus (e.g., a programmable processor, a computer, or multiple computers). A computer program (also known as a program, software, software application, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file. A program can be stored in a portion of a file that holds other programs or data, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and are interconnected by a communication network.

[0082] The processes and logic flows described in this specification, including the method steps of the subject matter described herein, can be performed by one or more programmable processors executing one or more computer programs to perform functions of the subject matter described herein by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus of the subject matter described herein can be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). The processes and logic flows can also be performed by, and apparatus of the subject matter described herein can be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).

[0083] Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for performing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. Information carriers suitable for embodying computer program instructions and data include all forms of non-volatile memory, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0084] To provide for interaction with a user, the subject matter described herein can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback, and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0085] The technology described herein can be implemented using one or more modules. As used herein, the term “module” refers to computing software, firmware, hardware, and / or various combinations thereof. At a minimum, however, a module is not software, alone, that is implemented in hardware, firmware, or is recorded on a non-transitory processor-readable storage medium (i.e., modules are not software alone that is an instruction to be executed by a processor). Rather, a “module” is a hardware- based component that is self-contained and can be removed and replaced, both individually and also as a group, without loss of functionality. Two different modules can share the same physical hardware. For example, two different modules can use the same processor and network interface. The modules described herein can be combined, integrated, separated, and / or duplicated to support various applications. Also, a function described herein as being performed at a particular module can be performed at one or more other modules and / or by one or more other devices. Furthermore, modules can be implemented across multiple devices and / or other components local or remote to one another. Additionally, the modules can be moved from one device and added to another device, and / or included in both devices.

[0086] The subject matter described herein can be implemented in a computing system that includes a back end component (e.g., a data server), a middleware component (e.g., an application server), or a front end component (e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the subject matter described herein), or any combination of such back end, middleware, and front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (“LAN”) and a wide area network (“WAN”), e.g., the Internet.

[0087] As used herein throughout the description and claims, approximate language can be used to modify any quantitative representation that can vary somewhat but not result in a change in the basic function described. Thus, a value modified by one or more terms such as “about,” “approximately,” and “substantially” should not be limited to the exact value so designated. In at least some instances, the approximate language can correspond to the precision of an instrument used to measure the value. Ranges can be combined and / or interchanged, unless context or language indicates otherwise, this range is recognized and includes all sub-ranges contained therein.

[0088] Other features and advantages of the present application will be apparent from the foregoing other embodiments, and specific examples thereof, and therefore, it is the object of the claims to cover all such modifications and equivalents. All publications and references cited herein are expressly incorporated herein by reference for the purpose of describing and disclosing compositions and methodologies which are considered to be part of the state of the art.

Claims

1. A method for detecting structures, the method comprising: receiving, by a data processor, first image data acquired during an inspection of an object, the first image data characterizing a region of interest of the object, the region of interest containing one or more structures of the object; determining, using the data processor and a computer vision algorithm, at least one structure of the one or more structures within the region of interest, the computer vision algorithm comprising a plurality of image filters configured to filter the first image data with respect to luminosity characteristics of a plurality of pixels of the first image data, each image filter of the plurality of image filters comprising one or more image filter parameters, wherein an image filter parameter value of each image filter parameter of the plurality of image filters of the computer vision algorithm is determined using a prediction model, the prediction model trained to determine the image filter parameter value based on training image data; and providing an indication of the at least one structure in second image data, wherein determining the at least one structure within the region of interest further comprises: receiving, using the data processor, training image data of the object, the training image data comprising at least one first annotation identifying at least one structure of an object present in at least one region of interest in the training image data; determining, using the data processor and a prediction model, at least one set of image filter parameter values associated with at least one image filter parameter of the plurality of image filters, the prediction model trained to receive the training image data comprising at least one first annotation and to determine the at least one set of image filter parameter values for each image filter parameter of the plurality of image filters based on applying at least one optimization technique using the training image data and the at least one first annotation; and updating at least one image filter of the computer vision algorithm based on the at least one set of image filter parameter values, and wherein the prediction model is trained to determine the at least one set of image filter parameter values for at least one of an average contour image filter parameter, a line tracking image filter parameter, or a stopping criteria image filter parameter.

2. The method of claim 1, wherein the at least one optimization technique comprises a grid search technique, a random search technique, a Levenberg-Marquardt technique, a gradient search technique, a Bayesian optimization technique, or a combination of two or more optimization techniques.

3. The method of claim 1, wherein in response to providing the indication of the at least one structure in the second image data, the method further comprises: modifying the indication in the second image data with at least one annotation, and including the second image data in the training image data, the second image data including the annotation of the at least one structure; characterizing the region of interest, wherein the characterizing comprises characterizing a defect in the region of interest or characterizing the region of interest as having a defect; and training the predictive model based on using the training image data including the second image data and the annotation of the at least one structure, determining the at least one set of image filter parameter values.

4. The method of claim 1, wherein the computer vision algorithm is configured to generate contour data associated with a portion of the plurality of pixels, the portion corresponding to at least one structure present within the region of interest, and the contour data comprises at least one of photometric data, tip detection data, particle detection data, or void detection data.

5. The method of claim 1, wherein the first image data comprises two-dimensional computed tomography image data, three-dimensional computed tomography image data, or x- radiographic data.

6. The method of claim 1, wherein the object is a lithium-ion battery, a nickel-metal hydride battery, or a nickel-cadmium battery.

7. The method of claim 1, wherein the region of interest comprises at least one of a contour of a cathode of a battery, an anode of a battery, a casing of a battery, a volume between a cathode of a battery and an anode of a battery, a centerline of a cathode of a battery, a centerline of an anode of a battery, and a shape of a battery.

8. The method of claim 1, further comprising: determining, using the data processor, at least one of a position of the object within the region of interest or an alignment of one or more portions of the first image data, the one or more portions comprising characterizing the region of interest of the object; and and providing an indication of the position of the object within the region of interest or the alignment of one or more portions of the first image data.

9. A system for detecting structures, the system comprising: a memory storing computer-readable instructions, at least one predictive model, and at least one computer vision algorithm; and a processor configured to execute the computer-readable instructions, which when executed, cause the processor to perform operations comprising: receiving first image data acquired during an inspection of an object, the first image data characterizing a region of interest of the object, the region of interest containing one or more structures of the object; determining at least one structure of the one or more structures within the region of interest using a computer vision algorithm, the computer vision algorithm comprising a plurality of image filters configured to filter the first image data with respect to luminosity characteristics of a plurality of pixels of the first image data, each image filter of the plurality of image filters comprising one or more image filter parameters, wherein an image filter parameter value of each image filter parameter of the plurality of image filters of the computer vision algorithm is determined using the at least one predictive model, the predictive model trained to determine the image filter parameter value based on training image data; and providing an indication of the at least one structure in second image data, wherein the instructions cause the processor to determine the at least one structure by performing operations further comprising: receiving training image data of the object, the training image data comprising at least one first annotation, the at least one first annotation identifying at least one structure of an object present in at least one region of interest in the training image data; determining at least one set of image filter parameter values associated with at least one image filter parameter of the plurality of image filters using a predictive model stored in the memory, the predictive model trained to receive the training image data comprising at least one first annotation and determine the at least one set of image filter parameter values for each image filter parameter of the plurality of image filters based on applying at least one optimization technique using the training image data and the at least one first annotation; and updating at least one image filter of the computer vision algorithm based on the at least one set of image filter parameter values, and wherein the predictive model is trained to determine the at least one set of image filter parameter values for at least one of an average profile image filter parameter, a line tracking image filter parameter, or a stopping criteria image filter parameter.

10. The system of claim 9, wherein the at least one optimization technique comprises a grid search technique, a random search technique, a Levenberg-Marquardt technique, a gradient search technique, a Bayesian optimization technique, or a combination of two or more optimization techniques.

11. The system of claim 9, wherein in response to providing the indication of the at least one structure of the one or more structures in the second image data, the instructions cause the processor to perform further operations comprising: modifying the indication in the second image data with at least one annotation and including the second image data in the training image data, the second image data comprising the annotation of the at least one structure and characterizing the region of interest of the object; characterizing the region of interest of the object, wherein the characterizing comprises characterizing a defect in the region of interest or characterizing the region of interest as having a defect; and and training the prediction model based on using the training image data including the second image data and the annotation of the at least one structure, determining the at least one set of image filter parameter values.

12. The system of claim 9, wherein the computer vision algorithm is configured to generate contour data associated with a portion of the plurality of pixels, the portion corresponding to at least one structure present within the region of interest, and the contour data including at least one of photometric data, tip detection data, particle detection data, or void detection data.

13. The system of claim 9, wherein the first image data includes two-dimensional computed tomography image data, three-dimensional computed tomography image data, or radiographic data.

14. The system of claim 9, wherein the object is a lithium-ion battery, a nickel-metal hydride battery, or a nickel-cadmium battery.

15. The system of claim 9, wherein the region of interest includes at least one of a contour of a cathode of the battery, an anode of the battery, a casing of the battery, a volume between the cathode of the battery and the anode of the battery, a centerline of the cathode of the battery, a centerline of the anode of the battery, and a shape of the battery.

16. The system of claim 9, wherein the instructions cause the processor to perform further operations comprising: determining at least one of a position of the object within the region of interest or an alignment of one or more portions of the first image data, the one or more portions including the region of interest characterizing the object; and providing an indication of the position of the object within the region of interest or the alignment of one or more portions of the first image data. ​

Images