Machine vision change detection for process monitoring

Abstract

The invention provides machine vision change detection for process monitoring. A method for metric determination includes extracting one or more features from a plurality of images; and classifying the plurality of images based on the extracted one or more features to form one or more image clusters, wherein the classification is performed based at least in part on machine learning. A hypothesis test is performed on the one or more image clusters based on one or more production elements, and a change in the one or more features resulting from the hypothesis test is identified. One or more production metrics are determined based on the identified changes.

Description

Technical Field

[0001] The present disclosure relates to monitoring of production processes. More particularly, the present disclosure relates to systems and methods for monitoring a production process to detect changes in the production process based on acquired images of the process. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and may not constitute prior art.

[0003] Manufacturing processes can be supervised using vision-based systems to monitor the process. Interpretation of acquired images can be challenging. Some monitoring systems use artificial intelligence and machine learning (AI / ML) techniques along with image datasets to facilitate identification of relevant features in the monitored process. However, generating sufficient data with the desired properties for algorithm training can be difficult. Introducing anomalies into the training process is also typically time-consuming, and it is difficult to know how anomalies will appear before actual production problems occur.

[0004] The present disclosure addresses these and other problems associated with monitoring a production process to detect changes in the production process based on acquired process images. Summary of the Invention

[0005] This section provides a general summary of the disclosure and is not a comprehensive disclosure of its full scope or all of its features.

[0006] The present disclosure provides a computerized method comprising: extracting one or more features from a plurality of images; classifying the plurality of images based on the extracted one or more features to form one or more image clusters, the classification being performed at least in part based on machine learning; performing hypothesis testing on the one or more image clusters based on one or more production factors; identifying changes in the one or more features resulting from the hypothesis testing; determining one or more indicators (e.g., production indicators) based on the identified changes; wherein the plurality of images comprises current images of products along a production line and past images of similar products along the production line; wherein each of the plurality of images is classified into one of two clusters, and the hypothesis testing identifies changes in the images of the two clusters using a null hypothesis; and further comprising: the probability of rejection is greater than a threshold value defined at a significance level; wherein the plurality of images comprise images of a product along an assembly line, and further comprising using the one or more indicators to monitor one or more production processes of the product; wherein the one or more production factors comprise one or more of inputs, noise, process variations, or a combination thereof; wherein the process variations comprise changes to at least one of production date and time, operator, tool, production, or a combination thereof; wherein the product comprises a vehicle high-voltage battery pack, and the one or more features relate to a cell tab weld of a battery cell within the vehicle high-voltage battery pack; and further comprising pre-processing the plurality of images, wherein the pre-processing comprises centering and cropping each of the plurality of images on a relevant area, and masking out other areas of each of the images corresponding to noise.

[0007] The present disclosure provides a system comprising: a plurality of cameras configured to acquire a plurality of images of products along a production line; and a monitoring system receiving the plurality of images and configured to: extract one or more features from the plurality of images; classify the plurality of images based on the extracted one or more features to form one or more image clusters, the classification being performed at least in part based on machine learning; perform hypothesis testing on the one or more image clusters based on one or more production factors; identify changes in the one or more features resulting from the hypothesis testing; and determine one or more indicators based on the identified changes to thereby monitor the products; wherein the plurality of images include current images of products along the production line and past images of similar products along the production line; wherein each of the plurality of images is classified into two one of the clusters, and the hypothesis testing uses a null hypothesis to identify changes in the images of the two clusters; wherein the monitoring system is further configured to reject the null hypothesis in response to an observed probability being greater than a threshold of a defined significance level; wherein the one or more production factors include one or more of inputs, noise, process variations, or a combination thereof; wherein the process variations include changes to at least one of production date and time, operator, tool, production, or a combination thereof; wherein the product includes a vehicle high-voltage battery pack, and the one or more features relate to a cell tab weld of a battery cell within the vehicle high-voltage battery pack; wherein the monitoring system is further configured to pre-process the multiple images, wherein the pre-processing includes centering and cropping each of the multiple images on a relevant area, and masking out other areas of each of the images corresponding to noise.

[0008] The present disclosure provides one or more non-transitory computer-readable media storing processor-executable instructions that, when executed by at least one processor, cause the at least one processor to: extract one or more features from a plurality of images; classify the plurality of images based on the extracted one or more features to form one or more image clusters, the classification being performed at least in part based on machine learning; perform hypothesis testing on the one or more image clusters based on one or more production factors; identify changes in the one or more features resulting from the hypothesis testing; and determine one or more indicators based on the identified changes; wherein Each of the plurality of images is classified into one of two clusters, and the hypothesis testing uses a null hypothesis to identify changes in the images of the two clusters, and wherein the at least one processor is also caused to: reject the null hypothesis in response to the observed probability being greater than a threshold of a defined significance level; wherein the product includes a vehicle high-voltage battery pack, and the one or more features relate to a cell tab weld of a battery cell within the vehicle high-voltage battery pack, and wherein the at least one processor is also caused to: pre-process the plurality of images, wherein the pre-processing includes centering and cropping each of the plurality of images on a relevant area, and masking out other areas of each of the images corresponding to noise.

[0009] Further areas of applicability will become apparent from the description provided herein.It should be understood that the description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order that the present disclosure may be better understood, various forms of the present disclosure will now be described by way of example with reference to the accompanying drawings, in which:

[0011] Figure 1 An overall system for monitoring a production process according to various embodiments is shown;

[0012] Figure 2 It can be done through Figure 1 A perspective view of an example of a battery cell produced by a production process;

[0013] Figure 3 yes Figure 2 a front cross-sectional view of a battery cell;

[0014] Figure 4 yes Figure 1 Block diagram of the checkpoint in ;

[0015] Figure 5 is a flowchart illustrating a machine learning pipeline according to an embodiment; and

[0016] Figure 6 is a flow chart illustrating an example method for metric determination according to various implementations.

[0017] The drawings described herein are for illustration purposes only and are not intended to limit the scope of the present disclosure in any way. DETAILED DESCRIPTION

[0018] The following description is merely exemplary in nature and is not intended to limit the present disclosure, application, or uses. It should be understood that throughout the drawings, corresponding reference numerals indicate like or corresponding parts and features.

[0019] The present disclosure provides a means for monitoring changes or alterations in a production process based on images and known production factors without using any data with known labels for the images. For example, various embodiments provide monitoring of a production process where the images provide useful information about the variability of the process. In some embodiments, machine vision variation (e.g., change) detection is performed on one or more production or manufacturing processes, such as using machine vision classification of images of cell tab welds of a battery pack to analyze the variability of high voltage battery pack production. It should be understood that the monitoring and visual classification of the various embodiments are not limited to applications with battery pack production, but can be used for any type of production or manufacturing process.

[0020] Machine vision classification methods can be broadly categorized as supervised or unsupervised, with the former using information about the production status of the image (labels). One or more methods described herein perform machine vision classification without labels (e.g., without using image labels), thereby eliminating the challenging and complex process of obtaining accurate labels.

[0021] Relative to existing unsupervised methods, various embodiments of the detection methods disclosed herein are unsupervised and have significant characteristics. That is, unsupervised methods detect changes or alterations in the process. Although existing unsupervised methods can cluster data, there is no explanation for the meaning of the clusters (e.g., labels). One or more examples use explanations with the help of hypothesis testing to evaluate whether there are changes (e.g., significant changes) in image features based on one or more known production factors (e.g., production date and time, operator, tool, production line, etc.). In addition, the nature of image capture and the reduced computational requirements of one or more algorithms described herein allow for more efficient monitoring of large production volumes, and therefore can also supplement the slower process of taking individual samples from the production line for inspection and testing. Therefore, various examples combine null hypotheses from probability theory with image clustering from machine learning to determine whether any changes in the manufacturing process lead to changes in the product being manufactured.

[0022] Figure 1 A schematic block diagram illustration of a process monitoring system 100, such as for inspection of a production (e.g., manufacturing or assembly) line 102, is shown. In one example, the process monitoring system 100 is configured to identify changes in one or more characteristics of a product 104 produced (e.g., manufactured or assembled) along the production line 102. As described in more detail herein, the changes are identified as a result of hypothesis testing, and one or more indicators are determined based on the identified changes. In one example, the product 104 is a vehicle high voltage battery pack, and the one or more characteristics relate to cell tab welds of battery cells within the vehicle high voltage battery pack. That is, the production line 102 is a battery manufacturing production line that includes at least welding of cell tabs for vehicle high voltage battery packs. However, the embodiments described herein may be used for monitoring to detect changes in other types of products and / or based on other types of monitoring applications.

[0023] In the example shown, the process monitoring system 100 includes a feature inspection station 106 that identifies changes in one or more features, such as weld features of a vehicle high voltage battery pack. To this end, in some examples, the feature inspection station 106 is a weld inspection station.

[0024] Continuing with the battery pack production example, the production line 102 includes a conveyor 'C' configured to transport products 104 (in this case, batteries) to and from an operating station, which is shown as a welding station 108, although batteries may be transported to and from the welding station 108 using other types of transport mechanisms or devices, including manual and / or robotic transport of the products 104. The welding station 108 includes a welder 110 (e.g., a laser welder) configured to perform cell tab welding on the battery pack. For example, cell tab welding may be performed on the battery cell assembly 200, as shown. Figure 2 and Figure 3 As shown in FIG. The battery cell assembly 200 can be configured as a pouch battery cell assembly or other battery cell assembly. More than one battery cell assembly 200 can be arranged in an array, with bus bars interconnecting the battery cell assemblies 200 in parallel or series to provide power to vehicle components. For example, multiple battery cell assemblies 200 can be electrically connected to each other within a traction battery.

[0025] The battery cell assembly 200 in the illustrated example includes a housing 204, a first intermediate terminal member 208, and a second intermediate terminal member 210. The first intermediate terminal member 208 and the second intermediate terminal member 210 can have many suitable shapes extending from opposite ends. Seals 212 can be located between the housing 204 and the first intermediate terminal member 208, and between the housing 204 and the second intermediate terminal member 210. In addition, the upper portion of the housing 204 and the lower portion of the housing 204 can be sealed to each other. The housing 204 can define a cavity 214 sized to hold the battery cell active materials, such as the anode, cathode, separator, and electrolyte. The housing 204 of the pouch battery cell can be made of a conformal polymer / aluminum laminate that is vacuum-sealed around the battery cell active materials. For example, two separate stacked laminates can be sealed on all four edges, or a single laminate folded in half can be sealed along three or all four edges.

[0026] The first terminal tab 216 can be embedded in the first intermediate terminal member 208, and the second terminal tab 220 can be embedded in the second intermediate terminal member 210. Each of the terminal tabs can be embedded with the corresponding intermediate terminal member through, for example, injection molding, 3D printing, or casting. In some examples, the welder 110 forms the weld tabs and / or electrically connects the first terminal tab 216 and the second terminal tab 220 to the control board or other components of the battery cell assembly 200 through a welding operation.

[0027] In some examples, after performing the welding operation (e.g., forming the cell tab welds), the conveyor C transports each of the batteries to an inspection station 106 (in this example, a weld inspection station), where the plurality of welds are automatically inspected using (i) an imaging device (such as a camera 112) and (ii) a control system 114, as described in more detail herein. Non-limiting examples of cameras 112 include two-dimensional (2D) cameras, such as 2D area cameras, 2D line scan cameras, and the like, and three-dimensional (3D) cameras, such as 3D laser scanners, 3D area cameras, and the like. In some variations, the results of the automated inspection are displayed on a display screen 116. And after leaving the inspection station 106, the inspected batteries are removed from the inspection station 106, which can be further processed.

[0028] refer to Figure 4, shows a camera 112, a control system 114, and a display screen 116. The control system 114 includes a camera control module 300 configured to command the camera 112 to capture one or more characteristic images (e.g., a plurality of welds) of each of the products 104 (in this example, welded batteries) entering the inspection station 106. In some embodiments, the camera control module 300 is configured to command or set the camera angle, focus, and zoom of the camera 112. In at least one example, the camera control module 300 is configured to command the camera 112 to capture a series of images of the welds as each battery moves past within the field of view (e.g., scanning area) of the camera 112, such that as the battery moves relative to the camera 112, the camera 112 scans and captures images of the plurality of welds. In examples where the camera 112 is a 2D line scan camera or a 3D laser scanner, the image compilation module 302 compiles the series of images into one or more compiled images that are analyzed by the change detection module 304.

[0029] The change detection module 304 is configured to detect or identify changes or variations in one or more features of the product 104 using hypothesis testing as described in more detail herein. The detected changes or variations allow one or more indicators of the production line 102 to be determined. In one example, a detected change associated with a cell tab weld of a battery cell within a battery pack allows the characteristics of the battery pack's seal to be determined (e.g., ensuring that the battery's electrolyte pack is sealed, such as ensuring that the soft pack of the battery cell assembly 200 is completely sealed). In some examples, the change detection module 304 allows for the detection of anomalies in a weld (e.g., a cell tab weld of a battery pack) that may be caused by a change in the production process, a change in the welding nozzle, etc. Thus, the change detection module 304 allows for the identification of characteristics (e.g., visual characteristics) and any changes in one or more features (e.g., a cell tab weld) to determine one or more indicators. It should be understood that the one or more features and the one or more indicators can be associated with any production process and are not limited to the production of battery packs.

[0030] In some embodiments, the change detection module 304 identifies changes and then allows for a score (e.g., a number from 1 to 10, representing an anomaly score) to be provided as a metric for identified production issues (e.g., anomalies on the surface of a weld) based on the severity or amount of each anomaly on the surface. Some examples of severity or amount of anomalies include the percentage of surface area with abnormal discoloration (e.g., determined via pixel color comparison), the size of the weld relative to a predefined size or predefined size range, the number of pits, the size of the pits, the number of cracks, the size of the cracks, the number of veins or valleys, and the size of the veins or valleys, etc. In some examples, the change detection module 304 identifies changes in one or more features (e.g., cell tab welds), which allows for an overall score for each feature (e.g., weld) to be determined or tabulated based on the score of each identified anomaly. In one or more examples, the determined metric (e.g., score) is transmitted to the notification module 306. In some embodiments, the notification module 306 provides a visual, audible, or tactile notification based on the identified changes.

[0031] In some embodiments, the change detection module 304 is trained (or uses a trained algorithm) to identify changes (e.g., anomalies). And in at least one embodiment, the change detection module 304 is continuously trained to identify the changes. In one or more embodiments, the change detection module 304 includes or receives data trained by a neural network having multiple input units, hidden units, and output units. And in at least one embodiment, the neural network is a feedforward network trained via backpropagation. In some embodiments, as described in more detail herein, the training of the algorithm allows feature extraction without image labels, which is then used for hypothesis testing to assess whether there are significant changes in image features based on known production factors. Thus, in some examples, the change detection module 304 is trained to distinguish between changes that affect the production process and changes that do not affect the production process (e.g., discoloration present on the surface of a "good" weld from contamination discoloration present on the surface of a "bad" weld). As described in more detail herein, one or more examples thus support inferring image changes rather than weld strength.

[0032] In some examples, the notification module 306 is configured to generate a report, such as based on the total score determined by the change detection module 304. In some embodiments, the report and / or a version of the report is transmitted and displayed on the display screen 116 so that the operator can review it. Non-limiting examples of information included in the report and / or a version of the report about the welding example displayed on the display screen 116 include the number of "good" welds, the number of "failed" welds, the number of "needs further inspection" welds, the types of anomalies detected or identified, the locations of the anomalies, images of the identified anomalies, partial images of the welds, images of all welds, and images of the welding cell showing one or more locations where one or more anomalies have been identified, etc.

[0033] In some variations, notification module 306 provides a list of identified changes (such as identified anomalies) to change database 308, causing change database 308 to be updated. Additionally, in at least one example, updated change database 308 is used to further train change monitoring module 304.

[0034] Figure 5 is a flow diagram illustrating a machine learning pipeline 400 that can be used to detect changes, such as utilizing the change detection module 304 in various examples. As described herein, in some examples, battery cell tab weld analysis is performed using a machine learning pipeline 400 that is configured as an AI / ML machine vision pipeline that allows for weld image analysis. The machine learning pipeline 400 in the illustrated example includes preprocessing at 402, feature extraction at 404, and classification / anomaly detection at 406. It should be understood that the machine learning pipeline 400 can be performed in multiple steps, each of which can be performed one or more times. The machine learning pipeline 400 identifies changes in an image feature (X), where in some examples, X=G(I,Z) and the weld strength (Y) is defined as Y=F(X), as described in more detail below.

[0035] In one embodiment, the machine learning pipeline 400 is configured to receive as input a production image 408 (e.g., a raw production image or an SDI image) and a production factor pair, as described in more detail herein. The machine learning pipeline 400 is configured to pre-process the input and generate one or more outputs, which can be used to generate a notification (such as using the notification module 306) and / or display a result (e.g., a change detected or not detected) on the display 116. In one example, the input data is SDI data, which includes an image (I) and a production factor pair, wherein the image is a pixel matrix and the production factor pair includes a production factor Z (e.g., date, label, line side, etc.) and an SDI label (i.e., I, Z→(Y<threshold)), which defines two populations as described in more detail herein.

[0036] Regarding the pre-processing at 402, one or more different types of image pre-processing are performed, such as centering and cropping the original image (e.g., centering and cropping the image on a relevant portion of the image (e.g., a weld)) and masking out noise (e.g., masking out any other irrelevant portions (noise) in the image). It should be understood that any type of image pre-processing technique can be used, such as to reduce the signal-to-noise ratio. At 404, one or more processed images 410 are output for feature extraction.

[0037] The machine learning pipeline 400 then performs feature extraction (G) at 404. For example, image features can be extracted by applying processing operations that have been trained for other tasks (e.g., neural network processing). In one example, a classifier and corresponding neural network feature extraction architecture are trained on a large database of publicly available natural images and labels. In one embodiment, after training, only the feature extraction portion is retained and used to extract features from the image to feed into the next clustering step. It should be understood that any type of neural network can be used to perform the feature extraction at 404. In addition, different processing can be performed, such as using brightness histograms or other data.

[0038] Classification / anomaly detection is then performed on the extracted features at 406, which in some examples includes hypothesis testing, as described in more detail herein. For example, the key output variables of the process are represented by vector Y, and the key input variables are represented by vector X. In various examples, the assumption of vision-based monitoring is that X is a function of the image (e.g., preprocessed image 410), represented by I (pixel matrix). In addition, classification and anomaly detection are performed so that the output is consistent across production factors represented by vector Z. The factors in some examples are derived from the date and time of production, operator, tool, production line, etc. In one embodiment, the model of the process is defined as: Y=F(X)=F(G(I,Z)), where the process is configured to infer G:I,Z→X without information about the output Y. Specifically, the process operates to infer whether there are significant differences between two image populations with different factors Z, and thereby infer (e.g., output) whether the process is stable across these image populations.

[0039] In one example, P1 and P2 represent two image populations, each with n images, where at least one element in the associated vectors Z1 and Z2 differs. The null hypothesis is defined as follows: H0: There is no difference between the two image populations P1 and P2. Next, clustering is applied to two feature sets {X = G(i, Z), i∈P}, k = 1, 2, and it is determined whether the populations can be separated with an accuracy significantly different from random chance.

[0040] The clustering operation is configured to group images represented by image features (e.g., pre-processed images 410) into two clusters. In one example, this is performed by selecting a similarity metric between images and an algorithm for finding cluster assignments so that the similarity between images in each cluster is less than the similarity between images in the other cluster. Any suitable technique may be used, such as using Euclidean distance with K-means and Gaussian mixture models with expectation maximization. However, other suitable techniques may be used.

[0041] The clustering operation assigns each image to one of two clusters, which can be represented by two sets, denoted by C1 and C2, containing images assigned to the respective sets. Next, the number of correct assignments, denoted by m, is calculated, given that each image set is completely isolated in either set, as follows: m = max{|P∩C| + |P∩C|, |P∩C| + |P∩C|}, where |P∩C| is the number of images in population P in cluster C1, k = 1, 2, l = 1, 2. The maximum operation is used in various examples because the number of clusters from the clustering algorithm is arbitrary. That is, by definition, n ≤ m ≤ 2n.

[0042] Under the null hypothesis, there is no difference between the two image populations, and the number of correct assignments, m, is random. Specifically, m follows a binomial distribution, and the random variable M∈Bin(2n,p) has probability p=0.5 (the ratio of correct answers). Therefore, the null hypothesis is restated as: H0:p=0.5. In various examples, if the observed probability (accuracy) m / 2n is greater than a threshold of significance level α (the populations are different), the null hypothesis is rejected. On the other hand, if the observed probability is not greater than the threshold, the null hypothesis is not rejected (no difference between the populations can be established). For example, for 2n=200 images, and significance level α=0.001 gives Reject H0, otherwise do not reject H0.

[0043] The classification and anomaly detection at 406 (which in some examples as described herein includes clustering) generate one or more outputs. For example, the classification at 406 generates a label probability 412 (e.g., the probability of the correct label based on the classification) as an output, the anomaly detection generates an anomaly score 414 as an output, and the clustering generates a cluster number 416 as an output. It should be noted that different processes can be used to perform classification, anomaly detection, and clustering, and the embodiments described herein are provided as examples.

[0044] Thus, having developed feature extraction, unsupervised machine learning as described herein is utilized to identify one or more anomalies and analyze image changes. It should be appreciated that in some examples, SDI labels with supervised methods are used to validate feature extraction.

[0045] In operation, when implementing one or more of the processes described herein (e.g., machine learning pipeline 400), one or more examples can be applied to monitoring a production process, such as using monitoring system 100 to detect changes in selected population pairs based on production factors (e.g., input, noise, and process). Returning to the example application for cell tab welding of high-voltage battery packs, one embodiment will now be described.

[0046] In the case of cell tab welding, in one example, the relevant factors are the production cycle and two production line related factors (one factor is level 1 or 2, and the other factor is level A or B). Then, the monitoring process according to one or more examples is configured as follows:

[0047] 1. Cycle-to-cycle variations for a given production line can be detected by selecting the following four population pairs: Population 1 from the current cycle and each of the four unique combinations of two production line elements. Population 2 from the previous cycle has the same combination of production line elements as Population 1.

[0048] 2. Line-to-line variations for the current cycle can be detected by selecting the following four pairs: Population 1 from the current cycle and each of the four unique combinations of two line elements. Population 2 from the current cycle where one of the line elements changed.

[0049] Process monitoring can be extended to include additional elements. In this example, it should be noted that for line-to-line variation, there are a total of Pairs, which include pairs where two production line elements are different within the same cycle. In general, combinations of such elements may be included, but it should be noted that the combinations are not relevant in this application.

[0050] Next, the population sample size n is selected to be the minimum value that constrains the available samples in the pair. Next, the process randomly samples n samples from the larger population and repeats the process multiple times (e.g., depending on the population size difference). Finally, the hypothesis test is evaluated by evaluating the statistical information of the observed accuracy m / 2n relative to a threshold (calculated according to Bin(2n, 0.5)) based on the selected significance level. Using this embodiment, process changes can thus be identified.

[0051] Figure 6 is a flow chart illustrating an example method 500 for determining a metric, such as a production metric related to a cell tab weld of a battery cell within a battery pack. At operation 502, one or more features are extracted from a plurality of images. As described in greater detail herein, in some examples, features are extracted from a pre-processed image (e.g., an original image centered and cropped over a region of interest and masked out from other regions) and are associated with a welding operation. In some examples, the images are current images of a product (e.g., a battery pack) along a production line and past images of similar products along the production line.

[0052] At operation 504, the images are classified based on the extracted features to form one or more image clusters. In some examples, the classification is performed at least in part based on machine learning, as described herein. Hypothesis testing is also performed on the one or more image clusters based on one or more production factors, such as input, noise, and process variation (such as at least one of production date and time, operator, tool, production, or a combination thereof). For example, the images are classified into one of two clusters, and the hypothesis testing uses the null hypothesis to identify variations in the images of the two clusters. In one example, the null hypothesis is rejected in response to the observed probability being greater than a threshold value of a significance level defined as described herein. The null hypothesis is accepted in response to the observed probability being less than a threshold value of a significance level defined as described herein.

[0053] At operation 506, a change in one or more features is identified. For example, a change resulting from a hypothesis test is identified. In one example, where the image includes an image of a product along an assembly line, one or more production processes of the product are monitored at 510 to determine production indicators. That is, in response to determining at 508 that a change (e.g., a weld anomaly) is detected in one or more of the production processes, one or more production indicators are determined based on the identified change at 510, and a corresponding alert may be generated as described in more detail herein. In response to no change being detected, method 800 stops at 512.

[0054] Thus, one or more embodiments provide for monitoring changes in a production process based on images and known production factors without using any data with known labels associated with the characteristics represented by the images.

[0055] Unless otherwise expressly indicated herein, all numerical values indicating mechanical / thermal properties, composition percentages, dimensions and / or tolerances or other characteristics when describing the scope of the present disclosure should be understood as modified by the word "about" or "approximately." Such modification is desirable for various reasons, including: industrial practice; material, manufacturing and assembly tolerances; and testing capabilities.

[0056] As used herein, the phrase at least one of A, B, and C should be construed to mean a logical (A or B or C), using a non-exclusive logical "or", and should not be construed to mean "at least one of A, at least one of B, and at least one of C."

[0057] In this application, the terms "controller" and / or "module" may refer to, be part of, or include: an application-specific integrated circuit (ASIC); digital, analog, or mixed analog / digital discrete circuits; digital, analog, or mixed analog / digital integrated circuits; combinatorial logic circuits; field-programmable gate arrays (FPGAs); processor circuits (shared, dedicated, or grouped) that execute code; memory circuits (shared, dedicated, or grouped) that store code executed by the processor circuits; other suitable hardware components that provide the described functionality (e.g., an operational amplifier circuit integrator as part of a heat flux data module); or a combination of some or all of the above, such as in a system on a chip.

[0058] The term memory is a subset of the term computer-readable medium. The term computer-readable medium, as used herein, does not encompass transient electrical or electromagnetic signals propagated through a medium (such as on a carrier wave); thus, the term computer-readable medium may be considered to be both tangible and non-transitory. Non-limiting examples of non-transitory tangible computer-readable media are non-volatile memory circuits (such as flash memory circuits, erasable programmable read-only memory circuits, or mask read-only circuits), volatile memory circuits (such as static random access memory circuits or dynamic random access memory circuits), magnetic storage media (such as analog or digital magnetic tape or hard drives), and optical storage media (such as CDs, DVDs, or Blu-ray discs).

[0059] The apparatus and methods described in this application may be implemented partially or completely by a special-purpose computer created by configuring a general-purpose computer to perform one or more specific functions embodied in a computer program. The functional blocks, flow chart components, and other elements described above serve as software specifications that can be translated into a computer program through routine work by a technician or programmer.

[0060] The description of the present disclosure is merely exemplary in nature and, thus, variations that do not depart from the essence of the disclosure are intended to be within the scope of the present disclosure. Such variations are not to be regarded as a departure from the spirit and scope of the present disclosure.

[0061] One or more non-transitory computer-readable media storing processor-executable instructions that, when executed by at least one processor, cause the at least one processor to: extract one or more features from a plurality of images; classify the plurality of images based on the extracted one or more features to form one or more clusters of images, the classification being performed at least in part based on machine learning; perform hypothesis testing on the one or more clusters of images based on one or more production factors; identify changes in the one or more features resulting from the hypothesis testing; and determine one or more production indicators based on the identified changes.

[0062] According to an embodiment, each of the plurality of images is classified into one of two clusters, and the hypothesis testing uses a null hypothesis to identify changes in the images in the two clusters, and wherein the at least one processor is further caused to reject the null hypothesis in response to an observed probability being greater than a threshold value of a defined significance level.

[0063] According to an embodiment, the product includes a vehicle high-voltage battery pack, and the one or more features relate to a cell tab weld of a battery cell within the vehicle high-voltage battery pack, and wherein the at least one processor is also caused to: pre-process the multiple images, wherein the pre-processing includes centering and cropping each of the multiple images on a relevant area, and masking out other areas of each of the images corresponding to noise.

Claims

1. A computerized method comprising: extracting one or more features from the plurality of images; classifying the plurality of images based on the extracted one or more features to form one or more image clusters, the classifying being performed at least in part based on machine learning; performing hypothesis testing on the one or more image clusters based on one or more production factors; identifying changes in the one or more characteristics resulting from the hypothesis testing; as well as One or more production indicators are determined based on the identified changes.

2. The computerized method of claim 1 wherein the plurality of images includes current images of products along a production line and past images of similar products along the production line.

3. The computerized method of claim 1, wherein each image in the plurality of images is classified into one of two clusters, and the hypothesis testing uses a null hypothesis to identify changes in the images of the two clusters.

4. The computerized method of claim 3, further comprising rejecting the null hypothesis in response to the observed probability being greater than a threshold value defined for a significance level.

5. The computerized method of claim 1, wherein the plurality of images include images of a product along an assembly line, and further comprising using the one or more production indicators to monitor one or more production processes of the product.

6. The computerized method of claim 5, wherein the one or more production factors include one or more of input, noise, process variation, or a combination thereof.

7. The computerized method of claim 6, wherein the process changes include changes to at least one of production date and time, operator, tool, production, or a combination thereof.

8. The computerized method of claim 5, wherein the product comprises a vehicle high voltage battery pack, and the one or more features relate to cell tab welds of battery cells within the vehicle high voltage battery pack.

9. The computerized method of claim 1 , further comprising pre-processing the plurality of images, wherein the pre-processing comprises centering and cropping each of the plurality of images on a relevant region and masking out other regions of each of the images corresponding to noise.

10. A system comprising: a plurality of cameras configured to capture a plurality of images of products along a production line; as well as a monitoring system that receives the plurality of images and is configured to: extracting one or more features from the plurality of images; classifying the plurality of images based on the extracted one or more features to form one or more image clusters, the classifying being performed at least in part based on machine learning; performing hypothesis testing on the one or more image clusters based on one or more production factors; identifying changes in the one or more characteristics resulting from the hypothesis testing; as well as One or more production indicators are determined based on the identified changes to thereby monitor production of the product.

11. The system of claim 10, wherein the plurality of images includes current images of the product along the production line and past images of similar products along the production line.

12. The system of claim 10 , wherein each of the plurality of images is classified into one of two clusters, and the hypothesis testing uses a null hypothesis to identify changes in the images in the two clusters, and wherein the monitoring system is further configured to reject the null hypothesis in response to an observed probability being greater than a threshold value defined at a significance level.

13. The system of claim 10, wherein the one or more production factors include one or more of inputs, noise, process variations, or a combination thereof, and wherein the process variations include changes to at least one of production date and time, operators, tools, production, or a combination thereof.

14. The system of claim 10, wherein the product comprises a vehicle high voltage battery pack, and the one or more features relate to cell tab welds of battery cells within the vehicle high voltage battery pack.

15. The system of claim 10, wherein the monitoring system is further configured to pre-process the plurality of images, wherein the pre-processing comprises centering and cropping each of the plurality of images on a relevant region and masking out other regions of each of the images corresponding to noise.

Images