System and method for detecting localized cracking using digital image correlation

Abstract

Optical inspection systems, methods, and media are configured to perform and / or implement receiving a time series of 3D point cloud images of a target object from an imaging system, wherein the 3D point cloud images include first directional position data, second directional position data, and change in first directional velocity data; applying a self-supervised machine learning model to the time series of 3D point cloud images, wherein the trained machine learning model has been trained on a plurality of training point cloud images to which simulated cracks have been added; and identifying a mechanical failure in the target object based on an output of the machine learning model.

Description

SYSTEM AND METHOD FOR DETECTING LOCALIZED CRACKING USINGDIGITAL IMAGE CORRELATIONCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to and the benefit of U.S. Provisional Application No. 63 / 519,151, filed August 11, 2023 and titled “System and Method for Detecting Localized Cracking in Non-ideal Environments Using Digital Image Correlation,” the entire contents of which are herein incorporated by reference for all purposes.STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

[0002] This invention was made with government support under DE-NE0009033 awarded by the Department of Energy. The government has certain rights in the invention.TECHNICAL FIELD

[0003] This disclosure relates generally to optical monitoring and, in particular, to systems and methods for non-contact monitoring to detect characteristic crack signals.BACKGROUND

[0004] Quality assurance and part health monitoring are important in high-performance aerospace, healthcare, and energy generation applications. Uncertainty is introduced at various times, from a part’s start of life during manufacturing through its use in potentially harsh environments. Digital image correlation (DIC) is a non-contact optical technique that can help reduce or eliminate this uncertainty. In three-dimensional DIC, two cameras are placed in a stereo vision setup, focused on the same object, and a calibration is performed to determine their relative position. These cameras capture a set of images at a given interval. A full-field profile of the part is created using the stereo set of images and the calibration. Then, this 3D topological measurement can be compared across the time dimension to measure displacement and strain. This method can measure out-of-plane displacement and can achieve micron-level accuracy in well-calibrated cases.

[0005] DIG has been used in manufacturing to measure the deformation of materials and components. However, the accuracy of DIC in comparative implementations can be limited by several factors, including large deformations, a significant distance between the structure and the 3D-DIC cameras, poor lighting, low-resolution cameras, and the measurement system’s noise. These issues can be exacerbated by obstructions or machine geometry dictating poor mounting points, dynamic light sources such as sparks, and high heat causing visual distortions. As a result of these factors, crack detection using comparative implementations is difficult or impossible. Comparative studies have shown that if these challenges are able to be overcome, DIC is able to detect cracks on the micron scale in a variety of materials. Thus, there exists a need for systems and methods to improve 3D-DIC use in unideal conditions containing a low signal-to-noise ratio.SUMMARY

[0006] The present disclosure addresses needs in the field by presenting systems, methods, and computer readable media for providing, among other things, a) detecting localized cracking in non-ideal environments using digital image correlation and / or b) a self-supervised model for detecting cracking in three-dimensional part reconstructions imaged under non-ideal conditions.

[0007] Although example embodiments of the present disclosure are explained in some instances in detail herein, it is to be understood that other embodiments are contemplated. Accordingly, it is not intended that the present disclosure be limited in its scope to the details of construction and arrangement of components set forth in the following description or illustrated in the drawings. The present disclosure is capable of other embodiments and of being practiced or carried out in various ways.

[0008] According to one aspect of the present disclosure, an optical inspection method is provided. The method comprises receiving a time series of 3D point cloud images of a target object from an imaging system, wherein the 3D point cloud images include first directional position data, second directional position data, and change in first directional velocity data; applying a selfsupervised machine learning model to the time series of 3D point cloud images, wherein the trained machine learning model has been trained on a plurality of training point cloud images to which simulated crack analogues have been added; and identifying a mechanical failure in the target object based on an output of the machine learning model.

[0009] According to another aspect of the present disclosure, an optical inspection system is provided. The system comprises an imaging system configured to capture a time series of 3D point cloud images of a target object, wherein the 3D point cloud images include first directional position data, second directional position data, and change in first directional velocity data; and at least one processor configured to receive the time series of 3D point cloud images, apply a selfsupervised machine learning model to the time series of 3D point cloud images, wherein the trained machine learning model has been trained on a plurality of training point cloud images to which simulated cracks have been added, and identify a mechanical failure in the target object based on an output of the machine learning model.

[0010] According to another aspect of the present disclosure, a method of training a machine learning model to identify a mechanical failure in a target object is provided. The method comprises obtaining a time series of 3D point cloud images of a training object, wherein the 3D point cloud images in the time series subset do not include the mechanical failure; applying at least one synthetic primitive to respective ones of the time series subset of 3D point cloud images; and applying a pixel dropout mask to respective ones of the time series subset of 3D point cloud images.BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Various objects, features, and advantages of the disclosed subject matter can be more fully appreciated with reference to the following detailed description of the disclosed subject matter when considered in connection with the following drawings, in which like reference numerals identify like elements.

[0012] FIG. 1 shows an example of an expanding plug test and instrumentation setup in accordance with various aspects of the present disclosure.

[0013] FIG. 2 shows an example of a positive example generation with random-sized pixel dropout and an example of characteristic crack signals in accordance with various aspects of the present disclosure.

[0014] FIG. 3 shows an example of a random sized noise generation process in accordance with various aspects of the present disclosure.

[0015] FIG. 4 shows an example of an input augmentation process in accordance with various aspects of the present disclosure.

[0016] FIG. 5 shows an example of a prediction model training loop in accordance with various aspects of the present disclosure.

[0017] FIG. 6 shows an example of predictions with various random-sized noise coefficients and minimum line magnitude values in accordance with various aspects of the present disclosure.

[0018] FIG. 7 shows an example of training inputs, prediction targets, and predictions in accordance with various aspects of the present disclosure.

[0019] FIG. 8 shows an example of results from a predictor configuration predicting crack locations at timesteps before, during, and after cracks in accordance with various aspects of the present disclosure.

[0020] FIG. 9 shows an example graph of detection image sum vs. DIC timestep in accordance with various aspects of the present disclosure.

[0021] FIG. 10 shows an example graph of AE normalized energy vs. DIC timestep in accordance with various aspects of the present disclosure.

[0022] FIG. 11 shows an example of results from a sensitive predictor configuration in accordance with various aspects of the present disclosure.

[0023] FIG. 12 shows an example graph of detection image sum vs. DIC timestep in accordance with various aspects of the present disclosure.

[0024] FIG. 13 A shows an example graph of detection image sum vs. DIC timestep for a second structure in accordance with various aspects of the present disclosure.

[0025] FIG. 13B shows an example graph of AE normalized energy vs. DIC timestep for a second structure in accordance with various aspects of the present disclosure.

[0026] FIG. 14 shows an example of a machine on which various aspects of the present disclosure can be implemented.

[0027] FIG. 15 shows an example of an inspection method in accordance with various aspects of the present disclosure.

[0028] FIG. 16 shows an example of a training method in accordance with various aspects of the present disclosure.DETAILED DESCRIPTION

[0029] The detailed description set forth below in connection with the appended drawings is intended as a description of various configurations and is not intended to represent the only configurations in which the subject matter described herein may be practiced. The detailed description includes specific details to provide a thorough understanding of various aspects of the present disclosure. However, it will be apparent to those skilled in the art that the various features, concepts, and aspects described herein may be implemented and practiced without these specific details.

[0030] It should be appreciated that any element, part, section, subsection, or component described with reference to any specific embodiment above may be incorporated with, integrated into, or otherwise adapted for use with any other embodiment described herein unless specifically noted otherwise or if it should render the embodiment device non-functional. Likewise, any step described with reference to a particular method or process may be integrated, incorporated, or otherwise combined with other methods or processes described herein unless specifically stated otherwise or if it should render the embodiment method nonfunctional. Furthermore, multiple embodiment devices or embodiment methods may be combined, incorporated, or otherwise integrated into one another to construct or develop further embodiments of the invention described herein.

[0031] It should be appreciated that any of the components or modules referred to with regards to any of the present invention embodiments discussed herein, may be integrally or separately formed with one another. Further, redundant functions or structures of the components or modules may be implemented. Moreover, the various components may be communicated locally and / or remotely with any user / operator / customer / client or with any machine / system / computer / processor. Moreover, the various components may be in communication via wireless and / or hardwire or other desirable and available communication means, systems, and hardware. Moreover, various components and modules may be substituted with other modules or components that provide similar functions.

[0032] It should be appreciated that the device and related components discussed herein may take on all shapes along the entire continual geometric spectrum of manipulation of x, y, and z planes to provide and meet the environmental, anatomical, and structural demands andoperational requirements. Moreover, locations and alignments of the various components may vary as desired or required.

[0033] It should be appreciated that various sizes, dimensions, contours, rigidity, shapes, flexibility, and materials of any of the components or portions of components in the various embodiments discussed throughout may be varied and utilized as desired or required.

[0034] It should be appreciated that while some dimensions are provided on the aforementioned figures, the device may constitute various sizes, dimensions, contours, rigidity, shapes, flexibility and materials as it pertains to the components or portions of components of the device, and therefore may be varied and utilized as desired or required.

[0035] It must also be noted that, as used in the specification and the appended claims, the singular forms “a,” “an” and “the” include plural referents unless the context clearly dictates otherwise. Ranges may be expressed herein as from “about” or “approximately” one particular value and / or to “about” or “approximately” another particular value. When such a range is expressed, other exemplary embodiments include from the one particular value and / or to the other particular value.

[0036] By “comprising” or “containing” or “including” is meant that at least the named compound, element, particle, or method step is present in the composition or article or method, but does not exclude the presence of other compounds, materials, particles, or method steps, even if the other such compounds, material, particles, or method steps have the same function as what is named.

[0037] In describing example embodiments, terminology will be resorted to for the sake of clarity. It is intended that each term contemplates its broadest meaning as understood by those skilled in the art and includes all technical equivalents that operate in a similar manner to accomplish a similar purpose. It is also to be understood that the mention of one or more steps of a method does not preclude the presence of additional method steps or intervening method steps between those steps expressly identified. Steps of a method may be performed in a different order than those described herein without departing from the scope of the present disclosure. Similarly, it is also to be understood that the mention of one or more components in a device or system does not preclude the presence of additional components or intervening components between those components expressly identified.

[0038] Some references, which may include various patents, patent applications, and publications, are cited in a reference list and discussed in the disclosure provided herein. The citation and / or discussion of such references is provided merely to clarify the description of the present disclosure and is not an admission that any such reference is “prior art” to any aspects of the present disclosure described herein. For example, the devices, systems, apparatuses, modules, compositions, articles of manufacture, materials, computer program products, non-transitory computer readable medium, and methods of various embodiments of the invention disclosed herein may utilize aspects (such as devices, apparatuses, modules, systems, compositions, articles of manufacture, materials, computer program products, non-transitory computer readable medium, and methods) disclosed in the following references, applications, publications and patents and which are hereby incorporated by reference herein in their entirety (and which are not admitted to be prior art with respect to the present invention by inclusion in this section): A. U.S. Utility Patent Application Serial No. 18 / 273,658, entitled “METHODS AND SYSTEMS FOR PRODUCING DISPERSED GRAPHENE FROM SPENT LITHIUM-ION BATTERIES”, filed July 21, 2023; B. International Patent Application Serial No. PCT / US2022 / 013542, entitled “METHODS AND SYSTEMS FOR PRODUCING DISPERSED GRAPHENE FROM SPENT LITHIUM-ION BATTERIES”, filed Januaiy 24, 2022; Publication No. WO 2022 / 177701, August 25, 2022; C. International Patent Application Serial No. PCT / US2023 / 068927, entitled “Hydrogen Passivation Aided Conversion of Cotton to High-Quality Graphene”, filed June 23, 2023; D. U.S. Utility Patent Application Serial No. 17 / 617,192, entitled “SYSTEM AND METHOD OF ACCELERATING POLYMER FIBER STABILIZATION VIA IRRADIATION TREATMENT”, filed December 07, 2021; Publication No. US 2022-0235516 Al, July 28, 2022; E. International Patent Application Serial No. PCT / US2020 / 037010, entitled “SYSTEM AND METHOD OF ACCELERATING POLYMER FIBER STABILIZATION VIA IRRADIATION TREATMENT”, filed June 10, 2020; Publication No. WO 2020 / 252031, December 17, 2020; F. U.S. Utility Patent Application Serial No. 15 / 753,866, entitled “METHOD OF FORMING GRAPHENE / METAL-OXIDE HYBRID REINFORCED COMPOSITES AND PRODUCT THEREOF”, filed February 20, 2018; U.S. Patent No. 10,781,142, issued September 22, 2020; G. International Patent Application Serial No. PCT / US2016 / 047714, entitled “METHOD OF FORMING GRAPHENE / METAL- OXIDE HYBRID REINFORCED COMPOSITES AND PRODUCT THEREOF”, filed August 19, 2016; Publication No. W02017031403, February 23, 2017; and H. U.S. Utility PatentApplication Serial No. 15 / 274,557, entitled “PROCESS OF FORMING ELECTRODES AND PRODUCTS THEREOF FROM BIOMASS”, filed September 23, 2016; U.S. Patent No. 10,446,329, issued October 15, 2019.

[0039] It should be appreciated that as discussed herein, a subject may be a human or any animal. It should be appreciated that an animal may be a variety of any applicable type, including, but not limited thereto, mammal, veterinarian animal, livestock animal or pet type animal, etc. As an example, the animal may be a laboratory animal specifically selected to have certain characteristics similar to human (e.g. rat, dog, pig, monkey), etc. It should be appreciated that the subject may be any applicable human patient, for example.

[0040] The term “about,” as used herein, means approximately, in the region of, roughly, or around. When the term “about” is used in conjunction with a numerical range, it modifies that range by extending the boundaries above and below the numerical values set forth. In general, the term “about” is used herein to modify a numerical value above and below the stated value by a variance of 10%. In one aspect, the term “about” means plus or minus 10% of the numerical value of the number with which it is being used. Therefore, about 50% means in the range of 45%-55%. Numerical ranges recited herein by endpoints include all numbers and fractions subsumed within that range (e.g. 1 to 5 includes 1, 1.5, 2, 2.75, 3, 3.90, 4, 4.24, 5, and so on). Similarly, numerical ranges recited herein by endpoints include subranges subsumed within that range (e.g. 1 to 5 includes 1-1.5, 1.5-2, 2-2.75, 2.75-3, 3-3.90, 3.90-4, 4-4.24, 4.24-5, 2-5, 3-5, 1-4, 2-4, and so on). It is also to be understood that all numbers and fractions thereof are presumed to be modified by the term “about.”

[0041] It is also to be understood that any reference to an element herein using a designation such as “first,” “second,” and so forth does not limit the quantity or order of those elements, unless such limitation is explicitly stated. Rather, these designations may be used herein as a convenient method of distinguishing between two or more elements or instances of an element. Thus, a reference to first and second elements does not mean that only two elements may be employed or that the first element must precede the second element in some manner.

[0042] Also as used herein, unless otherwise limited or defined, “or” indicates a nonexclusive list of components or operations that can be present in any variety of combinations, rather than an exclusive list of components that can be present only as alternatives to each other. For example, a list of “A, B, or C” indicates options of: A; B; C; A and B; A and C; B and C; andA, B, and C. Correspondingly, the term “or” as used herein is intended to indicate exclusive alternatives only when preceded by terms of exclusivity, such as, e.g., “either,” “one of,” “only one of,” or “exactly one of.” Further, a list preceded by “one or more” (and variations thereon) and including “or” to separate listed elements indicates options of one or more of any or all of the listed elements. For example, the phrases “one or more of A, B, or C” and “at least one of A, B, or C” indicate options of: one or more A; one or more B; one or more C; one or more A and one or more B; one or more B and one or more C; one or more A and one or more C; and one or more of each of A, B, and C. Similarly, a list preceded by “a plurality of’ (and variations thereon) and including “or” to separate listed elements indicates options of multiple instances of any or all of the listed elements. For example, the phrases “a plurality of A, B, or C” and “two or more of A, B, or C” indicate options of: A and B; B and C; A and C; and A, B, and C. In general, the term “or” as used herein only indicates exclusive alternatives (e.g., “one or the other but not both”) when preceded by terms of exclusivity, such as, e.g., “either,” “one of,” “only one of,” or “exactly one of.”

[0043] The present disclosure includes a description of various methods. For any methods described, regardless of whether the method is described in conjunction with a flow diagram, it should be understood that unless otherwise specified or required by context, any explicit or implicit ordering of steps performed in the execution of a method does not necessarily imply that those steps must be performed in the order presented, but instead the steps may be performed in a different order and / or in parallel.

[0044] The present disclosure may be implemented on or with the use of computing devices including control units, processors, and / or memory elements in some examples. As used herein, a “control unit” may be any computing device configured to send and / or receive information (e.g., including instructions) to / from various systems and / or devices. A control unit may comprise processing circuitry configured to execute operating routine(s) stored in a memory. The control unit may comprise, for example, a processor, microcontroller, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), and the like, any other digital and / or analog components, as well as combinations of the foregoing, and may further comprise inputs and outputs for processing control instructions, control signals, drive signals, power signals, sensor signals, and the like. All such computing devices and environments are intended to fall within the meaning of the term “controller,” “control unit,” “processor,” or “processing circuitry” as used herein unless a different meaning is explicitly provided or otherwise clear from the context.The term “control unit” is not limited to a single device with a single processor, but may encompass multiple devices (e.g., computers) linked in a system, devices with multiple processors, special purpose devices, devices with various peripherals and input and output devices, software acting as a computer or server, and combinations of the above. In some implementations, the control unit may be configured to implement cloud processing, for example by invoking a remote processor.

[0045] Moreover, as used herein, the term “processor” may include one or more individual electronic processors, each of which may include one or more processing cores, and / or one or more programmable hardware elements. The processor may be or include any type of electronic processing device, including but not limited to central processing units (CPUs), graphics processing units (GPUs), ASICs, FPGAs, microcontrollers, digital signal processors (DSPs), or other devices capable of executing software instructions. When a device is referred to as “including a processor,” one or all of the individual electronic processors may be external to the device (e.g., to implement cloud or distributed computing). In implementations where a device has multiple processors and / or multiple processing cores, individual operations described herein may be performed by any one or more of the microprocessors or processing cores, in series or parallel, in any combination.

[0046] As used herein, the term “memory” may be any storage medium, including a nonvolatile medium, e.g., a magnetic media or hard disk, optical storage, or flash memory, including read-only memory (ROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM); a volatile medium, such as system memory, e.g., random access memory (RAM) such as dynamic RAM (DRAM), synchronous dynamic RAM (SDRAM), static RAM (SRAM), extended data out (EDO) DRAM, extreme data rate dynamic (XDR) RAM, double data rate (DDR) SDRAM, etc.; on-chip memory; and / or an installation medium where appropriate, such as software media, e.g., a CD-ROM, a DVD-ROM, a Blu-ray disc, or floppy disks, on which programs may be stored and / or data communications may be buffered. The term “memory” may also include other types of memory or combinations thereof. For the avoidance of doubt, cloud storage is contemplated in the definition of memory.

[0047] Additional descriptions of aspects of the present disclosure will now be provided with reference to the accompanying drawings. The drawings form a part hereof and show, by way of illustration, specific embodiments or examples.

[0048] The present disclosure includes a discussion directed to enabling the effective utilization of three-dimensional digital image correlation (3D-DIC) in non-optimal situations with poor signal quality. 3D-DIC is a non-destructive optical monitoring technique capable of detecting displacements, stresses, and strains, providing valuable information for manufacturing and part monitoring processes. Research has demonstrated that micron-level cracking can be detected by monitoring part deformations with DIC. However, dynamic environments commonly found in modem processes can consist of heat, electric, and light fluctuations, causing characteristic signals indicative of part cracking to become masked by noise due to poor mounting locations, sparking, or other signal or optical noise. Note that, while the present disclosure provides examples directed to the specific case of 3D-DIC images, the systems, methods, and media set forth herein may be applicable to 3D point cloud datasets captured using other imaging modalities, such as time-of- flight (ToF) imaging.

[0049] The present disclosure also includes a discussion further enabling the use of 3D- DIC by utilizing machine learning to facilitate the detection of cracks present in the 3D-DIC data, allowing for the detection of cracks not able to be seen via comparative digital image processing techniques or even by the human eye. The present disclosure develops a method utilizing data gathered during an expanding plug test, described in more detail below. In comparative examples, structure failure and cracking during an expanding plug test have been detected via 3D-DIC by computing the hoop strain and detecting when the strain exceeds a threshold value. The present disclosure, in contrast, identifies localized cracking throughout the test, allowing for a more precise indication of the exact moment a structure begins to fail and where in the structure this failure originates from. The systems, methods, and media set forth herein are not limited to testing scenarios, and may also be applied to health monitoring of a part or machine in a high intensity environment (e.g., nuclear fuel cladding), leak detection for pressure vessels and / or pipelines, and the like. While several examples set forth herein discuss crack detection, in practical implementations the present disclosure may be implemented to detect any type of mechanical failure.

[0050] The systems, methods, and media set forth herein provide for a method of detecting crack in a non-contact, optical manner. Comparative systems, methods, and media utilize contact monitoring methods, such as an acoustic emission (AE) sensor. However, AE sensors are not usable in all scenarios and conditions; for example, AE sensors are not usable at certaintemperature conditions, may be unable to detect cracks that do not produce a sufficiently large acoustic signal (e.g., for highly ductile materials), may be unable to distinguish internal cracks from exterior cracks in multi-layer objects, and so on. Comparative examples of non-contact DIC monitoring modalities, on the other hand, may only be applicable to laboratory conditions and thus unusable in real world scenarios having lower signal to noise (SNR) ratios and / or wider variability in ambient conditions.

[0051] In examples, the present disclosure implements machine learning models that are trained in a self-supervised manner based on known images that do not include examples of cracks. Comparatively, supervised machine learning methods would require training images with cracking, and thus require destructive testing. The self-supervised methods set forth herein may provide advantages in the ability to detect different types or forms of cracks that are not anticipated, and / or by detecting cracks in different types of target objects.

[0052] In a proof-of-concept example, this disclosure looks to implement techniques in non-destructive sensing to detect localized cracking in a part undergoing an expanding plug test. A self-supervised machine learning model was created by utilizing image segmentation and classification models with a dataset preprocessing approach that only requires a small dataset on the scale of 50 images known not to exhibit any cracking. This image preprocessing technique can be tuned according to different hyperparameters to yield a model that can accurately detect cracks as they form, or to yield a model that is more sensitive and can potentially predict cracking sooner according to subtle shifts, as compared to the comparative examples.

[0053] Experimental Materials, Methods, and Results

[0054] To help explain the effects and benefits provided by systems, methods, and media according to the present disclosure, experiments were performed using DIC with an expanding test plug. In the experimental setup, two Grasshoppers digital cameras (FLIR) were set up in a stereoscopic array approximately 200 mm from the target part. Calibration targets, along with the software package Vic3D-8 (Correlated Solutions), were used to determine the exact position of the cameras relative to each other and the part. The part has a high-contrast surface optical texture to allow the DIC system to achieve good correlation. Thus, half of each part facing the camera was spray painted with black and white matte finish spray paint (Rust-Oleum) to achieve a non-uniform speckle pattern. Testing setup specifics can be seen below in Table 1.TABLE 1 - DIC Test Setup Properties

[0055] The expanding plug test was performed using an Admet universal testing machine with an lOkN load cell. It mirrored the expanding plug test found in a comparative example, except a displacement-regulated loading rate of 0.5 mm / min was used. An AE sensor (Mistras 1283 AE Node) was clamped onto the upper arm of the testing machine to provide a secondary validation of crack formation times. An illustration of the test setup (left) and image acquisition procedure (right) can be seen in FIG. 1.

[0056] Calibration targets, along with the software package Vic3D-8 (Correlated Solutions), were used to determine the exact position of the cameras relative to each other and the structure. Each pair of stereoscopic images was divided into many overlapping regions. Due to the non-uniform speckle pattern, each region is distinct, and thus regions can be matched between the image pair even in the presence of structure deformation. By matching up these regions, a transform is computed between the two cameras, which, combined with the calibration information, can determine the exact X, Y, and Z coordinate positions of each of the regions, resulting in a 3D point cloud topography of the structure. By selecting the same regions at each time step, this topography can be compared across time to determine strain and velocity at each point.

[0057] As the anticipated crack width of ~1 pm is less than the size of a pixel, direct observation of cracks forming in the digital images is unlikely. Instead, point velocity was calculated by comparing the point position in subsequent images, and this value was monitoredfor a large increase in a localized area. This has the advantage of potentially having a spatially larger signal, as the crack opening also causes neighboring points to experience higher-than- normal acceleration. In turn, this can also make it more difficult to pinpoint the location and size of every crack, especially if the DIC acquisition time is too long. The DIC correlation data was used to compute the X displacement and change in X velocity of each point. The X, Y, and change in X velocity for each point were then interpolated into a regular 224 pixel by 224 pixel grid, with the value at each pixel representing the change in X velocity. These 2D images were then able to be used in traditional image-based machine learning processes.

[0058] In order to allow for easier adoption, self-supervised learning was used for model training. This allows for the use of datasets with little to no human labels. This is especially important as identifying cracks quickly in the field can be a non-trivial task and would require previously failed parts to create labeled data from. Instead, the training dataset for the model is generated through the systematic creation of simplified simulated cracks on a subset of initial images.

[0059] To create images with simulated cracks, a set of seed images consisting of the first 50 images captured by the DIC system was used as the base of the synthetic dataset (i.e., as the background on which to create the cracks). These were chosen in view of the very high likelihood that these images would be at a load too small to exhibit any cracking but would have a good sample of the noise encountered during the test. Any cracks initially present in the seed images will not be flagged as valid cracks for the model to be trained on, and thus by selecting seed images that do not include cracks, the model is not at risk of inadvertently learning to ignore some types of cracks. To create the simulated cracks, 1 to 8 random lines were created using random values for width, length, and magnitude within predefined ranges. The crack analogs used in the creation of the training set are designed to be simple in order to allow for the model to be as generic as possible, and to minimize the amount of technical information needed to be able to implement the model in a production environment. The choice of straight lines for the simulated cracks was informed by the specific experimental setup and anticipated failure mode (namely, hoop stress resulting in long vertical cracking). In other scenarios where more complex curving cracks are anticipated, the analog primitives may be modified accordingly to ensure effective crack detection.

[0060] However, the basic lines are not representative of the characteristic crack signals (as shown in image b of FIG. 2) that the model is being designed to eventually detect. Tocompensate for this, a pixel dropout mask generated from random-sized noise was created to apply to the drawn lines, causing part of the lines to disappear. Then they were multiplied by a mask generated from random-sized noise to provide pixel dropout (see FIG. 3). This process is illustrated in FIG. 2.

[0061] This random sized noise is generated by creating an image with smaller dimensions than the target dataset, fdling the image with values sampled from a uniformly random distribution (referred to as noise), and then upscaling the image to the final target dataset dimension. This results in an image of noise where the size of each sample is potentially greater than a single pixel. To determine the dimensions of the smaller image used to initially generate the noise, a target sample pixel size was selected according to Equation (1): x = 1 + | \norm , cr) 11(1) Noise Size — clip(x, a, b)

[0062] In Equation (1), clip(x, a, b) represents the clip function that returns a if a > x, b if b < x, and x otherwise. The function norm^p, o) is a function that takes a random sample from a normal distribution with mean p and standard deviation o. The values used in this experiment were p = 2, o = 1.75, a = 1, b = 18.

[0063] The dataset image dimensions were then divided by this size value to obtain the dimensions of the smaller image. Noise was generated by sampling a uniform distribution between zero and a noise coefficient hyperparameter Cn, chosen depending on the amount of noise present in the seed images. This noise image was then scaled via nearest neighbor interpolation to the dataset image dimensions. Finally, a Gaussian blur was applied to the noise image, with a kernel size dictated by the amount of noise present in the seed dataset. In the pixel dropout mask generation, this noise was then passed through a threshold of — where all pixels less than this value were set to zero, and all others were set to one.

[0064] After the first set of lines was generated and applied, a second set of lines was generated in a similar fashion. However, these lines possessed characteristics distinct from the initial set, with the range of possible values for length and width different from the first set. This second set of lines served as a set of counter-examples for the model.

[0065] After the two sets of lines were added to the initial seed image, random-sized noise was applied to the entire image. This noise was generated in the same manner as for the pixel mask dropout, except no thresholding was performed. This was done to help prevent overfitting due to the small number of original input images, as well as to provide more obfuscation of the lines we are training the model to detect. Finally, augmentation techniques commonly employed in image processing, including rotation, translation, horizontal flipping, and scaling, were applied. This overall process is demonstrated in FIG. 4.

[0066] The Pytorch segmentation library was used as the underlying framework to construct the prediction model architecture. Multiple models including MANet, UNet and UNet++ were canvassed along with multiple pre-trained encoders (Inceptionv4, Resnet, VGG, DPN, ResNeXt, and ResNeSt) to determine a preferred model -encoder combination for this example.

[0067] In the initial testing, all of the models and encoders used were able to yield acceptable results given proper hyperparameter tuning, and none stood out as exceptionally good or bad compared to the others. The combination of a MANet segmentation model with an Inceptionv4 encoder yielded the best qualitative results in detecting known cracks and was subsequently used for all tests. Loss for each prediction was computed by taking the pixel-wise mean squared error between the predicted mask ( Y) and the positive example mask (T). An illustration of the training loop is shown in FIG. 5.

[0068] In evaluating the performance of different model configurations, the following criteria were investigated: 1) the ability of the model to accurately detect the artificial line analogs present in the augmented training set, without false positives; 2) the ability of the model to not falsely detect cracks in the initial 50 seed images known not to have any cracking; 3) the ability of the model to not falsely detect cracks in other images that were known not to have cracking, but not part of the seed image set (in our experiment, this comprised of the images taken at DIC timesteps 50-99); 4) the ability of the model to detect cracks in test images that are known to have cracks (such as those immediately leading up to the failure of the structure); and 5) the model’s agreement with data obtained from the acoustic emissions and load sensors.

[0069] The three primary hyperparameters that were varied for this analysis were: the threshold amount used while generating the random-sized noise pixel dropout of generated lines (y); the noise coefficient used while overlaying noise to the input image (C)t); and the minimum magnitude used while drawing the example lines (M). Other hyperparameters investigated includethe initial learning rate for the Adam optimizer and the total number of epochs the model was trained for. Learning rate values of 10'3to 10'4tended to yield good results, and total epochs varied between 50 and 500 epochs depending on the other hyperparameters.

[0070] An array of tests were conducted to discover the potential impact each hyperparameter would have on detection rates. FIG. 6 shows results from predictions for DIC timestep 744 with y = 0.6 and Cnand M equaling one of 0.01, 0.03, or 0.05. The spatial axes display pixels instead of mm for easier viewing, but they represent the same data as if mm were used.

[0071] The Cnand M hyperparameters are related, making the effect of each dependent on the other. As shown in FIG. 6, holding Cn = 0.01 and increasing M decreases the quality of the crack detection. The opposite is true when holding Cn = 0.05 and increasing M. This suggests that there is a preferred clarity of lines, where the model must learn to detect line type objects, without learning to only detect perfect line segments.

[0072] FIG. 7 shows the effect of increasing the minimum line magnitude while holding all other hyperparameters constant on the augmented training dataset. In FIG. 7, y = 0.6, Cn= 0.01, and M equals one of 0.01, 0.03, or 0.05. While the trend in FIG. 6 for Cn = 0.01 shows that increasing minimum line magnitude decreases detection certainty, the trend in FIG. 7 shows significantly less change. In isolation this may evidence that the crack signals correspond to line magnitudes on average that are closer to 0.01 than 0.05, however, the previously mentioned trend for FIG. 6 in the case of Cn = 0.05 does not support this conclusion. Alternatively, this may evidence that with low noise coefficients and high minimum line magnitude, the model may start to align too closely towards the simplistic line analogues (and for example, start looking for the perfectly rounded ends of the line segments), and thus become less reliable for detecting more abstract line type shapes (such as those that anticipated for cracks).

[0073] In the Cn = 0.05 case for FIG. 6, the lowest line magnitude scenario (M=0.01) contains noise that is five times the minimum line magnitude. This could result in training input - target pairs where the target lines are mostly or completely obscured by noise, resulting in impossible to predict scenarios. This could have the effect of causing the model to have many false positives as it tries to match non-existent signals, or for the model to effectively be trained on less data. As the minimum line magnitude increases, the signal to noise ratio would increase, reducingthese occurrences, and providing better detection certainty, which is consistent with the seen results.

[0074] The predictor had the poorest performance in FIG. 6 at Cn= M = 0.03, which seems not to follow the previously mentioned trends. This could suggest another factor at play, potentially concerning the size or amount of noise inherent in the data. The configuration that was the most consistent with the five evaluation criteria was y = 0.5, Cn= 0.03, M = 0.05. FIG. 8 demonstrates the model’s ability to accurately detect cracking in non-seed, non-augmented images (criteria 4) while not having large amounts of false positives in the preceding and subsequent images.

[0075] Multiplying the prediction mask with the input image creates a detection image for each timestep. This represents the detected cracks, as well as their magnitude. By summing each of these detection images, a plot is created which gives an overview of when in the process significant cracking occurs. A plot for the detection sums for the balanced model configuration can be seen in FIG. 9.

[0076] Referring back to the above-noted performance criteria, it can be seen that the model displays little to no false positives on images from the seed dataset (DIC timesteps 0 - 49) and images that are not expected to have cracking, and also not part of the seed dataset (DIC timesteps 50-99), satisfying criteria 2 and 3.

[0077] The acoustic emissions data obtained during the test (see FIG. 10) was evaluated and compared against this detection value plot to determine the prediction model’s ability to detect cracks accurately (criteria 5). Both plots indicate that significant cracking starts at approximately timestep 749 (shown as a vertical red line). Additionally, both plots show small detections around timestep 450. This increases confidence that the model could serve as a primary detection or complement other sensors as a detection validation mechanism. Additionally, it is apparent from FIG. 10 that the AE setup displays a larger amount of random noise compared to that of proposed model as shown in FIG. 9. Depending on the specific environment, this could cause either the AE sensor setup, or the proposed model to detect cracking earlier. A lower threshold applied to the structure 1 AE signal may have detected unwanted part loading as shown as a slight increase in AE signal slightly after the 600 DIC timestep, allowing earlier warning of a potential fault state. However, this signal is not always present (as is the case in FIG. 13), and can be signs of non- critical loading, such as those around the 300 DIC timestep. Likewise, the deformation of the speckle coating may appear as characteristic signals that the proposed model is able to detect butnot the AE sensor, allowing for the proposed model to potentially achieve earlier detection in certain specific applications.

[0078] Changing the model’s parameters to y = 0.7, Cn= 0.03, M = 0.01 increased its sensitivity and allowed the model to detect some of the initial formations of the detected cracks. As seen in FIG. 11, the eventual cracking observed in D1C timestep 744 is seen in timestep 743 (taken 0.25 seconds before). These cracks were previously not able to be detected. These detections may become more reliable by including more data in the model’s analysis for each timestep (such as giving it information about the previous timestep or the change in point Y velocity).

[0079] In FIG. 9, the original model’s detection value remains near zero for most of the test. The start of significant cracking takes the form of a slight baseline increase (around timestep 700) and higher values for a few points, eventually leading to high but varied values across all points. This is contrasted with the detection value plot for the more sensitive model in FIG. 12. In the more sensitive model, the model’s detection value has a higher mean and standard deviation in the pre-cracking phase. Additionally, instead of the sharp value increase exhibited in FIG. 9 at the onset of cracking, the sensitive model’s detection values seem to increase smoothly and gradually (although once cracking has started, a similar jump in detection value is seen). This gradual increase could be a negative side effect of the increased sensitivity in the form of additional false positives, or it could indicate small, less significant cracks forming on the structure’s surface or in the paint. However, no matter the cause, the smoother transition of values around the onset of significant cracking may increase the difficulty of detecting the onset of cracking with the sensitive model.

[0080] One advantage of the self-supervised nature of the model is that, because a majority of the input images are synthetic noise or positive / negative examples, a model can be trained on a given structure and then later used on another structure in a similar (but not necessarily identical) environment. To test this, the model previously trained with y = 0.7, Cn= 0.03, M = 0.01 was used without additional training to detect cracking in a different structure (referred to as structure 2). FIG. 13 A and FIG. 13B contain the detection value plot and AE energy plot for structure 2. Despite not being trained on any data from structure 2, the model’s detection value plot and AE plot both experienced a jump at around timestep 853 (shown as a red vertical line). In contrast to the results obtained from testing on the original structure (referred to above as structure 1), the detection plot seems to only increase at this value instead of slightly before it. This difference could be due tomodifications in the testing environments, or the original model might not have been sensitive enough for structure 2 in general. The inaccuracies encountered when transferring the model to new structures may be remedied by fine-tuning the model with some of structure 2’s initial testing data or using different hyperparameters.

[0081] Example Implementations

[0082] FIG. 14 is a block diagram illustrating an example of a machine upon which one or more aspects of embodiments of the present invention can be implemented.

[0083] Referring to FIG. 14, an aspect of an embodiment of the present invention includes, but not limited thereto, a system, method, and computer readable medium that provides: a) detecting localized cracking in non-ideal environments using digital image correlation and / or b)self-supervised model for detecting cracking in parts detected via digital image correlation under non-ideal conditions, which illustrates a block diagram of an example machine 400 upon which one or more embodiments (e.g., discussed methodologies) can be implemented (e.g., run).

[0084] Examples of machine 400 can include logic, one or more components, circuits (e.g., modules), or mechanisms. Circuits are tangible entities configured to perform certain operations. In an example, circuits can be arranged (e.g., internally or with respect to external entities such as other circuits) in a specified manner. In an example, one or more computer systems (e.g., a standalone, client or server computer system) or one or more hardware processors (processors) can be configured by software (e g., instructions, an application portion, or an application) as a circuit that operates to perform certain operations as described herein. In an example, the software can reside (1) on a non-transitory machine readable medium or (2) in a transmission signal. In an example, the software, when executed by the underlying hardware of the circuit, causes the circuit to perform the certain operations.

[0085] In an example, a circuit can be implemented mechanically or electronically. For example, a circuit can comprise dedicated circuitry or logic that is specifically configured to perform one or more techniques such as discussed above, such as including a special-purpose processor, an FPGA, or an ASIC. In an example, a circuit can comprise programmable logic (e.g., circuitry, as encompassed within a general-purpose processor or other programmable processor) that can be temporarily configured (e.g., by software) to perform the certain operations. It will be appreciated that the decision to implement a circuit mechanically (e.g., in dedicated andpermanently configured circuitry), or in temporarily configured circuitry (e.g., configured by software) can be driven by cost and time considerations.

[0086] Accordingly, the term “circuit” is understood to encompass a tangible entity, be that an entity that is physically constructed, permanently configured (e.g., hardwired), or temporarily (e.g., transitorily) configured (e.g., programmed) to operate in a specified manner or to perform specified operations. In an example, given a plurality of temporarily configured circuits, each of the circuits need not be configured or instantiated at any one instance in time. For example, where the circuits comprise a general -purpose processor configured via software, the general- purpose processor can be configured as respective different circuits at different times. Software can accordingly configure a processor, for example, to constitute a particular circuit at one instance of time and to constitute a different circuit at a different instance of time.

[0087] In an example, circuits can provide information to, and receive information from, other circuits. In this example, the circuits can be regarded as being communicatively coupled to one or more other circuits. Where multiple of such circuits exist contemporaneously, communications can be achieved through signal transmission (e.g., over appropriate circuits and buses) that connect the circuits. In embodiments in which multiple circuits are configured or instantiated at different times, communications between such circuits can be achieved, for example, through the storage and retrieval of information in memory structures to which the multiple circuits have access. For example, one circuit can perform an operation and store the output of that operation in a memory device to which it is communicatively coupled. A further circuit can then, at a later time, access the memory device to retrieve and process the stored output. In an example, circuits can be configured to initiate or receive communications with input or output devices and can operate on a resource (e.g., a collection of information).

[0088] The various operations of method examples described herein can be performed, at least partially, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors can constitute processor-implemented circuits that operate to perform one or more operations or functions. In an example, the circuits referred to herein can comprise processor-implemented circuits.

[0089] Similarly, the methods described herein can be at least partially processor- implemented. For example, at least some of the operations of a method can be performed by oneor processors or processor-implemented circuits. The performance of certain of the operations can be distributed among the one or more processors, not only residing within a single machine, but deployed across a number of machines. In an example, the processor or processors can be located in a single location (e.g., within a home environment, an office environment or as a server farm), while in other examples the processors can be distributed across a number of locations.

[0090] The one or more processors can also operate to support performance of the relevant operations in a "cloud computing" environment or as a "software as a service” (SaaS). For example, at least some of the operations can be performed by a group of computers (as examples of machines including processors), with these operations being accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., Application Program Interfaces (APIs).)

[0091] Example embodiments (e.g., apparatus, systems, or methods) can be implemented in digital electronic circuitry, in computer hardware, in firmware, in software, or in any combination thereof. Example embodiments can be implemented using a computer program product (e.g., a computer program, tangibly embodied in an information carrier or in a machine readable medium, for execution by, or to control the operation of, data processing apparatus such as a programmable processor, a computer, or multiple computers).

[0092] A computer program can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a software module, subroutine, or other unit suitable for use in a computing environment. A computer program can be deployed to be executed on one computer or on multiple computers at one site or distributed across multiple sites and interconnected by a communication network.

[0093] In an example, operations can be performed by one or more programmable processors executing a computer program to perform functions by operating on input data and generating output. Examples of method operations can also be performed by, and example apparatus can be implemented as, special purpose logic circuitry (e.g., an FPGA or an ASIC).

[0094] The computing system can include clients and servers. A client and server are generally remote from each other and generally interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In embodiments deploying aprogrammable computing system, it will be appreciated that both hardware and software architectures require consideration. Specifically, it will be appreciated that the choice of whether to implement certain functionality in permanently configured hardware (e.g., an ASIC), in temporarily configured hardware (e.g., a combination of software and a programmable processor), or a combination of permanently and temporarily configured hardware can be a design choice. Below are set out hardware (e.g., machine 400) and software architectures that can be deployed in example embodiments.

[0095] In an example, the machine 400 can operate as a standalone device or the machine 400 can be connected (e.g., networked) to other machines. In a networked deployment, the machine 400 can operate in the capacity of either a server or a client machine in server-client network environments. In an example, machine 400 can act as a peer machine in peer-to-peer (or other distributed) network environments. The machine 400 can be a personal computer (PC), a tablet PC, a set-top box (STB), a Personal Digital Assistant (PDA), a mobile telephone, a web appliance, a network router, switch or bridge, or any machine capable of executing instructions (sequential or otherwise) specifying actions to be taken (e.g., performed) by the machine 400. Further, while only a single machine 400 is illustrated, the term “machine” shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.

[0096] Example machine (e.g., computer system) 400 can include a processor 402 (e.g., a CPU, a GPU, or both), a main memory 404, and a static memory 406, some or all of which can communicate with each other via a bus 408. The machine 400 can further include a display unit 410, an alphanumeric input device 412 (e.g., a keyboard), and a user interface (UI) navigation device 411 (e.g., a mouse). In an example, the display unit 810, input device 417 and UI navigation device 414 can be a touch screen display. The machine 400 can additionally include a storage device (e.g., drive unit) 416, a signal generation device 418 (e.g., a speaker), a network interface device 420, and one or more sensors 421, such as a global positioning system (GPS) sensor, compass, accelerometer, or other sensor.

[0097] The storage device 416 can include a machine readable medium 422 on which is stored one or more sets of data structures or instructions 424 (e.g., software) embodying or utilized by any one or more of the methodologies or functions described herein, and may be any type of “memory” described above. The instructions 424 can also reside, completely or at least partially,within the main memory 404, within static memory 406, or within the processor 402 during execution thereof by the machine 400. In an example, one or any combination of the processor 402, the main memory 404, the static memory 406, or the storage device 416 can constitute machine readable media.

[0098] While the machine readable medium 422 is illustrated as a single medium, the term “machine readable medium” can include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) that configured to store the one or more instructions 424. The term “machine readable medium” can also be taken to include any tangible medium that is capable of storing, encoding, or carrying instructions for execution by the machine and that cause the machine to perform any one or more of the methodologies of the present disclosure or that is capable of storing, encoding or carrying data structures utilized by or associated with such instructions. The term “machine readable medium” can accordingly be taken to include, but not be limited to, solid-state memories, and optical and magnetic media. Specific examples of machine readable media can include non-volatile memory, including, by way of example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices); magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and CD- ROM and DVD-ROM disks.

[0099] The instructions 424 can further be transmitted or received over a communications network 426 using a transmission medium via the network interface device 420 utilizing any one of a number of transfer protocols (e.g., frame relay, IP, TCP, UDP, HTTP, etc.). Example communication networks can include a local area network (LAN), a wide area network (WAN), a packet data network (e.g., the Internet), mobile telephone networks (e.g., cellular networks), Plain Old Telephone (POTS) networks, and wireless data networks (e.g., IEEE 802.11 standards family known as Wi-Fi®, IEEE 802.16 standards family known as WiMax®), peer-to-peer (P2P) networks, among others. The term “transmission medium” shall be taken to include any intangible medium that is capable of storing, encoding or carrying instructions for execution by the machine, and includes digital or analog communications signals or other intangible medium to facilitate communication of such software.

[0100] FIG. 15 shows a flow chart of an example optical inspection method 500. The method 500 includes an operation 510 of receiving a time series of 3D point cloud images of a target object from an imaging system. The 3D point cloud images may include first directionalposition data, second directional position data, and change in first directional velocity data. In examples, the 3D point cloud images are based on image data captured by a 3D-DIC imaging system. Next, at operation 520, a self-supervised ML model (e.g., as described above) is applied to the time series of 3D point cloud images. The ML model may be trained on a plurality of training point cloud images. The training point cloud images may, in their raw state, exhibit no mechanical failures but may have simulated mechanical failures added thereto. The method 500 further includes an operation 530 of identifying a mechanical failure (e.g., a crack) in the target object based on an output of the machine learning model. In some examples, operations 510-530 are performed in real time using a pre-trained (or partially pre-trained) ML model.

[0101] The trained ML model applied in operation 520 may be trained using a training method of the type set forth above. FIG. 16 shows a flow chart of an example training method 600. The method 600 includes an operation 610 of obtaining a time series subset (e.g., a subset of the images obtained at operation 510, or another time series of images) of 3D point cloud images of a training object, wherein the 3D point cloud images in the time series subset do not include the mechanical failure. The training object may be the target object itself, or may be an object that is similar to the target object (e.g., a distinct part of the same or a similar design / type). The method 600 further includes an operation 620 of applying at least one synthetic primitive to the 3D point cloud training images obtained at operation 610. The at least one synthetic primitive may be a simulation of an expected form of the mechanical failure. For example, if the target object is being monitored for the formation of linear (or curved) cracks, the synthetic primitive may be a simulation of linear (or curved) cracks.

[0102] At operation 630, a pixel dropout mask is applied to the results of operation 620; that is, to the training images that have had one or more synthetic primitives added thereto. The pixel dropout mask may be based on applying a random size noise to the time series subset of 3D point cloud images including the at least one synthetic primitive. Then, using the training images with the synthetic primitive(s) and pixel dropout mask applied thereto, the ML model may be trained. In the training process, one or more hyperparameters (e.g., a pixel dropout threshold, a noise coefficient, and / or a minimum magnitude of the at least one synthetic primitive) may be varied or determined.

[0103] As set forth above, a self-supervised model was created using machine learning segmentation methods and data preprocessing to detect cracking in structures detected via digitalimage correlation under non-ideal conditions. This method does not need human-labeled data other than a small dataset known not to have cracks, enhancing its adaptability, and making it more viable as a field technique that can be set up without expensive equipment or significant labor. Additionally, the model’s sensitivity can be adjusted based on some preprocessing parameters, allowing users to tailor the model for their specific application. The model also demonstrated acceptable performance on data from structures other than the one initially trained on, decreasing the time and effort required for implementation.

[0104] In summary, while the present invention has been described with respect to specific embodiments, many modifications, variations, alterations, substitutions, and equivalents will be apparent to those skilled in the art. The present invention is not to be limited in scope by the specific embodiment described herein. Indeed, various modifications of the present invention, in addition to those described herein, will be apparent to those of skill in the art from the foregoing description and accompanying drawings. Accordingly, the invention is to be considered as limited only by the spirit and scope of the disclosure (and claims) including all modifications and equivalents.

[0105] Still other embodiments will become readily apparent to those skilled in this art from reading the above-recited detailed description and drawings of certain exemplary embodiments. It should be understood that numerous variations, modifications, and additional embodiments are possible, and accordingly, all such variations, modifications, and embodiments are to be regarded as being within the spirit and scope of this application. For example, regardless of the content of any portion (e.g., title, field, background, summary, abstract, drawing figure, etc.) of this application, unless clearly specified to the contrary, there is no requirement for the inclusion in any claim herein or of any application claiming priority hereto of any particular described or illustrated activity or element, any particular sequence of such activities, or any particular interrelationship of such elements. Moreover, any activity can be repeated, any activity can be performed by multiple entities, and / or any element can be duplicated. Further, any activity or element can be excluded, the sequence of activities can vary, and / or the interrelationship of elements can vary. Unless clearly specified to the contrary, there is no requirement for any particular described or illustrated activity or element, any particular sequence or such activities, any particular size, speed, material, dimension or frequency, or any particular interrelationship of such elements. Accordingly, the descriptions and drawings are to be regarded as illustrative in nature, and not as restrictive. Moreover, when any number or range is described herein, unlessclearly stated otherwise, that number or range is approximate. When any range is described herein, unless clearly stated otherwise, that range includes all values therein and all sub ranges therein. Any information in any material (e.g., a United States / foreign patent, United States / foreign patent application, book, article, etc.) that has been incorporated by reference herein, is only incorporated by reference to the extent that no conflict exists between such information and the other statements and drawings set forth herein. In the event of such conflict, including a conflict that would render invalid any claim herein or seeking priority hereto, then any such conflicting information in such incorporated by reference material is specifically not incorporated by reference herein.

Claims

CLAIMSWhat is claimed is:

1. An optical inspection method, comprising: receiving a time series of 3D point cloud images of a target object from an imaging system, wherein the 3D point cloud images include first directional position data, second directional position data, and change in first directional velocity data; applying a self-supervised machine learning model to the time series of 3D point cloud images, wherein the machine learning model has been trained on a plurality of training point cloud images to which simulated failures have been added; and identifying a mechanical failure in the target object based on an output of the machine learning model.

2. The method of claim 1, wherein the 3D point cloud images are based on image data captured by a 3D digital image correlation (DIC) imaging system.

3. The method of claim 1, wherein the mechanical failure is a crack.

4. The method of claim 1, wherein the plurality of training point cloud images are generated by: obtaining a time series subset of 3D point cloud images of a training object, wherein the 3D point cloud images in the time series subset do not include the mechanical failure; applying at least one synthetic primitive to respective ones of the time series subset of 3D point cloud images; and applying a pixel dropout mask to respective ones of the time series subset of 3D point cloud images.

5. The method of claim 4, wherein the at least one synthetic primitive is at least one simulation of an expected form of the mechanical failure.

6. The method of claim 4, wherein the pixel dropout mask is based on applying a random size noise to the time series subset of 3D point cloud images including the at least one synthetic primitive.

7. The method of claim 4, wherein the training object is the target object.

8. The method of claim 4, wherein the trained machine learning model has been trained using a predetermined value for at least one hyperparameter, wherein the at least one hyperparameter includes at least one of a pixel dropout threshold, a noise coefficient, or a minimum magnitude of the at least one synthetic primitive.

9. The method of claim 1, wherein the operations of receiving, applying, and identifying are performed in real time.

10. An optical inspection system, comprising: an imaging system configured to capture a time series of 3D point cloud images of a target object, wherein the 3D point cloud images include first directional position data, second directional position data, and change in first directional velocity data; and at least one processor configured to: receive the time series of 3D point cloud images, apply a self-supervised machine learning model to the time series of 3D point cloud images, wherein the trained machine learning model has been trained on a plurality of training point cloud images to which simulated cracks have been added, and identify a mechanical failure in the target object based on an output of the machine learning model.

11. The system of claim 9, wherein the imaging system is a 3D digital imaging correlation DIC imaging system.

12. The system of claim 9, wherein the plurality of training point cloud images are generated by:obtaining a time series subset of 3D point cloud images of a training object, wherein the 3D point cloud images in the time series subset do not include the mechanical failure; applying at least one synthetic primitive to respective ones of the time series subset of 3D point cloud images; and applying a pixel dropout mask to respective ones of the time series subset of 3D point cloud images.

13. The system of claim 12, wherein the at least one synthetic primitive is at least one simulation of an expected form of the mechanical failure.

14. The system of claim 12, wherein the pixel dropout mask is based on applying a random size noise to the time series subset of 3D point cloud images including the at least one synthetic primitive.

15. The system of claim 12, wherein the trained machine learning model has been trained using a predetermined value for at least one hyperparameter, wherein the at least one hyperparameter includes at least one of a pixel dropout threshold, a noise coefficient, or a minimum magnitude of the at least one synthetic primitive.

16. A method of training a machine learning model to identify a mechanical failure in a target object, the method comprising: obtaining a time series of 3D point cloud images of a training object, wherein the 3D point cloud images in the time series subset do not include the mechanical failure; applying at least one synthetic primitive to respective ones of the time series subset of 3D point cloud images; and applying a pixel dropout mask to respective ones of the time series subset of 3D point cloud images.

17. The method of claim 16, wherein the at least one synthetic primitive is at least one simulation of an expected form of the mechanical failure.

18. The method of claim 16, wherein the pixel dropout mask is based on applying a random size noise to the time series of 3D point cloud images including the at least one synthetic primitive.

19. The method of claim 16, wherein the training object is the target object.

20. The method of claim 16, further comprising setting a value for at least one hyperparameter, wherein the at least one hyperparameter includes at least one of a pixel dropout threshold, a noise coefficient, and a minimum magnitude of the at least one synthetic primitive.

Images