Multi-modal industrial defect detection method based on chaotic neural network

Abstract

The invention discloses a multi-modal industrial defect detection method based on a chaotic neural network, and the method comprises the steps: obtaining and aligning industrial defect data of multiple modals, carrying out the feature extraction and fusion through a multi-branch neural network, and generating a unified defect feature vector; taking the feature vector as an initial condition of a preset high-dimensional chaotic system, and generating a binary hash code which is extremely sensitive to the initial condition through iterative operation; in a pre-constructed defect instance library in which historical defect hash codes and associated production data thereof are stored, rapid retrieval is performed by using the hash codes to obtain historical instances similar to the current defect, and the associated production data thereof are aggregated to perform automatic root cause analysis; according to the method, the sensitivity of a chaotic system to an initial value is utilized to amplify feature tiny differences, so that the precision and efficiency of nonlinear feature retrieval are remarkably improved, seamless integration from efficient retrieval to intelligent root cause analysis is realized, and a data closed loop from defect detection to process optimization is formed.

Description

Technical Field

[0001] This invention relates to the field of industrial data processing technology, and in particular to a multimodal industrial defect detection method based on chaotic neural networks. Background Technology

[0002] In the modern industrial system centered on Industry 4.0 and intelligent manufacturing, product quality control is a crucial factor determining a company's core competitiveness. To achieve zero tolerance for defects in industrial products, defect detection technology is evolving from single-modality to multi-modal integration. Traditional detection methods based on single visual images often suffer from limitations such as insufficient information dimensions and susceptibility to environmental factors like lighting when dealing with complex textured surfaces, microscopic three-dimensional morphological defects, or subsurface internal flaws. Therefore, multi-modal detection technologies that integrate high-resolution visual images, 3D point clouds or depth maps, infrared thermal imaging, and ultrasonic scanning have emerged. These technologies utilize advanced models such as deep neural networks (DNNs), especially convolutional neural networks (CNNs), to extract features and deeply fuse these multi-modal data, significantly improving the accuracy and robustness of defect identification, and have become the mainstream research paradigm in the field of industrial defect detection. However, defect detection is not the end of quality control, but rather the starting point for data-driven process optimization and root cause analysis (RCA). Once a defect is accurately identified, the key to achieving closed-loop control of the production process and improving product qualification rate lies in how to quickly and accurately retrieve similar defect instances from massive amounts of historical production data, and associate them with corresponding production process data (such as equipment operating conditions, process parameters, material batches, etc.) to gain insight into the root cause of the defect.

[0003] However, while existing technologies have made some progress in defect detection and retrieval, several technical bottlenecks remain to be addressed in their practical industrial applications. First, there is a fundamental contradiction between retrieval efficiency and accuracy. The multimodal fusion features extracted by deep neural networks are typically high-dimensional vectors. In industrial historical databases containing tens or even hundreds of millions of records, performing precise K-nearest neighbor (KNN) searches leads to the "curse of dimensionality," with computational complexity increasing exponentially, failing to meet the real-time response requirements of industrial sites. To address this, academia and industry have proposed approximate nearest neighbor search algorithms such as Locality Sensitive Hash (LSH), which accelerate retrieval by mapping high-dimensional vectors to low-dimensional binary hash codes. However, LSH and its variants largely rely on random linear projections, while the feature distribution of industrial defects often exhibits a complex, highly nonlinear manifold structure. This linear hash function, independent of data distribution, severely disrupts the local neighborhood structure of the original feature space during the mapping process, leading to a significant decrease in retrieval accuracy and recall, and making it highly susceptible to missing crucial similar cases. Second, there is a "semantic gap" in root cause analysis within existing retrieval systems. Even when existing systems can retrieve defect cases with similar visual or physical features, they typically stop at simply presenting these similar images or data fragments. The system itself cannot automatically and intelligently correlate and mine these apparent similarities with multi-dimensional production data such as process parameters and equipment status hidden in the background database. This means that root cause analysis still heavily relies on engineers' personal experience, requiring them to manually switch between different systems (such as quality management systems and manufacturing execution systems, MES) and compare data. This process is cumbersome, inefficient, and makes it difficult to uncover deep-seated problems caused by non-obvious, multi-factor coupling.

[0004] Therefore, a technological paradigm is needed that can overcome the efficiency and accuracy challenges of high-dimensional nonlinear feature retrieval, and seamlessly integrate efficient retrieval with intelligent root cause analysis, thereby achieving a data closed loop from defect detection to process optimization. Summary of the Invention

[0005] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0006] In view of the aforementioned existing problems, this invention is proposed. Therefore, this invention provides a multimodal industrial defect detection method based on chaotic neural networks to solve the problems mentioned in the background art.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a multimodal industrial defect detection method based on chaotic neural networks, comprising:

[0008] Acquire and align industrial defect data of multiple modalities, extract and fuse the industrial defect data of the multiple modalities by creating a multi-branch neural network, and generate a unified defect feature vector;

[0009] The defect feature vector is used as the initial condition of a preset high-dimensional chaotic system. The chaotic system is operated on iteratively to obtain the chaotic state after the evolution of the chaotic system. Based on the chaotic state, a binary hash code is generated to represent the defect.

[0010] In a pre-built defect instance library that stores historical defect hash codes and their associated production data, the binary hash codes are used for retrieval to obtain historical defect instances similar to the current defect, and root cause analysis is performed based on the associated production data of the historical defect instances.

[0011] As a preferred embodiment of the multimodal industrial defect detection method based on chaotic neural networks described in this invention, at least one nonlinear activation function of the multi-branch neural network is a chaotic modulation activation function, the parameters of which are dynamically generated by a low-dimensional chaotic mapping.

[0012] As a preferred embodiment of the multimodal industrial defect detection method based on chaotic neural networks described in this invention, the high-dimensional chaotic system is a coupled mapping lattice system, and the defect feature vector is mapped to the initial state matrix of the coupled mapping lattice system through a trainable seeding layer.

[0013] As a preferred embodiment of the multimodal industrial defect detection method based on chaotic neural networks described in this invention, the process of generating the binary hash code includes: sampling the system state at multiple preset time points in the evolution of the coupled mapping lattice system to form a trajectory vector, and performing binarization processing on the trajectory vector to generate the binary hash code.

[0014] As a preferred embodiment of the multimodal industrial defect detection method based on chaotic neural networks described in this invention, the binarization process employs a binarization method, mapping elements in the trajectory vector that are greater than or equal to the median of the vector to a first binary value, and elements that are less than the median of the vector to a second binary value.

[0015] As a preferred embodiment of the multimodal industrial defect detection method based on chaotic neural networks described in this invention, the defect instance library is constructed using a multi-indexed hash table structure.

[0016] As a preferred embodiment of the multimodal industrial defect detection method based on chaotic neural networks described in this invention, the retrieval process includes: dividing the binary hash code to be queried into multiple sub-hash codes, using the sub-hash codes to search for candidate instances in the corresponding hash table, and sorting them by calculating the Hamming distance between the complete hash code of the candidate instance and the hash code to be queried.

[0017] As a preferred embodiment of the multimodal industrial defect detection method based on chaotic neural networks described in this invention, the root cause analysis includes: aggregating the associated production data of the retrieved historical defect instances, automatically identifying process parameters strongly correlated with such defects using data mining algorithms, and generating a root cause analysis report.

[0018] Compared with existing technologies, the beneficial effects of the invention are:

[0019] 1. This invention generates hash codes by using deep feature vectors as initial conditions for high-dimensional chaotic systems. It leverages the extreme sensitivity of chaotic systems to initial conditions to amplify minute differences between features, thereby improving the retrieval accuracy and recall rate for complex nonlinear defect features while ensuring reduced retrieval efficiency.

[0020] 2. This invention enhances the neural network's ability to dynamically capture and nonlinearly express complex defect patterns by introducing a chaotic modulation activation function into the neural network, thereby improving the robustness and discriminativeness of the extracted features by utilizing the dynamic non-periodic characteristics of chaos.

[0021] 3. This invention deeply integrates efficient defect instance retrieval with automated root cause correlation analysis, realizing a data closed loop from "defect characterization" to "process tracing". It can automatically discover common production factors behind similar defect cases, providing engineers with data-driven decision support, thereby shortening the fault diagnosis and process optimization cycle. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0023] Figure 1 This is a flowchart illustrating the overall process of a multimodal industrial defect detection method based on a chaotic neural network according to an embodiment of the present invention. Detailed Implementation

[0024] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0025] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0026] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0027] This invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of this invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not adhering to the usual scale. Furthermore, the schematic diagrams are merely examples and should not be construed as limiting the scope of protection of this invention. In actual fabrication, the three-dimensional spatial dimensions of length, width, and depth should be included.

[0028] Furthermore, in the description of this invention, it should be noted that the terms "upper," "lower," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used solely for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. In addition, the terms "first," "second," or "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0029] Unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" in this invention should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; similarly, they can refer to mechanical connections, electrical connections, or direct connections, or indirect connections through an intermediate medium, or internal connections between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0030] Example 1

[0031] Reference Figure 1This is the first embodiment of the present invention, which provides a multimodal industrial defect detection method based on a chaotic neural network, comprising:

[0032] S1. Acquire and align industrial defect data of multiple modalities, extract and fuse industrial defect data of multiple modalities by creating a multi-branch neural network, and generate a unified defect feature vector;

[0033] Specifically, multimodal industrial defect data can be obtained from data modalities, including but not limited to visual modalities, topological modalities, internal structure modalities, and physical response modalities;

[0034] Specifically, visual modalities include: 2D high-resolution images, line scan images, 3D stereoscopic images, etc.

[0035] Specifically, topological modes include: three-dimensional point cloud or depth map data generated by laser contour scanning, structured light, or confocal microscopy;

[0036] Specifically, internal structural modes include: internal or subsurface data of an object obtained by X-ray (such as CT scan), ultrasound scan, terahertz imaging, etc.

[0037] Specifically, physical response modes include: infrared thermal imaging sequences under thermal excitation, acoustic emission timing signals, electromagnetic response signals, etc.

[0038] It's important to note that in multimodal industrial inspection, data alignment refers to the precise spatial and temporal transformation and mapping of multiple data sources from different sensors, with different coordinate systems, resolutions, and even physical dimensions, unifying them into a common, standardized reference framework. Its core objective is to establish a unified, high-precision spatiotemporal coordinate system. Spatial alignment addresses the "where" question, typically selecting a mode as an "anchor point" or "reference coordinate system," causing data from all other modes to align to this reference mode. In industrial scenarios, topological modes (3D point clouds / mesh) are the ideal reference modes because their characteristics directly describe the physical shape of the product. Therefore, the acquired high-precision 3D point cloud data can be defined as a standard product object coordinate system, which is fixed and represents the "body" of the product. For example, the origin (0, 0, 0) can be taken as the geometric center of the product, and its principal axes can be used as coordinate axes.

[0039] Specifically, using camera calibration methods, a mapping relationship is established between 2D image pixels (u, v) in each aligned visual modality and 3D spatial points (X, Y, Z) in the product object coordinate system. For the intrinsic parameter calibration of the camera calibration method, in the offline stage, a standard calibration board (such as a checkerboard) is used to determine the intrinsic parameters of each visual camera, including focal length, principal point coordinates, and lens distortion coefficients. For the extrinsic parameter calibration of the camera calibration method, there are two cases: for fixed cameras, the fixed extrinsic parameters of the camera are calculated by photographing a workpiece with known 3D surface points; for mobile cameras (such as those on a robotic arm), the instantaneous pose of the camera in the product object coordinate system needs to be calculated by combining the forward kinematics of the robotic arm (knowing the pose of the end flange) and the pre-calibrated "hand-eye relationship" (the pose of the camera relative to the flange) as the extrinsic parameter calibration of the camera.

[0040] It should be noted that, once the intrinsic and extrinsic parameters are available, for any pixel in the image, a ray originating from the optical center of the camera and passing through the pixel can be calculated through back projection. The intersection of this ray with the three-dimensional product model in the coordinate system of the product object is the three-dimensional spatial point corresponding to that pixel.

[0041] Specifically, a mapping relationship is established between voxel coordinates generated by CT scans in the internal structural modality and 3D spatial points (X, Y, Z) in the product object coordinate system through 3D registration. Two 3D data sets are acquired: one from the surface model of the topological modality and the other from the internal volume data of the CT scan (which has its own scanning coordinate system). Reference markers are attached or fabricated on the product surface; these markers must be clearly visible in both the 3D surface scan and the CT image (e.g., high-density small metal spheres). By matching the coordinates of the corresponding reference markers in the two coordinate systems (voxel coordinates and product object coordinate system), the rigid body transformation matrix between the two coordinate systems can be solved using mathematical methods (such as singular value decomposition SVD). Furthermore, if there are no reference markers, the iterative nearest point algorithm can be used to extract the object's surface from the CT data. Then, through iterative optimization, an optimal rotation and translation transformation is found that minimizes the distance between the CT-extracted surface and the surface model of the topological modality. After obtaining the transformation matrix, any voxel coordinate in the CT volume data can be transformed into the surface model of the topological modality, thereby achieving the unification of the internal and external structures.

[0042] Specifically, if imaging data exists in the physical response mode, the processing procedure is the same as that for the visual mode. If it is not imaging data, the three-dimensional coordinates of each acoustic emission sensor in the surface model of the topological mode are measured first. Then, when an internal defect (such as microcrack propagation) generates an acoustic emission signal (which will arrive at different sensors at different times), the source coordinates (X, Y, Z) of the three-dimensional spatial point where the acoustic emission event occurs are calculated by analyzing the time difference of the signal arriving at each sensor. These coordinates are in the surface model of the topological mode.

[0043] It should be noted that in addition to data alignment, time alignment is also required. The goal of time alignment is to solve the "when" problem, so as to ensure that all modal data has a unified and high-precision timestamp.

[0044] Specifically, time alignment employs a high-precision network time synchronization protocol. This involves deploying a high-precision clock server within the system network of an industrial environment. All data acquisition devices (including industrial cameras, 3D scanners, CT controllers, acoustic emission acquisition cards, etc.) operating systems or hardware on this server are required to support and be configured as NTP (Network Time Protocol) or the more precise PTP (Precision Time Protocol, IEEE 1588) clients. Each device is configured to retrieve the current timestamp from its synchronized clock server when acquiring each frame of image, each scan, or each signal sample, and then package and record this timestamp along with the data.

[0045] Furthermore, after completing the spatiotemporal alignment of industrial defect data of multiple modalities, this invention constructs and trains a specially designed multi-branch neural network to achieve efficient extraction and deep fusion of multi-modal information. The multi-branch neural network includes four parallel feature extraction branches and a feature fusion module, wherein each feature extraction branch is used to process different categories of modalities.

[0046] Specifically, for visual modalities, the first feature extraction branch adopts a mature convolutional neural network (CNN) architecture, which extracts high-level semantic features about defect texture, shape and contour from image pixels through multi-layer convolution, pooling and non-linear activation operations.

[0047] Specifically, for topological modalities, the second feature extraction branch adopts a network architecture suitable for processing unordered point sets, such as PointNet++ or Dynamic Graph Convolutional Network (DGCNN), because such networks can operate directly on the point cloud, learn its local geometry and global shape distribution, and can effectively capture three-dimensional morphological features such as pits and scratches.

[0048] Specifically, for internal structural modalities, the third feature extraction branch uses a three-dimensional convolutional neural network (3D-CNN), whose convolution kernel is three-dimensional and can slide and calculate simultaneously in three dimensions, thereby directly extracting the three-dimensional spatial features of defects such as internal cavities and inclusions from voxel data.

[0049] Specifically, for physical response modes, the fourth feature extraction branch employs variants of recurrent neural networks (RNNs), such as long short-term memory networks (LSTM) or gated recurrent units (GRUs), to capture the dynamic changes and dependencies of the signal in the time dimension.

[0050] Furthermore, to enhance the ability of multi-branch neural networks to express complex nonlinear defect features, in at least one of the feature extraction branches, the traditional static nonlinear activation function (such as ReLU, Sigmoid) is replaced with a chaotic modulation activation function M(s):

[0051] M(s) = α(t) × B(x)

[0052] Where B(x) represents the basic activation function (such as ReLU, Sigmoid), and α(t) represents a modulation coefficient dynamically generated by a low-dimensional chaotic mapping, with a value range of (0, 1);

[0053] Furthermore, for this modulation coefficient, the classic Logistic mapping can be used as the low-dimensional chaotic mapping. During each forward propagation of the multi-branch neural network, a new pseudo-random value, α(t), is generated iteratively from the Logistic mapping for the feature extraction branch in the network. At this time, α(t) is no longer a fixed or learnable scalar, but a value dynamically generated by the chaotic mapping in each training batch (forward propagation process). Based on the non-periodicity and ergodicity of the chaotic sequence, the multi-branch neural network can be trained in a controlled, deterministic random manner to explore the loss of the multi-branch neural network, which plays a role of "dynamic regularization". This can effectively help the model escape sharp local minima and learn more generalizable feature representations.

[0054] It should be noted that since the features extracted from each branch have different dimensions and semantic emphasis, the task of the feature fusion module is to effectively integrate these feature vectors from heterogeneous branches to generate the final unified defect feature vector.

[0055] Specifically, the fusion module employs an attention-based fusion strategy, aligning the feature vectors output from all feature extraction branches in terms of dimension. These aligned feature vectors are then input into an attention network, which learns to dynamically assign a weight coefficient to the feature vector of each modality branch. This weight coefficient represents the relative importance of modal information in the comprehensive judgment of the current defect. For example, for a tiny surface scratch, the visual and topological modalities have higher weights; while for an internal pore, the internal structure modalities have higher weights. Finally, the feature vectors of all modal branches are weighted and summed according to their weights, and then nonlinearly mapped through one or more fully connected layers, ultimately outputting a fixed-dimensional, unified defect feature vector.

[0056] Optionally, to make the generated defect feature vectors more accurate, a metric learning-based training strategy can be used to train the entire multi-branch neural network (including all extraction branches and fusion modules) end-to-end. Specifically, the training strategy can use a triplet loss function, where the neural network selects a triplet sample {Anchor, Positive, Negative} from the training set each time during training. Here, Anchor and Positive represent two different samples of the same type of defect, while Negative represents samples of different types of defects. The goal of this loss function is to minimize the distance between Anchor and Positive, while maximizing the distance between Anchor and Negative, and ensuring that there is a safe boundary between Anchor, Positive, and Negative. This forces the neural network to learn a feature space in which feature vectors of similar defects will be geometrically close to each other, while feature vectors of dissimilar defects will be geometrically far apart.

[0057] S2. Using the defect feature vector as the initial condition of the preset high-dimensional chaotic system, the chaotic system is operated on iteratively to obtain the chaotic state after the evolution of the chaotic system. Based on the chaotic state, a binary hash code for characterizing the defect is generated.

[0058] It should be noted that a compact binary hash code can be generated by utilizing the defect feature vector, enabling fast retrieval in a massive industrial defect database. However, traditional hashing methods suffer from accuracy loss when dealing with highly nonlinear data distributions like industrial defects. To address this issue, this invention employs a hash encoding mechanism based on a high-dimensional chaotic system. Preferably, this high-dimensional chaotic system is a coupled mapping lattice system, because it consists of a large number of mutually coupled nonlinear mapping units (lattices), exhibiting extremely complex spatiotemporal chaotic behavior, making it suitable for constructing hash functions that are extremely sensitive to initial conditions.

[0059] Specifically, a typical coupled-mapping lattice system can be described by the following equation:

[0060]

[0061] Where i represents the position number in the system, and if the system has N nonlinear mapping units, the value of i usually ranges from 1 to N (or from 0 to N-1); t and t+1 both represent the time points of evolution, for example, calculating the state at time t+1 from the system state at time t; x t (i) represents the state value at time t and the i-th cell; f(·) represents the local nonlinear mapping function, which describes the dynamic behavior of each nonlinear mapping unit itself, that is, how the state of the nonlinear mapping unit will evolve without the influence of its neighbors. It can be initially set to 4 to ensure that each nonlinear mapping unit is in a completely chaotic state; ε represents the coupling coefficient, which usually takes values ​​between [0.1, 0.5] and controls the mutual influence between adjacent nonlinear mapping units. The initial value is set to 0.3 to maintain the local chaotic characteristics while achieving sufficient spatial coupling.

[0062] It should be noted that since the defect feature vector cannot be directly assigned a value because the dimension and numerical range of the defect feature vector may not match the initial state matrix required by the coupled mapping lattice system, this invention introduces a trainable seeding layer in order to use the defect feature vector as the initial condition of the coupled mapping lattice system.

[0063] Specifically, the seeding layer is a small feedforward neural network (using one or two fully connected layers). It takes a defect feature vector as input and outputs a matrix that meets the initial state requirements of the coupled mapping lattice system. For example, if the coupled mapping lattice system contains N nonlinear mapping units, the seeding layer maps the defect feature vector into a 1×N feature vector and uses a tanh or sigmoid activation function to constrain the value of each element in the feature vector to the legal range of the coupled mapping lattice system state variables (e.g., [0, 1]). The seeding layer and the multi-branch neural network are trained using the same end-to-end method. In this way, the neural network can not only learn how to extract highly discriminative features, but also how to embed the defect feature vector into the coupled mapping lattice system. This allows similar defect feature vectors to generate similar evolutionary trajectories in the coupled mapping lattice system, while dissimilar defect feature vectors generate drastically different evolutionary trajectories, making full use of the "butterfly effect" of initial conditions in chaotic systems.

[0064] Furthermore, since the process of generating binary hash codes does not only take the final state, but samples the entire spatiotemporal dynamic process of the coupled mapping lattice system evolution, after mapping the defect feature vector to the initial state matrix of the coupled mapping lattice system through the seeding layer, it is necessary to perform iterative evolution according to the above coupled mapping lattice system equations.

[0065] Specifically, the iterative evolution process is as follows:

[0066] The coupled mapping lattice system is subjected to a certain number of preheating iterations (determined based on the number of actual industrial defect samples, for example, 1000 iterations). The purpose of the preheating iterations is to allow the system to escape the initial transient state and enter the inherent chaotic attractor region.

[0067] After warming up, continue iterating M times. At K preset time points (e.g., the 101st, 105th, 110th, etc.), the state of all N nonlinear mapping units of the system is sampled in a snapshot manner to form a K×N state matrix. Then, all elements of the matrix are connected into a long vector, which is the chaotic trajectory vector. The spatiotemporal evolution pattern determined by the initial defect feature vector can be captured through the chaotic trajectory vector.

[0068] The chaotic trajectory vector can be converted into a binary hash code. However, in order to enhance the robustness and balance of the hash code, this invention adopts a preferred binarization method. Specifically, by calculating the median of the entire chaotic trajectory vector, the elements in the trajectory vector that are greater than or equal to the median are mapped to the first binary value (e.g., "1"), while the elements that are less than the median are mapped to the second binary value (e.g., "0").

[0069] It should be noted that through the above iterative evolution process, each input defect feature vector is uniquely and deterministically converted into a fixed-length binary hash code, which contains all the information of the defect feature vector. Furthermore, due to the amplification characteristics of the chaotic system, the binary hash code also has high discriminative power.

[0070] S3. In a pre-built defect instance library that stores historical defect hash codes and their associated production data, use binary hash codes to search for historical defect instances similar to the current defect, and perform root cause analysis based on the associated production data of the historical defect instances.

[0071] It should be noted that a structured defect instance library needs to be built in advance before the system goes live or during operation. This instance library is mainly used to store historical defect information.

[0072] Specifically, this defect instance library is constructed using a multi-index hash table structure, and the construction process is as follows:

[0073] For each historical defect instance in the instance library, it is configured to contain three types of core information: multimodal data of the defect itself: namely, the original visual, topological, internal structure, and physical response modalities; the defect's hash code: namely, the generated binary hash code; and associated production data: snapshot data of the production process closely related to the time point of the defect's occurrence, obtained from sources such as Manufacturing Execution System (MES), equipment control system (such as PLC logs), and Quality Management System (QMS). This snapshot data includes, but is not limited to: equipment ID, operator number, material batch number, processing timestamp, and a series of key process parameters, such as the cutting speed, feed rate, and spindle temperature of CNC machine tools, the melt temperature, holding pressure, and cooling time of injection molding machines, and the current, voltage, and welding speed of welding robots.

[0074] Furthermore, to avoid the binary hash code performing a linear scan of the entire instance library during retrieval, this invention divides each bit of the hash code into L non-overlapping sub-hash codes. For example, it can be divided into 8 16-bit sub-hash codes. Accordingly, L (i.e. 8) independent hash tables need to be constructed.

[0075] Furthermore, for each historical defect instance, the sub-hash code obtained from its segmentation is used as the key and stored in the corresponding hash table. In each independent hash table, the key is the sub-hash code, and the value is a list that stores the unique identifiers of all historical defect instances with that sub-hash code. Each identifier ID uniquely points to the complete information of that instance (multimodal data, defect hash code, associated production data).

[0076] Furthermore, when a new defect to be queried is detected and a corresponding binary hash code is generated, the retrieval process will use the above-mentioned multi-index hash table structure to divide each hash code to be queried into a sub-hash code in the same way as when building the instance library. Then, the sub-hash code is used to search in the corresponding hash table (the search method is based on sequential search). Since the search complexity of the hash table is close to O(1), the process will be very fast. When the corresponding candidate instance ID list is retrieved from the hash table (for example, if there are 8 hash tables, then there are also 8 candidate instances retrieved), these lists are merged into a set of non-repeating candidate instance IDs.

[0077] It should be noted that the obtained candidate instance ID set is only the instances that are similar to the query defect in the "local" hash, and is not the most similar global result. Therefore, in order to obtain the most similar global result, it is necessary to sort them.

[0078] Furthermore, the hash codes of all candidate instances are retrieved in batches from the instance library, and the Hamming distance between them and the hash code to be queried is calculated. The candidate instances are sorted in ascending order of Hamming distance, and the P closest candidate instances are returned as the final similarity defect results.

[0079] It should be explained that the Hamming distance refers to the number of different characters at corresponding positions between two binary strings of equal length. The smaller the distance, the higher the similarity.

[0080] Specifically, by aggregating all production data associated with historical defect instances represented by P similar defect results retrieved from the defect instance library, a small, high-quality production dataset highly correlated with the current defect type can be obtained. By applying data mining algorithms to analyze this production dataset, process parameters or operations strongly correlated with this type of defect can be automatically identified.

[0081] Furthermore, data mining algorithms can use association rule mining (such as the Apriori algorithm) to discover anomalous combinations of parameters in production datasets. For example, it can be found that "when the melt temperature is above 235°C and the cooling time is less than 15 seconds, the probability of 'silver thread' defects is 85%". Alternatively, ensemble learning models (such as random forests) or gradient boosting trees (such as XGBoost) can be used to train the production dataset to predict the occurrence of defects. Then, by analyzing the internal structure of the model (such as feature importance scores), the contribution of each process parameter (such as pressure, speed, and temperature) to the occurrence of this type of defect can be quantified, and the analysis results can be integrated into a structured and visualized root cause analysis report for output.

[0082] Specifically, the analysis report can highlight the combination of process parameters that cause such defects and the abnormal range of the corresponding process parameters in the form of charts (such as parameter distribution histograms, correlation heatmaps, Pareto charts).

[0083] It should be noted that the visualized root cause analysis report provides clear, intuitive, and data-driven decision support for field engineers, greatly shortening the troubleshooting and process optimization cycle and realizing an intelligent closed loop from defect detection to process improvement.

[0084] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0085] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0086] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0087] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0088] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0089] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A multimodal industrial defect detection method based on chaotic neural networks, characterized in that, include: Acquire and align industrial defect data of multiple modalities, extract and fuse the industrial defect data of the multiple modalities by creating a multi-branch neural network, and generate a unified defect feature vector; The defect feature vector is used as the initial condition of a preset high-dimensional chaotic system. The chaotic system is operated on iteratively to obtain the chaotic state after the evolution of the chaotic system. Based on the chaotic state, a binary hash code is generated to represent the defect. In a pre-built defect instance library that stores historical defect hash codes and their associated production data, the binary hash codes are used for retrieval to obtain historical defect instances similar to the current defect, and root cause analysis is performed based on the associated production data of the historical defect instances.

2. The multimodal industrial defect detection method based on chaotic neural networks as described in claim 1, characterized in that, At least one nonlinear activation function of the multi-branch neural network is a chaotic modulation activation function, the parameters of which are dynamically generated by a low-dimensional chaotic mapping.

3. The multimodal industrial defect detection method based on chaotic neural networks as described in claim 1 or 2, characterized in that, The high-dimensional chaotic system is a coupled mapping lattice system, in which the defect feature vector is mapped to the initial state matrix of the coupled mapping lattice system through a trainable seeding layer.

4. The multimodal industrial defect detection method based on chaotic neural networks as described in claim 3, characterized in that, The process of generating the binary hash code includes: sampling the system state at multiple preset time points during the evolution of the coupled mapping lattice system to form a trajectory vector, and performing binarization processing on the trajectory vector to generate the binary hash code.

5. The multimodal industrial defect detection method based on chaotic neural networks as described in claim 4, characterized in that, The binarization process employs a binarization method, mapping elements in the trajectory vector that are greater than or equal to the median of the vector to a first binary value, and elements that are less than the median of the vector to a second binary value.

6. The multimodal industrial defect detection method based on chaotic neural networks as described in claim 1, characterized in that, The defect instance library is constructed using a multi-indexed hash table structure.

7. The multimodal industrial defect detection method based on chaotic neural networks as described in claim 1, characterized in that, The retrieval process includes: dividing the binary hash code to be queried into multiple sub-hash codes, using the sub-hash codes to search for candidate instances in the corresponding hash table, and sorting them by calculating the Hamming distance between the complete hash code of the candidate instances and the hash code to be queried.

8. The multimodal industrial defect detection method based on chaotic neural networks as described in claim 1 or 7, characterized in that, The root cause analysis includes: aggregating the associated production data of the retrieved historical defect instances, using data mining algorithms to automatically identify process parameters strongly correlated with this type of defect, and generating a root cause analysis report.

Images