Workshop production quality online detection system based on machine vision and AI

Abstract

The application discloses a workshop production quality online detection system based on machine vision and AI, and particularly relates to the field of machine vision detection, and comprises the following steps: through a multi-modal data acquisition and preprocessing module, physical signals, process parameters and actual quality values of products are synchronously acquired, and after data cleaning, semantic alignment and standardization processing, structured data blocks of associated product information are generated; then, through a feature extraction and mixed vector generation module, physical information features and multi-modal data driven features are extracted and fused into a mixed vector, a network is trained to output a comprehensive quality index and a corresponding time sequence; finally, through a quality field energy coupling decision module, a quality toughness coefficient and a process fluctuation conduction coefficient are calculated, three types of energy fields, i.e., compliance, consistency and risk, are defined, a comprehensive decision value is obtained through coupling, four-level quality decisions, i.e., high-quality, qualified, to-be-checked and rejected, are output according to the value and field characteristics, production circulation is guided, and the accuracy and efficiency of quality detection are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of machine vision inspection technology, and more specifically, to an online inspection system for workshop production quality based on machine vision and AI. Background Technology

[0002] As the manufacturing industry transforms towards intelligent and large-scale operations, the pace of workshop production has accelerated significantly, and the complexity of product structures and the diversity of processes are constantly increasing. This has led to increasingly stringent demands for the real-time, comprehensive, and accurate nature of quality inspection. Currently, workshop production quality inspection mainly relies on two technical approaches: First, traditional manual sampling, which relies on the experience of inspectors to judge basic indicators such as product appearance and dimensions. While this can cover some obvious defects, it is limited by sampling rates and cannot cover occasional defects in large-scale continuous production. Furthermore, inspection efficiency decreases significantly with increasing working hours, and subjective judgment differences can easily lead to inconsistent standards. Second, single-equipment inspection, which often uses tools such as 2D vision cameras, single-point temperature sensors, or pressure sensors. These can only independently collect surface texture, local temperature, or single-point process parameters, and cannot simultaneously acquire multi-dimensional information such as 3D geometry, material composition, and the entire process parameter sequence. Although some companies have attempted to introduce AI algorithms to assist inspection, they mostly focus on a single data dimension and have not achieved deep correlation between multi-source inspection data and the production process. The overall inspection system is difficult to adapt to the core needs of modern production for comprehensive and multi-dimensional quality control throughout the entire process.

[0003] However, it still has some drawbacks in practical use, such as:

[0004] 1. Data collection dimension is limited and synchronization is poor: Existing technologies mostly collect 2D images or single physical signals, lacking integration of multimodal data such as 3D geometry, spectral materials, and temperature fields. In addition, the timestamps of data collection from different devices are inconsistent, the data correlation is low, and it is impossible to fully reflect the product quality status, making it easy to miss key defect information.

[0005] 2. Feature extraction is one-sided and lacks fusion: It extracts image texture or simple physical features alone, without combining physical mechanism features such as product heat conduction and structural stress with multimodal data-driven features. The feature dimension is single, making it difficult to accurately depict the essence of complex defects, which leads to misjudgment and omission in subsequent quality assessment.

[0006] 3. The quality decision-making logic is subjective and lacks quantification: it relies on preset thresholds or simple classification models for decision-making, does not consider the dynamic impact of process fluctuations on quality, does not build a quantitative decision-making system with multi-dimensional energy field coupling, and the decision results are greatly affected by experience, resulting in insufficient stability and reliability under different operating conditions.

[0007] 4. Poor adaptability and weak scalability: For different scenarios such as hot processing and assembly, hardware and algorithm parameters need to be significantly adjusted, and it is impossible to adapt to the detection needs of multiple scenarios through a unified framework; when adding new defect types, the model iteration cost is high and it is difficult to respond quickly to changes in production processes. Summary of the Invention

[0008] To overcome the aforementioned deficiencies of the prior art, the present invention provides an online inspection system for workshop production quality based on machine vision and AI, which solves the problems mentioned in the background art through the following solutions.

[0009] To achieve the above objectives, the present invention provides the following technical solution: an online inspection system for workshop production quality based on machine vision and AI, comprising:

[0010] Multimodal data acquisition and preprocessing module: Synchronously acquires physical signals, process parameters, and actual quality values ​​of the target product, and ensures data timestamp consistency through clock synchronization and trigger control; after preprocessing the acquired raw data, it generates standardized data blocks;

[0011] Feature extraction and hybrid vector generation module: Based on standardized data blocks, physical information features and data-driven features are extracted, fused by attention, partitioned, and then the physical information features and data-driven features are concatenated to generate hybrid vectors; the hybrid vectors are used to train the network model and output the Comprehensive Quality Index (CQI) and CQI time series.

[0012] Quality Field Energy Coupling Decision Module: Based on the output of the preceding module, it calculates the quality toughness coefficient and the process fluctuation transmission coefficient, and defines the compliance energy field, the consistency energy field, and the risk energy field; based on historical data, it determines the energy transfer coefficient matrix and the field phase difference, couples and calculates the total energy, and generates the standardized comprehensive decision value QCI; based on the QCI and field characteristics, it outputs the quality decision.

[0013] The technical effects and advantages of this invention are as follows:

[0014] 1. Multimodal data synchronous acquisition with comprehensive dimensions: It integrates 2D image, 3D geometry, spectral material, temperature field and process parameter acquisition, and ensures time stamp consistency through PTP clock synchronization and FPGA trigger control, comprehensively covering multi-dimensional quality information of products, laying a complete data foundation for accurate detection;

[0015] 2. Deep feature fusion and accurate characterization: Combine physical mechanisms to extract thermophysical and geometric physical features, fuse multimodal data-driven features, and form a high-dimensional hybrid vector through attention fusion to accurately capture the essential attributes of defects, effectively improve feature representation capabilities, and reduce misjudgment and missed judgment rates;

[0016] 3. Quantitative decision-making system with dynamic adaptation: By constructing three types of energy fields—compliance, consistency, and risk—through quality resilience and process fluctuation transmission coefficient, and combining historical data to calculate a comprehensive decision value, the impact of process fluctuations on quality is quantified. The decision-making logic is objective and can dynamically respond to changes in production conditions.

[0017] 4. Multi-scenario adaptability and strong scalability: It adopts a modular design. The temperature field acquisition unit is used in the hot processing scenario, and the physical feature extraction is adapted through zero filling in other scenarios. When adding a new defect type, only training samples need to be supplemented. There is no need to make major adjustments to the framework. It adapts to multiple scenarios and the model iteration cost is low. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the overall structure of the present invention.

[0019] Figure 2 This is a schematic diagram of the multimodal data acquisition and preprocessing module of the present invention.

[0020] Figure 3 This is a schematic diagram of the feature extraction and hybrid vector generation module of the present invention.

[0021] Figure 4 This is a schematic diagram of the mass field energy coupling decision module structure of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] refer to Figures 1-4 The online inspection system for workshop production quality based on machine vision and AI shown includes:

[0024] Multimodal data acquisition and preprocessing module: Synchronously acquires physical signals, process parameters, and actual quality values ​​of the target product, and ensures data timestamp consistency through clock synchronization and trigger control; after preprocessing the acquired raw data, it generates standardized data blocks;

[0025] It should be further explained that the multimodal data acquisition and preprocessing module includes a data acquisition submodule and a preprocessing submodule;

[0026] This embodiment requires specific explanation of the data acquisition submodule, which includes: a physical signal acquisition unit, a process parameter acquisition unit, and an actual quality value acquisition unit. The specific acquisition process is as follows:

[0027] By deploying various types of sensors and interfaces at the testing station, the physical signals, process parameters, and actual quality values ​​of the product are collected simultaneously to generate the raw dataset.

[0028] Physical signal acquisition unit:

[0029] 2D Surface Image Acquisition: A 5-megapixel, 2592×1944 resolution, 60fps area array camera is deployed 500-800mm directly above the inspection station, equipped with an 8mm fixed-focus lens and a ring LED light source with switchable brightness. When the product enters the station through the photoelectric sensor, the camera simultaneously captures three images with brightness levels of 3000 lux, 6000 lux, and 10000 lux, respectively. The Laplacian variance algorithm is used to select one image with no reflection and complete details in BMP format. This image contains information on the shape and location of the product's surface texture and planar defects. The product texture includes roughness and gloss, and the planar defects include scratches, stains, and missing characters.

[0030] 3D geometric shape acquisition: A line laser scanner with a point cloud density of 100 points / mm, a scanning speed of 500 lines / second, and a laser wavelength of 660nm is deployed at a 30° and -45° angle to the side of the camera, and is triggered synchronously with the 2D camera. The scanner performs line scanning on the product surface, generating single-frame point cloud data containing three-dimensional coordinates x, y, z in mm, with an accuracy of ±0.01mm, and can extract spatial features of surface height differences and three-dimensional defects.

[0031] Hyperspectral material acquisition: A hyperspectral imager is deployed on the other side of the camera. The parameters of the hyperspectral imager are: wavelength 400-1000nm, spectral resolution 5nm, 120 bands, spatial resolution 640×512, and the detection angle is perpendicular to the product surface. The hyperspectral cube of the product surface is acquired simultaneously in ENVI format, which contains the original reflectance values ​​at different wavelengths, and can reflect the spectral characteristics of material composition differences and hidden sensory defects.

[0032] Temperature field acquisition: An infrared thermal imager is deployed 1-2m to the side of the welding or heat treatment station. Its parameters are: 640×512 resolution, temperature measurement range of 20-500℃, 30fps frame rate, and error of ±2℃. Ten frames of temperature field matrix are continuously acquired. The peak temperature moment of the molten pool or heat treatment area is located by the peak detection algorithm. The selected frame of data includes the temperature characteristics of heat input uniformity and thermal defects.

[0033] Process parameter acquisition unit:

[0034] An OPC UA protocol interface module is deployed in the industrial control box next to the testing station. It is connected to the equipment PLC via hardwire to collect process parameters that are highly relevant to quality. The sampling frequency is 30Hz, and time series data is generated with a total of 100 sampling points, including timestamps, accurate to milliseconds. The data is stored according to product ID, parameter name, timestamp, and numerical format to reflect the dynamic fluctuations of the process.

[0035] Actual quality value acquisition unit:

[0036] Online testing equipment is deployed at the production line: dimensional errors are checked using a coordinate measuring machine for machined parts; tensile strength is checked using a tensile testing machine for welded parts; and insulation performance is checked using an insulation resistance meter for electronic components. Testing personnel record the product ID, testing item, measured value, and time as the baseline for quality judgment.

[0037] It should be further explained that the specific process of synchronizing the collected data is as follows:

[0038] A PTP precision clock server with an accuracy of ±100ns is deployed next to the workstation to bind the clock source of all sensors, PLCs and acquisition devices to ensure that the timestamp error is ≤1ms; a 20ms periodic trigger signal is generated by the FPGA to control the start and stop synchronization of the sensors and parameter acquisition.

[0039] This embodiment requires a detailed explanation of the preprocessing submodule process as follows:

[0040] Data cleaning:

[0041] 2D image cleaning: Zhang's calibration method (10×10 checkerboard grid, grid spacing 20mm) was used to correct distortion (pixel error ≤0.5); the foreground was segmented using the Otsu algorithm (product area extraction accuracy ≥99%); the contrast of low-light areas was enhanced using the CLAHE algorithm.

[0042] 3D point cloud cleaning: statistical filtering (neighborhood points = 50, standard deviation = 3) removes outliers (integrity ≥ 98%); voxel filtering (0.05 mm³) downsamples to 500,000 points; aligns to the design coordinate system based on the reference hole coordinates (error ≤ 0.02 mm).

[0043] Hyperspectral data cleaning: Reflectance is corrected using a whiteboard with 99% reflectance and dark current; bands with SNR≥30 are selected and 80-100 are retained; Gaussian filtering (σ=1.0) is used to smooth the spectral curve and remove high-frequency noise.

[0044] Temperature field cleaning: subtract the ambient temperature field to eliminate interference; threshold segmentation retains the effective area.

[0045] Semantic alignment:

[0046] Defect region localization: The bounding boxes (x1, y1, x2, y2) and type labels (10 typical defect types) of defects in 2D images are manually labeled; the RPN network (IOU≥0.85) is trained to automatically generate the coordinates of the defect region, guiding the 3D point cloud, hyperspectral, and temperature field data to be located in the same physical region (spatial deviation≤1mm).

[0047] It should be further explained that after preprocessing the collected raw data, standardized data blocks are generated, which specifically include:

[0048] 2D image patch: cropped to 256×256×3, pixel values ​​normalized to 0-1;

[0049] 3D point cloud fragment: sampled to 1024 points, coordinates normalized to [-1, 1];

[0050] Hyperspectral cube: cut to 64×64×30, reflectance normalized to 0-1;

[0051] Temperature field matrix (heat processing scenario): trimmed to 256×256, temperature values ​​normalized to [-1, 1].

[0052] Additional information: Product ID, timestamp, defect label, process parameter time series P(t), and actual quality value, packaged as an HDF5 file, can be directly input into the feature extraction and hybrid vector generation module.

[0053] Feature extraction and hybrid vector generation module: Based on standardized data blocks, physical information features and data-driven features are extracted, fused by attention, partitioned, and then the physical information features and data-driven features are concatenated to generate hybrid vectors; the hybrid vectors are used to train the network model and output the Comprehensive Quality Index (CQI) and CQI time series.

[0054] It should be further explained that the feature extraction and hybrid vector generation module includes a physical information feature extraction unit, a data-driven feature extraction unit, and a hybrid vector generation unit;

[0055] This embodiment requires specific explanation of the physical information feature extraction unit, which includes: thermophysical feature extraction and geometric physical feature extraction;

[0056] It should be further explained that the specific process of thermophysical feature extraction is as follows:

[0057] Input a 256×256 temperature field matrix with temperature values ​​normalized to [-1, 1] after preprocessing by the multimodal data acquisition and preprocessing module, for 10 consecutive frames.

[0058] Preprocessing steps: Perform a two-dimensional Fourier transform on the temperature field matrix of each frame to obtain the frequency domain features (containing the spatial frequency components of the temperature distribution); calculate the temperature gradient (horizontal direction ∂T / ∂x, vertical direction ∂T / ∂y), peak temperature (the highest temperature value in the region), and temperature uniformity index (standard deviation / mean) for each frame.

[0059] Network training: A 3D convolutional network is used, with the input being a 4D tensor (256×256×10×1) of 10 frames of temperature field matrices stacked together.

[0060] Network structure: 4 layers of 3D convolutional layers, with 3×3×3 convolutional kernels, stride 1, padding=1, each layer is followed by a BN layer and a ReLU activation function, and finally outputs a 128-dimensional feature vector through global average pooling;

[0061] Training data: Temperature field sequences of 1000 heat-processed products (including 500 normal samples and 500 defective samples).

[0062] Physical constraints: The loss function incorporates a regularization term from the heat conduction equation, and its specific mathematical function is as follows:

[0063] Here, 0.1 is a weighting coefficient used to balance the ratio of physical constraint loss to other model losses, preventing physical constraints from excessively suppressing the learning of data features. Let T be the partial derivative of temperature T with respect to time t, which physically represents the rate of change of temperature over time. The thermal diffusivity of the material, Let T be the Laplace operator for temperature T.

[0064] Output features: 128-dimensional thermophysical feature vectors, including features with clear physical meanings such as heat conduction efficiency (temperature diffusion rate), molten pool stability (peak temperature fluctuation), and heat input uniformity (gradient distribution).

[0065] It needs to be further explained that the specific process of geometric physical feature extraction is as follows:

[0066] Input: 3D point cloud fragments output by the multimodal data acquisition and preprocessing module;

[0067] Physical parameter calculations: Extract crack length a, depth c, dimensional deviation δ, and surface roughness, and calculate the stress concentration factor. , The radius of curvature at the tip;

[0068] Network training: A fully connected network is used, with inputs including crack length, depth, dimensional deviation, surface roughness, stress concentration factor, and point cloud density. The network structure consists of 3 fully connected layers (6, 64, 128, 128), each followed by a BN layer and a ReLU activation function.

[0069] Training data: 3D point clouds of 1000 products and corresponding online detection physical parameters;

[0070] Loss function: mean squared error (the deviation between predicted and measured parameters);

[0071] Training metric: Validation set parameter prediction error ≤ 5%.

[0072] Output features: 128-dimensional geometric and physical feature vectors, including fatigue risk factor R and structural integrity index (based on the ratio of dimensional deviation to design tolerance, with a value range of 0-1).

[0073] Physical feature fusion operation: Concatenate the 128-dimensional thermophysical feature vector with the 128-dimensional geometric physical feature vector to form a 256-dimensional physical information feature vector. ;

[0074] Special handling: In non-thermal processing scenarios, only geometric and physical features are used, and the 256 dimensions are supplemented by zero padding;

[0075] Failure acceleration feature extraction: from Extract failure acceleration features from the last 64 dimensions It reflects the crack propagation rate and structural failure trend, with a value range of 0-1 (learned from historical failure data through an SVM model).

[0076] It should be further explained that the fatigue risk factor is calculated as follows:

[0077] Calculation of actual working stress amplitude based on process parameters acquired by the multimodal data acquisition and preprocessing module Further adjust the actual stress amplitude: Based on the material SN curve formula: Calculate the theoretical fatigue life under the corresponding stress amplitude: Then calculate the life safety factor. Finally, the fatigue risk factor was obtained: , where m and C are the fatigue strength index and material fatigue constant, respectively, both of which are inherent properties of the material and are measured experimentally, and N is the number of stress cycles.

[0078] This embodiment requires a detailed explanation of the data-driven feature extraction unit process as follows:

[0079] Dataset construction: 10,000 samples were selected from the output of the multimodal data acquisition and preprocessing module, including 10 types of defects: scratches, cracks, dents, bumps, stains, missing characters, uneven plating, assembly misalignment, dimensional deviation, and material impurities. 1,000 samples were selected for each type and divided into training set and validation set in an 8:2 ratio.

[0080] Data augmentation: Randomly rotate (±15°), randomly scale (0.8-1.2 times), and add Gaussian noise (σ=0.01) to the training set images to expand the sample size to 20,000.

[0081] Multi-branch network training:

[0082] 2D Feature Branch (ResNet18):

[0083] Input a 2D image block (256×256×3) into the multimodal data acquisition and preprocessing module;

[0084] Network structure:

[0085] Layer 1: A 7×7 convolutional layer with 64 channels, stride 2, and padding 3; followed by a BN layer and ReLU activation function with 3×3 max pooling and stride 2.

[0086] Layers 2-5: 4 residual blocks, each containing 2 layers of 3×3 convolutions, with the number of channels being 64, 128, 256, and 512 respectively;

[0087] Layer 6: Global average pooling and 256-channel 1×1 convolution;

[0088] Output: 256-dimensional texture feature vector, including edge gradient direction, color distribution entropy, local binary mode (LBP) statistics, etc.

[0089] Training objective: Defect classification using cross-entropy loss, validation set accuracy ≥ 98%.

[0090] 3D feature branches:

[0091] Input: 3D point cloud fragments preprocessed by the multimodal data acquisition and preprocessing module (1024 points × 3 coordinates, normalized to [-1, 1]);

[0092] Network structure:

[0093] Layer 1: T-Net, 3×3 transformation matrix, aligns point cloud pose;

[0094] Layers 2-4: MLP, with 64, 128, and 256 channels respectively, ReLU activated;

[0095] Layer 5: Global Max Pooling and MLP (512, 256 channels, ReLU activated);

[0096] Output: 256-dimensional geometric feature vectors, including point cloud distribution entropy, normal vector direction, rate of curvature change, etc.

[0097] Training objective: Defect classification using cross-entropy loss, validation set accuracy ≥ 97%.

[0098] Hyperspectral feature branch (3D convolutional network):

[0099] The hyperspectral cube input to the multimodal data acquisition and preprocessing module has dimensions of 64×64×30 and reflectance normalized to 0-1.

[0100] Network structure:

[0101] Layers 1-3: 3D convolution (3×3×3 kernel, number of channels 32, 64, 128, stride 1, padding=1) followed by BN layer and ReLU activation function;

[0102] Layer 4: 3×3×30 convolution, global average pooling, 256-channel 1×1 convolution;

[0103] Output: 256-dimensional spectral feature vectors, including reflectance of feature bands, spectral angle mapper (SAM), inter-band correlation, etc.

[0104] The training objective is the same as that of the 2D branch.

[0105] The process of generating the hybrid vector unit needs to be specifically explained in this embodiment as follows:

[0106] Feature fusion and partitioning:

[0107] Train the attention weight network with inputs of 2D, 3D, and hyperspectral branch feature vectors, each with 256 dimensions;

[0108] Network structure: 3 fully connected layers (768, 256, 3), Softmax activation, outputting 3 weights w1, w2, w3, with a total sum of 1;

[0109] Training data: branch features and corresponding defect labels of 1000 samples;

[0110] Training objective: To use a cross-entropy loss method that fuses features to make the weights adaptive to the defect type.

[0111] Fusion computing: ,in, It is a 256-dimensional texture feature vector. It is a 256-dimensional geometric eigenvector. It is a 256-dimensional spectral eigenvector;

[0112] Feature partitioning: Divided into 3 sub-areas based on function:

[0113] First 64 dimensions Process and quality correlation characteristics include features strongly correlated with process parameters of the multimodal data acquisition and preprocessing module, screened by Pearson correlation coefficient, |r|≥0.7;

[0114] 128 dimensions in the middle Visual discrimination features include discriminative features related to pure image texture, color, and shape;

[0115] 64-dimensional : Spectral material characteristics, including spectral features related to material composition and oxidation level;

[0116] Hybrid Feature Concatenation: Concatenating 256-dimensional data-driven features With 256-dimensional physical information features 512-dimensional hybrid features were obtained. ;

[0117] By using t-SNE dimensionality reduction visualization, it was verified that the mixed features can be clearly clustered according to defect type in the low-dimensional space, and the clustering purity of the same type of defect is ≥95%.

[0118] It should be further explained that the comprehensive quality index (CQI) is output based on the training of the network model using hybrid vectors. The specific calculation process is as follows:

[0119] Input data: 512-dimensional mixed features Time series of process parameters with multimodal data acquisition and preprocessing module The time series of process parameters is k-dimensional, where k is the number of parameters;

[0120] Network training:

[0121] Network structure: 3-layer fully connected network (512+k, 128, 64, 1), with the output layer having Sigmoid activation and mapped to 0-100;

[0122] Training data: 8000 samples , And the actual quality value of the multimodal data acquisition and preprocessing module;

[0123] Training objectives: The mean squared error (MSE) between the predicted and actual CQI quality values ​​should be ≤5, and the Pearson correlation coefficient should be ≥0.95;

[0124] Training optimizer: Adam optimizer, learning rate 0.001, decay rate 1e-5, batch size=32, epoch=50.

[0125] Output results: Single-value CQI (0-100, higher values ​​indicate better quality); CQI time series, and data from the multimodal data acquisition and preprocessing module. The timestamps are aligned and have a length of 100 dots, reflecting the dynamic changes in quality during the processing.

[0126] Quality Field Energy Coupling Decision Module: Based on the output of the preceding module, it calculates the quality toughness coefficient and the process fluctuation transmission coefficient, and defines the compliance energy field, the consistency energy field, and the risk energy field; based on historical data, it determines the energy transfer coefficient matrix and the field phase difference, couples and calculates the total energy, and generates the standardized comprehensive decision value QCI; based on the QCI and field characteristics, it outputs the quality decision.

[0127] This embodiment requires specific explanation of the process for defining the compliance energy field, consistency energy field, and risk energy field, as follows:

[0128] Definition of mass-energy field:

[0129] Dynamic parameter calculation:

[0130] Mass toughness coefficient Reflecting the quality's ability to withstand process fluctuations: The variance of the process parameter time series is calculated based on the process parameter time series, denoted as... Process and quality correlation features based on process parameter time series variance, actual quality values, and feature extraction and mixed vector generation modules. Calculate the mass toughness parameters: , ;in, for The Pearson correlation coefficient with the actual quality value is taken as the absolute value, and the range is [0, 1].

[0131] Process fluctuation transmission coefficient Quantifying the impact of process fluctuations on quality: Standard deviation of process parameter time series data based on multimodal data acquisition and preprocessing module. CQI standard deviation compared to feature extraction and hybrid vector generation modules as well as right The Granger causality coefficient (0-1) is used to calculate the process fluctuation transmission coefficient, and its specific function is: If ,but ,otherwise , .

[0132] Energy field calculation:

[0133] Compliant Energy Field : Its value range is [0, 2], and it is related to CQI and process stability. The larger the value, the higher the quality compliance.

[0134] Consistent energy field This reflects the consistency of product quality with the average level within the same batch and between different batches; a higher value indicates better consistency. Its function is: ,in, This represents the deviation of the CQI from the average of the top 50 products in the same batch. The deviation of CQI from the mean across 3 batches;

[0135] Risk energy field : The closer the value is to 0, the lower the risk.

[0136] It should be further explained that the specific process of energy coupling and QCI generation is as follows:

[0137] Calculate the energy transfer coefficient matrix (K): Based on training with historical quality incident data from 1000 anomaly records from the multimodal data acquisition and preprocessing module, matrix elements... This indicates the intensity of energy influence between fields:

[0138] ,when When the value is ≤-5, the risk significantly weakens compliance capabilities;

[0139] ,when When the value is ≤0.5, the consistency difference amplifies the risk energy.

[0140] ,when When the value is ≥1.5, high compliance enhances consistency energy;

[0141] the remaining , , The values ​​are 0.3, 0.2, and 0.1 respectively.

[0142] Calculate the phase difference of the field Enter 10 consecutive products , , Sequence, calculate the rate of change of each field intensity. ; The calculation function is: , This reflects the synchronicity of changes in the field. For in-phase enhancement, This is an antiphase cancellation.

[0143] Total Energy and QCI Calculation:

[0144] Total coupling energy: ;

[0145] QCI Standardization: Minimum and worst quality of total historical energy. and maximum optimal quality ; , A higher value indicates better overall quality.

[0146] The specific process of decision generation in this embodiment is as follows:

[0147] Based on QCI and energy field characteristics, it outputs clear quality judgment results to guide production flow.

[0148] Decision-making rules:

[0149] Premium grade: QCI≥85 ≥1.8, ≥1.5, ≥-1.0, output a direct release instruction and mark it as a benchmark product;

[0150] Acceptable grade: 70≤QCI<85 ≥1.2, ≥1.0, If the value is ≥-3.0, a normal release command will be output, and a quality report will be generated.

[0151] Level to be checked: 50≤QCI<70: Exists If this indicates a significant risk due to poor consistency, a pause instruction will be issued for targeted review.

[0152] Rejection level: QCI < 50 Output a scrap command and initiate root cause tracing.

[0153] Secondly: The accompanying drawings of the embodiments disclosed in this invention only involve the structures involved in the embodiments disclosed in this invention. Other structures can refer to the general design. In the absence of conflict, the same embodiment and different embodiments of this invention can be combined with each other.

[0154] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A workshop production quality online inspection system based on machine vision and AI, characterized in that, include: Multimodal data acquisition and preprocessing module: synchronously acquires the physical signals, process parameters, and actual quality values ​​of the target product, and ensures data timestamp consistency through clock synchronization and trigger control; After preprocessing, the collected raw data is used to generate standardized data blocks; Feature extraction and hybrid vector generation module: Based on standardized data blocks, physical information features and data-driven features are extracted, and after attention fusion, they are partitioned and then the physical information features and data-driven features are concatenated to generate hybrid vectors; The network model is trained based on hybrid vectors, and the comprehensive quality index (CQI) and CQI time series are output. The comprehensive quality index includes: Using the hybrid vector generated in the feature extraction stage and the time series of process parameters output by the multimodal data acquisition and preprocessing module as input, a 3-layer fully connected network is trained. The network output layer is activated by Sigmoid to map the result to the range of 0-100. The training process aims to achieve a mean square error between the CQI predicted value and the actual quality detection value of less than or equal to 5, and a Pearson correlation coefficient of greater than or equal to 0.

95. The final output includes a single-value CQI and a CQI time series aligned with the process parameter timestamps. Quality Field Energy Coupling Decision Module: Based on the output of the preceding module, it calculates the quality toughness coefficient and process fluctuation transmission coefficient, and defines the compliance energy field, consistency energy field, and risk energy field; based on historical data, it determines the energy transfer coefficient matrix and field phase difference, couples and calculates the total energy, and generates a standardized comprehensive decision value (QCI); based on the QCI and field characteristics, it outputs the quality decision. The definitions of compliance energy field, consistency energy field, and risk energy field include: First, based on the process parameter variance and actual quality value output by the multimodal data acquisition and preprocessing module, and the process and quality correlation features output by the feature extraction and hybrid vector generation module, the quality toughness coefficient is calculated. Based on the standard deviation of process parameters output by the multimodal data acquisition and preprocessing module, and the standard deviation of CQI output by the feature extraction and mixed vector generation module, as well as the Granger causality coefficient of process parameters on CQI, the process fluctuation transmission coefficient is calculated. ; Combined , And CQI, define a compliance energy field that reflects quality compliance; combined with , CQI intra-batch deviation Z, inter-batch deviation Define a uniform energy field that reflects the consistency of mass; Combination The fatigue risk factor and failure acceleration features output by the feature extraction and hybrid vector generation module are used to define a risk energy field for quantifying quality risk. The comprehensive decision value includes: Based on historical quality incident data, an energy transfer coefficient matrix characterizing the influence intensity between various energy fields is determined. Then, the rate of change of the compliance energy field, consistency energy field, and risk energy field of continuous target products is calculated to obtain the field phase difference reflecting the synchronicity of changes in each field. Subsequently, the compliance energy field, consistency energy field, and risk energy field are coupled with the energy transfer coefficient matrix and field phase difference to obtain the total energy. Finally, the best and worst values ​​of the historical total energy are statistically analyzed, and the current total energy is standardized to generate a comprehensive decision value. The output quality decision includes: Based on the comprehensive decision value QCI and compliance energy field Consistent energy field Risk energy field In the energy transfer coefficient matrix Based on the historical total energy extreme value, quality levels are classified and corresponding decisions are output: If QCI≥85 and ≥1.8、 ≥1.5、 If the value is ≥-1.0, it is judged as a high-quality product, and an immediate release instruction is issued and it is marked as a benchmark product. If 70≤QCI<85 and ≥1.2、 ≥1.0、 If the value is ≥-3.0, it is judged as qualified, and a normal release instruction is output and a quality report is generated. If 50 ≤ QCI < 70 and exists If the case is classified as pending investigation, a pause command will be issued and a targeted review will be triggered. for and The field phase difference; If QCI < 50 and If the condition is determined to be rejected, a scrapping command is output and root cause tracing is initiated. The current total energy, This represents the worst historical total energy value. This represents the optimal value for total energy in history.

2. The online inspection system for workshop production quality based on machine vision and AI according to claim 1, characterized in that: The multimodal data acquisition includes: The system employs a 2D surface image acquisition unit to collect surface texture information and the morphology and location information of planar defects of the target product; a 3D geometric shape acquisition unit to collect three-dimensional geometric shape information and spatial feature information of three-dimensional defects of the target product; a hyperspectral material acquisition unit to collect surface spectral reflectance information and spectral feature information of material composition differences and latent sensory defects of the target product; a temperature field acquisition unit to collect temperature distribution information of the heat-processed area of ​​the target product and temperature feature information of heat input uniformity and thermal defects; a process parameter acquisition unit to collect time series information of process parameters that are strongly correlated with the quality of the target product; and an actual quality value acquisition unit to collect quality inspection value information that reflects the actual performance of the target product.

3. The online inspection system for workshop production quality based on machine vision and AI according to claim 1, characterized in that: The preprocessing includes: The data from each acquisition unit are cleaned separately: 2D images undergo distortion correction, foreground segmentation, and contrast enhancement; 3D data undergoes outlier removal, downsampling, and coordinate alignment; hyperspectral data undergoes reflectance correction, effective band selection, and smoothing; temperature data undergoes ambient temperature elimination and effective region segmentation; then, based on manually labeled training regions, a network is proposed to locate defective regions, guiding the spatial semantic alignment of multimodal data; finally, the aligned data is cropped, sampled to a fixed dimension, and normalized to generate standardized data blocks.

4. The online inspection system for workshop production quality based on machine vision and AI according to claim 1, characterized in that: The physical information features include: Based on the temperature field data in the standardized data block, a two-dimensional Fourier transform is performed on each frame of temperature field data and the temperature gradient, peak temperature, and temperature uniformity index are calculated. A 3D convolutional network containing the regularization term of the heat conduction equation is used to extract 128-dimensional thermophysical features, which include heat conduction efficiency, molten pool stability, and heat input uniformity. Based on 3D point cloud fragments in standardized data blocks, crack length, depth, dimensional deviation, surface roughness, and stress concentration factor are calculated. 128-dimensional geometric physical features are extracted through a fully connected network, which include fatigue risk factors and structural integrity index. Thermophysical features and geometric physical features are concatenated to form a 256-dimensional physical information feature vector. In non-thermal processing scenarios, only geometric physical features are used and zero-padding is applied to make the 256-dimensional vector. At the same time, failure acceleration features reflecting crack propagation rate and structural failure trend are extracted from the physical information feature vector.

5. The online inspection system for workshop production quality based on machine vision and AI according to claim 1, characterized in that: The data-driven features include: Samples are selected from the standardized data blocks output by the multimodal data acquisition and preprocessing module, and the training set and validation set are divided and data augmentation is performed. Through feature extraction networks of corresponding modalities, texture features are extracted from 2D image blocks, geometric features are extracted from 3D point cloud fragments, and spectral features are extracted from hyperspectral cubes, respectively. The attention weight network is trained to weight and fuse the above three types of features to obtain a data-driven feature vector. The data-driven feature vector is then divided into functional partitions: process and quality correlation features that are strongly related to process parameters, visual discrimination features that reflect image discriminability, and spectral material features that reflect material properties, providing data-driven feature input for the generation of hybrid vectors.

Images