Product monitoring method and system based on machine vision and convolutional neural network

Abstract

The application discloses a product monitoring method and system based on machine vision and a convolutional neural network, and relates to the technical field of intelligent manufacturing.The method comprises the following steps: collecting and preprocessing images of automobile bearings to obtain preprocessed automobile bearing two-dimensional image features and automobile bearing three-dimensional point cloud data; inputting the preprocessed automobile bearing two-dimensional image features and the preprocessed automobile bearing three-dimensional point cloud data into a pre-trained multi-task convolutional neural network, extracting fused features through a shared feature encoder, generating an automobile bearing normal score by a first output head, and generating a defect preliminary judgment and a high-dimensional feature deep feature vector by a second output head; and according to current production line real-time throughput rate and historical quality statistical information.The application forms a dual protection mechanism that takes into account detection efficiency and result reliability, and the output comprehensive report contains specific defect diagnosis and labeled detection confidence states, thereby providing complete decision-making basis for quality control.

Description

Technical Field

[0001] This invention relates to the field of intelligent manufacturing technology, and in particular to product monitoring methods and systems based on machine vision and convolutional neural networks. Background Technology

[0002] In the production of precision automotive bearing components, dimensional accuracy directly affects the reliability and safety of the product. Although traditional contact measurement methods have high accuracy, they have limitations such as low efficiency and inability to perform full inspection. In recent years, machine vision technology with convolutional neural networks as its core has brought innovation to the field. Existing technical solutions use industrial cameras to acquire two-dimensional images of products and train deep convolutional neural networks to achieve automatic classification and recognition of defects. By utilizing the powerful feature extraction capabilities of CNNs, the level of automation and consistency of inspection has been improved, and it has become the mainstream technical direction in this field.

[0003] When dealing with complex industrial scenarios requiring high precision, existing technologies still face challenges. In terms of real-time performance, complex neural network models designed to ensure accuracy have long inference times and are difficult to adapt to the changing throughput rates in continuous production: they can easily cause processing bottlenecks when the production line accelerates and lead to idle computing power when it decelerates. In terms of decision reliability, conventional CNN models lack the ability to quantitatively evaluate their own decision-making confidence. For unknown defects or boundary samples outside the training data, they may still output high-confidence erroneous judgments, which poses a risk in precision manufacturing that emphasizes zero defects. Existing technologies have failed to effectively combine production cycle information with model uncertainty assessment to achieve a dynamic balance between system efficiency and reliability. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a product monitoring method based on machine vision and convolutional neural networks, which solves the problems of insufficient detection efficiency and lack of quantitative assessment of the reliability of unknown defect judgment in existing technologies under varying production line throughput rates.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a product monitoring method based on machine vision and convolutional neural networks, which includes: acquiring and preprocessing images of automobile bearings to obtain preprocessed two-dimensional image features of automobile bearings and three-dimensional point cloud data of automobile bearings. The pre-processed 2D image features of the automobile bearing and the pre-processed 3D point cloud data of the automobile bearing are input into a pre-trained multi-task convolutional neural network. The fused features are extracted through a shared feature encoder. The first output head generates the normal score of the automobile bearing, and the second output head generates the depth feature vector of the preliminary defect judgment and high-dimensional features. Based on the current real-time throughput rate of the production line and historical quality statistics, a first-level decision threshold is generated. The normal score of the automotive bearing is compared with the first-level decision threshold, and the qualified and process end status and the in-depth verification status are output respectively. Based on the deep feature vector, the second output head with Monte Carlo drop-off operation enabled is forward-propagated to obtain the cognitive uncertainty quantification value. Based on the quantification of cognitive uncertainty and real-time constraints, the final uncertainty decision threshold is determined. By combining the preliminary diagnostic opinions of automotive bearings, a comprehensive report on the final monitoring results of automotive bearings is generated.

[0007] As a preferred embodiment of the product monitoring method based on machine vision and convolutional neural networks described in this invention, the following steps are included: Image acquisition and preprocessing of automotive bearings to obtain preprocessed two-dimensional image features and three-dimensional point cloud data of the automotive bearings. An industrial area array camera is used to acquire a two-dimensional grayscale image of the car bearing. Then, Canny edge detection is performed to extract the preprocessed two-dimensional image features of the car bearing. The same car bearing was scanned using a line laser 3D scanner to obtain a 3D point cloud of the car bearing, and a statistical outlier removal operation was performed to obtain a noise-removed 3D point cloud of the car bearing. The noise-removed 3D point cloud of automotive bearings is analyzed, and the surface orientation information of each point in the point cloud is estimated. This is the preprocessed 3D point cloud data of automotive bearings.

[0008] As a preferred embodiment of the product monitoring method based on machine vision and convolutional neural networks described in this invention, the method includes the following steps: inputting pre-processed two-dimensional image features of automobile bearings and pre-processed three-dimensional point cloud data of automobile bearings into a pre-trained multi-task convolutional neural network. A training set is constructed based on the preprocessed 2D image features of automobile bearings and the preprocessed 3D point cloud data of automobile bearings. The multi-task convolutional neural network is trained using the training set to obtain a pre-trained multi-task convolutional neural network. The preprocessed 2D image features of the automobile bearing are input into the 2D convolution branch of a pre-trained multi-task convolutional neural network to generate a 2D feature map. The pre-processed 3D point cloud data of the automobile bearing is input into the 3D convolution branch of a pre-trained multi-task convolutional neural network to generate a 3D feature vector.

[0009] As a preferred embodiment of the product monitoring method based on machine vision and convolutional neural networks described in this invention, the method includes the following steps: extracting fused features through a shared feature encoder, generating a normal score for automobile bearings from a first output head, and generating a deep feature vector for preliminary defect judgment and high-dimensional features from a second output head. The two-dimensional feature map and the three-dimensional feature vector are concatenated in the fusion layer of the shared feature encoder to obtain the fused features; The fused features are input into the first output head of a pre-trained multi-task convolutional neural network. The first output head is configured to perform a simplified binary classification task, extracting a global pattern from the fused features to determine whether the car bearing is qualified overall, and outputting a normal score for the car bearing. The fused features are input into the second output head of a pre-trained multi-task convolutional neural network. The second output head is configured to perform fine-grained multi-task learning, parse the specific semantic information for defect classification and the geometric features for size regression from the fused features, and output the preliminary defect judgment and the deep feature vector of high-dimensional features.

[0010] As a preferred embodiment of the product monitoring method based on machine vision and convolutional neural networks described in this invention, the method for generating a first-level decision threshold based on the current real-time throughput rate of the production line and historical quality statistics includes the following steps: Read the current real-time throughput rate of the production line from the data interface of the production line controller; Query the historical database to obtain historical quality statistics of normal scores for automotive bearings in historical batches that match the current time period; Based on the current real-time throughput rate of the production line and historical quality statistics of the normal score of automotive bearings, the first-level decision threshold is calculated through a predefined decision function.

[0011] As a preferred embodiment of the product monitoring method based on machine vision and convolutional neural networks described in this invention, the following steps are included: comparing the normal score of the automotive bearing with the first-level decision threshold, and outputting the qualified and process-ending status and the deep verification status respectively: The normal score of the automotive bearing is compared numerically with the first-level decision threshold.

[0012] When the normal score of the automotive bearing is higher than the first-level decision threshold, the automotive bearing is judged to be qualified and the monitoring process ends. When the normal score of the automotive bearing is not higher than the first-level decision threshold, the status of the monitoring process is marked as requiring in-depth verification.

[0013] As a preferred embodiment of the product monitoring method based on machine vision and convolutional neural networks described in this invention, the method involves: forward propagating the second output head with Monte Carlo dropout operation enabled based on the deep feature vector to obtain a cognitive uncertainty quantification value, including the following steps: The deep feature vector is input into the second output head with Monte Carlo dropout enabled. The second output head with Monte Carlo dropout enabled is forward propagated to obtain the probability distribution of multiple sets of second output head outputs generated by multiple forward propagations. Statistical calculations were performed on the probability distributions of multiple sets of second output heads to obtain the quantification value of cognitive uncertainty.

[0014] As a preferred embodiment of the product monitoring method based on machine vision and convolutional neural networks described in this invention, the method includes the following steps: determining the final uncertainty decision threshold based on the cognitive uncertainty quantification value and real-time constraints. The remaining allowable decision time is calculated based on the preset total processing time limit and the time already consumed in the current process, serving as a real-time constraint; Based on the quantification of cognitive uncertainty and real-time constraints, the final uncertainty decision threshold is dynamically determined through a preset trade-off strategy.

[0015] As a preferred embodiment of the product monitoring method based on machine vision and convolutional neural networks described in this invention, the method includes the following steps: Generating a comprehensive report on the final monitoring results of the automotive bearings by combining preliminary diagnostic opinions; Compare the quantified value of cognitive uncertainty with the final uncertainty decision threshold; When the quantified value of cognitive uncertainty is lower than the final uncertainty decision threshold, the preliminary diagnostic opinion of the automotive bearing is adopted as the final diagnostic opinion of the automotive bearing. When the cognitive uncertainty quantification value is not lower than the final uncertainty decision threshold, the final diagnosis opinion of the automotive bearing is marked as a high uncertainty state that requires manual review; By integrating the final diagnostic opinion of automotive bearings, the quantified value of cognitive uncertainty, and the specific information in the preliminary diagnostic opinion of automotive bearings, a comprehensive report on the final monitoring results of automotive bearings is generated.

[0016] Secondly, the present invention provides a product monitoring system based on machine vision and convolutional neural networks, including a data processing module for acquiring and preprocessing images of automobile bearings to obtain preprocessed two-dimensional image features and three-dimensional point cloud data of automobile bearings. The perception module inputs the pre-processed 2D image features of the car bearing and the pre-processed 3D point cloud data of the car bearing into a pre-trained multi-task convolutional neural network. The shared feature encoder extracts the fused features. The first output head generates the normal score of the car bearing, and the second output head generates the depth feature vector of the preliminary defect judgment and high-dimensional features. The decision module generates a first-level decision threshold based on the current real-time throughput rate of the production line and historical quality statistics. It compares the normal score of the automotive bearing with the first-level decision threshold and outputs the qualified and process end status and the in-depth verification status respectively. The evaluation module, based on deep feature vectors, performs forward propagation on the second output head with Monte Carlo drop-off enabled to obtain the cognitive uncertainty quantification value; The decision-making module determines the final uncertainty decision threshold based on the quantified value of cognitive uncertainty and real-time constraints. By combining the preliminary diagnostic opinions of the automotive bearings, it generates a comprehensive report on the final monitoring results of the automotive bearings.

[0017] The beneficial effects of this invention are as follows: By employing a multimodal data fusion and multi-task collaborative processing architecture, and using a shared feature encoder to synchronously generate normal scores and deep feature vectors for automotive bearings, a dual improvement in detection efficiency and feature reusability is achieved. A first-level decision threshold is dynamically generated based on the real-time throughput rate of the production line and historical quality statistics, constructing an intelligent diversion mechanism that adapts to production rhythm and resolves the contradiction between fixed computing resources and changing production demands. Based on the deep feature vector, a forward propagation of the Monte Carlo dropout network is performed to obtain a quantified value of cognitive uncertainty, establishing an objective evaluation system for the reliability of model decisions. Furthermore, through a final uncertainty decision threshold linked to real-time constraints, a dual guarantee mechanism that balances detection efficiency and result reliability is formed. The output comprehensive report includes specific defect diagnoses and indicates the detection confidence state, providing a complete decision-making basis for quality control. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of a product monitoring method based on machine vision and convolutional neural networks.

[0020] Figure 2 This is a schematic diagram of a product monitoring system based on machine vision and convolutional neural networks. Detailed Implementation

[0021] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0022] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0023] Secondly, the term "one embodiment" or "example" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the invention. The appearance of an embodiment in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that mutually excludes other embodiments.

[0024] Reference Figures 1-2 This is one embodiment of the present invention, which provides a product monitoring method based on machine vision and convolutional neural networks, including the following steps: S1. Image acquisition and preprocessing of the automobile bearing are performed to obtain the preprocessed two-dimensional image features and three-dimensional point cloud data of the automobile bearing.

[0025] S1.1. Use an industrial area array camera to acquire a two-dimensional grayscale image of the automobile bearing, and then perform Canny edge detection to extract the preprocessed two-dimensional image features of the automobile bearing.

[0026] Furthermore, the 2D grayscale images of automotive bearings acquired by industrial area array cameras under uniform lighting conditions can completely preserve the texture details and macroscopic morphological features of the bearing surface. By performing Canny edge detection on the 2D grayscale images of automotive bearings, and by calculating the image gradient magnitude and direction and applying dual threshold detection and edge connection, the contours of the inner and outer rings of the automotive bearing, the roller edges, and the boundary lines of possible surface cracks, pits, and other defects can be effectively identified. The edge feature-based representation method transforms the high-dimensional 2D grayscale images of automotive bearings into more structured and discriminative preprocessed 2D image features of automotive bearings, which significantly reduces the data dimension of subsequent neural network processing and highlights the geometric and topological information most relevant to bearing size measurement and defect identification.

[0027] Specifically, Canny edge detection is implemented to extract preprocessed 2D image features of automotive bearings, transforming the original pixel information into a higher-level feature representation directly related to the bearing monitoring task. This transformation not only reduces the computational burden but also enhances the neural network's ability to perceive key geometric features and abnormal regions of the bearing by highlighting edge and contour information, avoiding overfitting on irrelevant textures or noise. By implementing Canny edge detection on the 2D grayscale image of the automotive bearing, preprocessed 2D image features of the automotive bearing are extracted.

[0028] S1.2. Use a line laser 3D scanner to scan the same car bearing to obtain the 3D point cloud of the car bearing, and perform a statistical outlier removal operation to obtain a noise-removed 3D point cloud of the car bearing.

[0029] Furthermore, the line laser 3D scanner, through the principle of laser triangulation, accurately records the three-dimensional spatial coordinates of each sampling point on the bearing surface in the 3D point cloud of the automotive bearing. It provides depth and three-dimensional geometric shape information that the two-dimensional grayscale image of the automotive bearing does not have. This is crucial for accurately measuring the dimensions of the bearing, such as the inner diameter, outer diameter, and groove curvature, as well as detecting three-dimensional morphological defects. Due to environmental interference, bearing surface reflection, or dust during the scanning process, the 3D point cloud of the automotive bearing inevitably contains some discrete noise points that deviate from the main point group, i.e., outliers.

[0030] Specifically, a statistical outlier removal operation is performed to extract the statistical distribution of the average distance between each point and its neighbors. Points whose mean distance exceeds several times the standard deviation are identified and removed, effectively filtering out scanning noise and purifying the 3D point cloud of the automotive bearing. The outlier judgment criteria are automatically determined based on the density distribution of the local neighborhood of the point cloud, which can robustly handle possible point cloud density changes in different parts of the bearing. The statistical outlier removal operation yields a noise-removed 3D point cloud of the automotive bearing.

[0031] S1.3 Analyze the noise-removed 3D point cloud of the automotive bearing, estimate the surface orientation information of each point in the point cloud, and preprocess the 3D point cloud data of the automotive bearing.

[0032] Furthermore, while the noise-removed 3D point cloud of the automotive bearing contains accurate 3D coordinates, it lacks descriptions of local surface geometric properties. Estimating the surface orientation information for each point in the point cloud is crucial. Normal vector estimation is typically based on principal component analysis (PCA). By analyzing the eigenvalues ​​and eigenvectors of the local neighborhood covariance matrix formed by each point and its neighbors, the eigenvector corresponding to the smallest eigenvalue approximates the surface normal direction at that point. The estimated surface orientation information enriches the data content of the noise-removed 3D point cloud of the automotive bearing, upgrading it from a simple set of geometric points to a 3D feature set containing descriptions of local surface orientations—that is, preprocessed 3D point cloud data of the automotive bearing. The normal vector information in the preprocessed 3D point cloud data of the automotive bearing has multiple values ​​for subsequent 3D feature extraction.

[0033] Specifically, normal vectors are used to obtain local shape descriptors such as curvature, which helps to identify specific geometric feature areas such as bearing raceways and chamfers. The consistency of normal vectors can be used to detect surface anomalies, such as sudden changes in direction that may indicate scratches or pits. In neural network processing, normal vectors can serve as an additional input channel, providing clues to the orientation of local surfaces and enhancing the network's understanding of 3D geometry. A normal estimation method based on principal component analysis is used to perform normal estimation on noise-removed 3D point clouds of automotive bearings to obtain preprocessed 3D point cloud data of automotive bearings containing point coordinates and normal vectors.

[0034] S2. Input the pre-processed 2D image features of the automobile bearing and the pre-processed 3D point cloud data of the automobile bearing into a pre-trained multi-task convolutional neural network.

[0035] S2.1. A training set is constructed based on the preprocessed 2D image features of the automobile bearing and the preprocessed 3D point cloud data of the automobile bearing. The multi-task convolutional neural network is trained using the training set to obtain a pre-trained multi-task convolutional neural network.

[0036] Furthermore, the training set is constructed by using each pair of strictly aligned preprocessed 2D image features of the automotive bearing and preprocessed 3D point cloud data of the automotive bearing as training samples, along with normal bearing score labels and preliminary diagnostic opinion labels obtained from actual measurements and manual annotations of the bearing. The training process of the multi-task convolutional neural network adopts a supervised joint learning paradigm, where the training objectives include two aspects: first, minimizing the difference between the automotive bearing normal score predicted by the first output head of the multi-task convolutional neural network and the automotive bearing normal score label; and second, minimizing the difference between the automotive bearing preliminary diagnostic opinion predicted by the second output head of the multi-task convolutional neural network and the automotive bearing preliminary diagnostic opinion label. Through backpropagation algorithm and optimizer, the network parameters of the shared feature encoder, the first output head, and the second output head are iteratively updated, so that the total loss function of the network on the training set for these two objectives continuously decreases until convergence. The shared feature encoder is forced to learn a general and powerful feature representation that can simultaneously support fast qualification judgment and fine defect diagnosis.

[0037] Specifically, the first output head focuses on extracting global patterns highly correlated with overall compliance from the fused features, while the second output head focuses on parsing fine-grained patterns related to specific defect types and size deviations. This avoids feature learning redundancy and computational resource waste caused by training independent networks for two tasks separately. It also enables high-quality outputs serving different decision stages to be obtained simultaneously in subsequent inference with a single forward propagation. By training the multi-task convolutional neural network on the training set, a pre-trained multi-task convolutional neural network with optimized network parameters and the ability to simultaneously output the normal score of automobile bearings and the preliminary judgment of defects is finally obtained.

[0038] S2.2 Input the preprocessed two-dimensional image features of the automobile bearing into the two-dimensional convolution branch of the pre-trained multi-task convolutional neural network to generate a two-dimensional feature map.

[0039] Furthermore, the preprocessed 2D image features of the automotive bearing are structured images extracted by Canny edge detection, highlighting the bearing contour and defect boundaries. The 2D convolutional branch of the pre-trained multi-task convolutional neural network is usually composed of multiple stacked convolutional layers, activation function layers, and pooling layers. During training, the layers have learned the weight parameters for extracting multi-level visual features from the bearing edge image. When the preprocessed 2D image features of the automotive bearing are input into this 2D convolutional branch, the shallow convolutional kernels are responsible for detecting basic low-level features such as edge directions and corners. As the multi-task convolutional neural network deepens, subsequent convolutional layers combine low-level features to gradually abstract higher-level semantic features, such as contour fragments of specific shapes, texture patterns, and abnormal line segment connection patterns that may indicate cracks.

[0040] Specifically, the final output of the two-dimensional convolution branch is a high-dimensional feature map, where each spatial location corresponds to a local region of the input image, and each channel encodes a specific high-level visual feature of the region. The transformation from preprocessed two-dimensional image features of the car bearing to a two-dimensional feature map transforms explicit edge geometric information into a more compact and discriminative implicit feature representation. The preprocessed two-dimensional image features of the car bearing are input into the two-dimensional convolution branch of a pre-trained multi-task convolutional neural network, and feature extraction and transformation are performed layer by layer through the learned convolutional filter bank to generate a two-dimensional feature map.

[0041] S2.3 Input the pre-processed 3D point cloud data of the automobile bearing into the 3D convolution branch of the pre-trained multi-task convolutional neural network to generate 3D feature vectors.

[0042] Furthermore, the preprocessed 3D point cloud data of the automotive bearing contains the 3D coordinates of each point and the estimated surface normal vector. It is an irregular spatial data. The 3D convolutional branch of the pre-trained multi-task convolutional neural network adopts a network structure that can directly process point cloud data, such as PointNet or its variants. The branch improves the coordinates and normal features of each point point by point through a shared multilayer perceptron, and then aggregates the global point cloud features through symmetric functions such as max pooling to generate an overall feature representation that does not change the order of the points. During training, the 3D convolutional branch learns how to capture key geometric properties from the bearing's point cloud data, such as the cylindrical characteristics of the cylindrical surface, the groove shape of the raceway, the smooth transition of the chamfer, and local deformations such as possible depressions or protrusions.

[0043] Specifically, the pre-processed 3D point cloud data of the automotive bearing is input into this pre-trained 3D convolutional branch. The network processes the point cloud information layer by layer and finally outputs a compact, fixed-dimensional 3D feature vector. The 3D feature vector comprehensively encodes the global geometric shape and key local shape features of the entire bearing point cloud, providing a direct geometric basis for the dimensional accuracy assessment and 3D morphological defect detection of the bearing. Unlike 2D feature maps, which focus on capturing appearance features from a projection perspective, 3D feature vectors essentially describe the geometric entity of the bearing in 3D space. The two have a natural complementarity. The pre-processed 3D point cloud data of the automotive bearing is input into the 3D convolutional branch of the pre-trained multi-task convolutional neural network, and a 3D feature vector is generated through its point cloud feature extraction and aggregation mechanism.

[0044] S3. Extract fused features through a shared feature encoder. The first output head generates the normal score of the automobile bearing, and the second output head generates the deep feature vector of the preliminary defect judgment and high-dimensional features.

[0045] S3.1. The two-dimensional feature map and the three-dimensional feature vector are concatenated in the fusion layer of the shared feature encoder to obtain the fused features.

[0046] Furthermore, the 2D feature map carries high-level visual semantic information extracted from the preprocessed 2D image features of the automotive bearing, such as specific contour patterns, texture anomalies, and edge features of defects. The 3D feature vector encodes global and local geometric attributes abstracted from the preprocessed 3D point cloud data of the automotive bearing, such as overall shape, surface curvature, and dimensional deviation trends. The fusion layer of the shared feature encoder is responsible for integrating these two heterogeneous but complementary feature representations. A typical fusion method is to perform a concatenation operation in the channel dimension, that is, to concatenate the feature vector of the 2D feature map (flattened or further convolved and compressed) with the 3D feature vector into a longer joint feature vector. The concatenation operation retains all the effective information extracted by the 2D and 3D branches respectively.

[0047] Specifically, for example, a texture pattern indicating a surface scratch in a two-dimensional feature may corroborate a small depth depression or abrupt change in normal direction in the corresponding area of ​​a three-dimensional feature. Cross-modal consistency will enhance the model's confidence in judging real defects. If the two-dimensional feature shows an anomaly while the three-dimensional geometric feature is normal, it may indicate that it is a lighting artifact or surface stain rather than a physical defect. The fused feature achieved through feature stitching constructs a unified multimodal feature space, in which each sample is defined by a common representation of its visual appearance and geometric shape. This avoids the data alignment problems that may be encountered in early fusion and also avoids the loss of low-level interactive information that may be lost in late fusion. The two-dimensional feature map and the three-dimensional feature vector are combined in an intermediate layer after their respective branches have been fully refined. The two-dimensional feature map and the three-dimensional feature vector are stitched together in the channel dimension in the fusion layer of the shared feature encoder to obtain the fused feature.

[0048] S3.2. Input the fused features into the first output head of the pre-trained multi-task convolutional neural network. The first output head is configured to perform a simplified binary classification task, extract the global pattern from the fused features to determine whether the car bearing is qualified overall, and output the normal score of the car bearing.

[0049] Furthermore, a highly abstract binary classification decision is executed: determining whether the current bearing is obviously normal. The training process drives the first output head to learn to filter out interfering information related to minor defects, complex backgrounds, or weak contradictions between modes from the fused features, and instead focus on global, high-confidence patterns that can strongly distinguish between qualified and potential problems. The pattern may be a high degree of consistency between two-dimensional and three-dimensional features in describing the overall integrity, symmetry, and compliance of the bearing's main dimensions. For example, for a qualified bearing, the circular outline delineated by its two-dimensional edge features will highly match the cylindricity information reflected by its three-dimensional features, and there will be no sharp abnormal corners or abrupt curvature. The role of the first output head is to extract a quantitative measure of the degree of harmony and consistency from the fused features and map it to a scalar value between zero and one, namely the normal score of the automotive bearing. The higher the score, the more consistently the multimodal evidence presented by the fused features points to the conclusion that there are no abnormalities.

[0050] Specifically, by inputting the fused features into the first output head, a lightweight branch is used in the early stages of the network to make a quick decision on the vast majority of obviously qualified samples, avoiding these samples from continuing to flow through subsequent more complex computational paths, thereby significantly reducing the average processing time. This is based on the realistic insight that qualified products account for the vast majority in industrial production lines, and the main computational resources are reserved for the few suspicious samples that really need detailed analysis. The fused features are input into the first output head of the pre-trained multi-task convolutional neural network, and through its simplified binary classification decision function, the normal score of the automobile bearing, which represents the confidence level of the overall qualification of the bearing, is output.

[0051] S3.3 Input the fused features into the second output head of the pre-trained multi-task convolutional neural network. The second output head is configured to perform fine-grained multi-task learning, parse the specific semantic information for defect classification and the geometric features for size regression from the fused features, and output the preliminary defect judgment and the deep feature vector of high-dimensional features.

[0052] Furthermore, multiple fully connected layers or more complex subnetworks typically undertake a dual task. On the one hand, they need to classify defects, i.e., identify the specific types of defects present in the bearing, such as inner ring cracks, missing rollers, and out-of-tolerance outer diameter. This is essentially a multi-class classification problem. On the other hand, they also need to perform dimensional regression, i.e., accurately predict the deviation of key dimensions, such as the difference between the actual measured values ​​and standard values ​​of the inner and outer diameters. This is a continuous regression problem. The second output head optimizes these two objectives simultaneously through a multi-task learning framework, forcing the shared underlying fusion features to contain sufficiently rich and refined information to support these two types of discrimination. From the fusion features, the second output head needs to parse out specific semantic information. For example, the feature patterns that identify cracks are usually associated with two-dimensional linear dark pattern features and three-dimensional local narrow depression features; while dimensional deviations are more related to the overall scale offset reflected by three-dimensional features. At the same time, it is also necessary to parse out the geometric features used for accurate regression, which may involve decoding the components of the fusion features that are sensitive to absolute size.

[0053] Specifically, the second output head ultimately outputs two parts: first, a preliminary defect judgment, i.e., the defect category label and its probability distribution or specific numerical value of size deviation; second, a deep feature vector of high-dimensional features. This is a high-dimensional representation of an intermediate layer generated by the second output head during the multi-task judgment process. It contains higher-level and more task-related semantic information than the original fused features. The deep feature vector is the direct input for subsequent cognitive uncertainty quantification, reflecting the internal feature state on which the multi-task convolutional neural network relies when making specific defect judgments. Inputting the fused features into the second output head embodies the idea of ​​functional separation and collaboration. The first output head makes a fast yes / no judgment, while the second output head performs a fine-grained analysis of what is and how much it differs. Both share the same fused feature input, ensuring the consistency of the analysis basis and achieving seamless connection from rapid screening to deep diagnosis and on-demand allocation of computing resources. The fused features are input into the second output head of the pre-trained multi-task convolutional neural network, which, through its fine-grained multi-task learning network, outputs the preliminary defect judgment and the deep feature vector of high-dimensional features.

[0054] S4. Generate the first-level decision threshold based on the current real-time throughput rate of the production line and historical quality statistics.

[0055] S4.1 Read the current real-time throughput rate of the production line from the data interface of the production line controller.

[0056] Furthermore, production line controllers, typically serving as the core unit of the manufacturing execution system, continuously monitor and record the production line's operational status. A key performance indicator is the real-time throughput rate of the production line, which directly reflects the number of automotive bearings passing through the inspection station per unit time—that is, the production line's cycle time. Through standardized industrial communication protocols, such as OPC UA or PROFINET, a connection can be established with the production line controller's data interface to read the real-time throughput rate of the production line.

[0057] Specifically, it realizes the injection of external production status information into the internal quality monitoring decision-making process, dynamically links quality inspection behavior with real-time production rhythm, breaks the limitation of the independence of quality judgment and production scheduling in traditional inspection schemes, and reads the real-time throughput rate of the current production line as the first step in building an adaptive monitoring strategy. The real-time throughput rate of the current production line is read from the data interface of the production line controller.

[0058] S4.2 Query the historical database to obtain historical quality statistics of the normal scores of automobile bearings recorded in historical batches that match the current time period.

[0059] Furthermore, the historical database stores all relevant data on inspected automotive bearings over a past period, especially the normal scores of automotive bearings generated by the first output head of the multi-task convolutional neural network. This data is used to assess the quality stability trend of the current production batch or time period. Historical data comparable to the current moment can be retrieved from the historical database. For example, the normal score sequence of several consecutive automotive bearings produced under the same production shift and equipment parameters can be retrieved to obtain historical quality statistics reflecting the distribution characteristics. One of the most important indicators is the standard deviation of the score, namely the historical quality statistical standard deviation of the normal scores of automotive bearings.

[0060] Specifically, the historical quality statistical standard deviation quantifies the degree of fluctuation in recent production quality. A smaller standard deviation indicates stable product quality with a concentrated distribution of normal scores; a larger standard deviation indicates greater product quality fluctuations with a dispersed distribution of normal scores. The purpose of querying historical databases to obtain historical quality statistics on the normal scores of automotive bearings is to introduce a contextual factor based on historical experience that reflects the inherent fluctuations of the process into decision-making. This embodies the idea of ​​process-based statistical quality control, where decision-making involves not only looking at the score of a single current sample but also obtaining historical quality statistics on the normal scores of automotive bearings.

[0061] S4.3 Based on the real-time throughput rate of the current production line and the historical quality statistics of the normal score of automotive bearings, the first-level decision threshold is calculated through a predefined decision function.

[0062] Furthermore, the first-level decision threshold is compared with the normal score of the automotive bearing to determine whether to perform fast release for the current bearing. The predefined decision function defines how the two dynamic inputs, the current production line real-time throughput rate and the historical quality statistics of the automotive bearing normal score, jointly affect the value of the first-level decision threshold. The decision function achieves adaptive matching between the inspection strictness and production needs and quality status. The decision function is specifically reflected in the following relationship: the first-level decision threshold is equal to the basic decision threshold plus the quotient obtained by multiplying the throughput rate adjustment coefficient by the target production line throughput rate minus the current production line real-time throughput rate and dividing by the target production line throughput rate, and then minus the quality stability adjustment coefficient multiplied by the historical quality statistics standard deviation of the automotive bearing normal score. The basic decision threshold is a benchmark value set according to the long-term quality target.

[0063] Specifically, the difference between the current production line's real-time throughput rate and the target production line's throughput rate affects the first-level decision threshold through a throughput rate adjustment coefficient. When the current production line's real-time throughput rate is lower than the target production line's throughput rate, the production line operates at a slower speed, allowing for relatively ample processing time. In this case, the quotient obtained by subtracting the current production line's real-time throughput rate from the target production line's throughput rate and then dividing by the target production line's throughput rate is positive, leading to an increase in the first-level decision threshold. An increased threshold means that the requirements for the normal score of automotive bearings become more stringent; only bearings with higher scores and greater certainty of qualification will be released quickly. This is beneficial for implementing stricter quality control when time pressure is low. Conversely, when the current production line's real-time throughput rate is higher than the target production line's throughput rate, the production line operates at high speed with significant time pressure. In this case, the quotient obtained by subtracting the current production line's real-time throughput rate from the target production line's throughput rate and then dividing by the target production line's throughput rate is negative, leading to a reduction in the first-level decision threshold. A reduction in the first-level decision threshold means that the standard for rapid release is relaxed, allowing more bearings with slightly lower scores—that is, bearings that are slightly questionable but may still be qualified—to be released quickly. This avoids bottlenecks caused by overly strict initial screening on high-speed production lines, ensuring production line throughput. On the other hand, the historical quality statistical standard deviation of the normal scores of automotive bearings affects the threshold through a quality stability adjustment coefficient.

[0064] It should be noted that the larger the historical quality statistical standard deviation of the normal score of automotive bearings, the more drastic the recent quality fluctuations and the more unstable the process. Subtracting the quality stability adjustment coefficient multiplied by the historical quality statistical standard deviation of the normal score of automotive bearings will lower the first-level decision threshold. When the process itself is unstable, the overall distribution of the normal score of automotive bearings may have shifted or widened. Continuing to maintain a high threshold may lead to a large number of borderline qualified products that could have been released being sent for in-depth verification. Appropriately lowering the threshold is a robust strategy to adapt to process fluctuations. In special periods, it is necessary to adjust the judgment benchmark to prevent systemic decision-making errors caused by rigid standards. The method of encoding production throughput pressure and historical quality fluctuations together into a dynamic threshold deeply integrates the external operating status of the production system with the internal decision-making logic of quality inspection, realizing the transformation of quality monitoring strategy from static preset to dynamic situation awareness. The first-level decision threshold is output by comprehensively calculating the current real-time throughput rate of the production line and the historical quality statistics of the normal score of automotive bearings through a predefined decision function.

[0065] The expression for the first-level decision threshold is: ; in, This is the first-level decision threshold. Basic decision threshold, This is the throughput rate adjustment coefficient. To achieve the target production line throughput rate, This represents the current real-time throughput rate of the production line. This is the quality stability adjustment coefficient. This represents the historical standard deviation of the normal score for automotive bearings.

[0066] S5. Compare the normal score of the automotive bearing with the first-level decision threshold, and output the qualified and end process and in-depth verification status respectively.

[0067] S5.1 Compare the normal score of the automobile bearing with the first-level decision threshold.

[0068] Furthermore, the normal score of the automotive bearing is a scalar value generated by the first output head of the multi-task convolutional neural network. The first-level decision threshold is another scalar value calculated by a predefined decision function based on the current real-time throughput rate of the production line and the historical quality statistics of the normal score of the automotive bearing. The numerical comparison operation is to perform a simple arithmetic judgment to determine whether the value of the normal score of the automotive bearing is strictly greater than the value of the first-level decision threshold. The comparison operation transforms the probabilistic normal score of the automotive bearing into a discrete binary decision signal. Through the comparison operation, it can be determined whether the current qualification confidence of the automotive bearing meets the fast release standard set in the current production situation, thereby deciding whether to directly determine qualification and end the process or enter the deep inspection state.

[0069] S5.2 When the normal score of the automotive bearing is higher than the first-level decision threshold, the automotive bearing is judged to be qualified and the monitoring process ends.

[0070] Furthermore, when the normal score of the automotive bearing is higher than the first-level decision threshold, the automotive bearing is judged to be qualified and the current monitoring process ends. The conditional branch means that after the multimodal features of the current automotive bearing are evaluated by the neural network, the normal score of the automotive bearing is not only high, but also exceeds the dynamic qualification standard determined by the current production situation. The conclusion of being qualified is the final decision on the bearing. Ending the current monitoring process means immediately terminating the step. Based on the uncertainty quantification and final decision of deep feature vectors, in industrial mass production, the vast majority of products are qualified. By setting up a fast decision based on lightweight comparison at the front end, obviously qualified samples can pass through instantly with extremely low computational cost.

[0071] S5.3 When the normal score of the automotive bearing is not higher than the first-level decision threshold, the status of the monitoring process is marked as requiring in-depth verification.

[0072] Furthermore, when the normal score of the automotive bearing is not higher than the first-level decision threshold, the monitoring process is marked as requiring in-depth verification. This triggers another processing path. The fact that the normal score of the automotive bearing is not higher than the first-level decision threshold indicates that the current neural network evaluation score of the bearing has failed to meet the rapid release standard required by the dynamic situation. This could be due to minor anomalies in the bearing itself, poor image quality, or the model's generally conservative scoring under current production fluctuations. Marking the monitoring process as requiring in-depth verification is a state setting operation. It does not output a final conclusion but provides a clear instruction signal for subsequent processes. This signal guides the process to a more complex, resource-intensive, but also more precise analysis stage, ensuring that no sample is overlooked and all suspicious or boundary cases undergo more rigorous review.

[0073] Specifically, the two-tier processing architecture is reflected in its risk management approach: the first-tier dynamic threshold comparison is responsible for efficient and low-risk initial screening; once the initial screening is questionable, it automatically upgrades to the second-tier high-reliability analysis, avoiding inefficiency caused by over-examining all samples and preventing the risk of missed detection that may be caused by a single threshold judgment.

[0074] S6. Based on the deep feature vector, the second output head with Monte Carlo drop-off operation enabled is forward-propagated to obtain the cognitive uncertainty quantification value.

[0075] S6.1 Input the depth feature vector into the second output head with Monte Carlo dropout enabled, perform forward propagation on the second output head with Monte Carlo dropout enabled, and obtain the probability distribution of multiple sets of second output head outputs generated by multiple forward propagations.

[0076] Furthermore, the deep feature vector is a high-dimensional semantic feature representation generated by the second output head of the multi-task convolutional neural network during the initial diagnosis. It carries all the key information for fine-grained defect classification and size regression. The second output head with Monte Carlo dropout operation enabled means that during the inference phase, the fully connected layers in the output head network structure still randomly drop some neuron connections according to the probability set during training, but the network weights remain fixed. When the same deep feature vector is repeatedly input into this network with random dropout function, each forward propagation will generate a slightly different data path due to the random deactivation of neuron connections, resulting in small random fluctuations in the output of each propagation, namely the probability distribution of defect categories and the regression prediction value of key dimensions.

[0077] Specifically, performing this forward propagation multiple times, such as dozens of times, yields multiple probability distributions of the second output head outputs. Monte Carlo dropout, a technique typically used to prevent overfitting during training, is reused purposefully. Randomness is actively preserved and utilized during inference, not to improve generalization, but to transform it into a tool for probing cognitive stability. For a clearly understood sample, regardless of how randomly the internal paths of the multi-task convolutional neural network change, the final judgment should tend to be consistent, with minimal fluctuations in the output probability distribution. Conversely, for a vaguely understood sample, random perturbations in the multi-task convolutional neural network are amplified, leading to differences in the probability distributions of multiple outputs. This volatility of multiple outputs is itself a direct reflection of the multi-task convolutional neural network's grasp of the current input. By enabling the Monte Carlo dropout operation in the second output head for forward propagation, multiple probability distributions of the second output head outputs with inherent randomness are obtained.

[0078] S6.2 Statistical calculations are performed on the probability distributions of multiple sets of second output heads to obtain the cognitive uncertainty quantification value.

[0079] Furthermore, the probability distributions of multiple sets of second output heads constitute a set of multiple random predictions for the same input sample. The goal of cognitive uncertainty quantification is to use a scalar value to summarize the degree of dispersion among these multiple sets of predictions, i.e., the degree of cognitive divergence. Statistical calculations process the outputs of classification and regression tasks separately. For the classification task, i.e., the probability distribution of multiple defect categories, the calculation is performed for each defect category. The variance of the predicted probability in the second propagation, and then for all The variances of each category are summed and averaged. For a regression task, i.e., regression predictions for multiple key dimensions, the following calculation is performed: The variance of each predicted value is determined by introducing a weighting coefficient for the regression task uncertainty component, since the classification uncertainty component and the regression uncertainty component may have different dimensions and importance. The regression variance is weighted, and the weighted regression variance is added to the aforementioned average classification variance to obtain a comprehensive quantitative value of cognitive uncertainty.

[0080] Specifically, it transforms the abstract concept of uncertainty into a precisely measurable, mathematically defined continuous value through Monte Carlo sampling and statistical variance calculation. The cognitive uncertainty quantification value has clear interpretability: the closer the cognitive uncertainty quantification value is to zero, the more consistent the results of multiple random predictions are, and the more confident one is in one's own judgment. The larger the value, the higher the dispersion of multiple prediction results. The sample may belong to an unknown pattern outside the training distribution, a multi-class boundary case, or an ambiguous complex sample. Quantification capability is the cornerstone of this method to achieve reliable decision-making. The statistical variance of the probability distribution and regression value of multiple sets of second output heads is calculated to finally obtain the cognitive uncertainty quantification value.

[0081] The expression for the quantification value of cognitive uncertainty is: ; in, To quantify cognitive uncertainty, For Monte Carlo forward propagation number, For the number of Monte Carlo forward propagations, The total number of defect categories. For the defect category, For the first The next forward propagation predicts the category to which the current sample belongs. The probability, For category The predicted probability is Mean of the next forward propagation The weighting coefficients are for the uncertainty components of the regression task. For the first Regression predictions of critical dimensions during the second forward propagation This is the set of regression predicted values.

[0082] S7. Determine the final uncertainty decision threshold based on the quantified value of cognitive uncertainty and real-time constraints.

[0083] S7.1 Calculate the remaining allowable decision time based on the preset total processing time limit and the time already consumed in the current process, as a real-time constraint.

[0084] Furthermore, the preset total processing time limit is the maximum time threshold allowed from the moment the automotive bearing enters the inspection station to the final report output, set to ensure the continuous and smooth operation of the production line. The time consumed in the current process includes the time consumed in the data acquisition and preprocessing steps, the time consumed in the forward propagation step of the multi-task convolutional neural network, and the time consumed in the first step of routing decision-making. This time is obtained by accumulating the actual execution time. Subtracting the total time consumed in the current process from the preset total processing time limit yields the remaining allowable decision-making time. The macroscopic and fixed production line cycle requirements are transformed into a pressure parameter affecting the current sample decision-making process through real-time time budgeting. The remaining allowable decision-making time is no longer an external abstract constraint, but is internalized as a key input variable in the decision-making logic.

[0085] Specifically, based on the concept of decision-making time resources, and recognizing that assessing the deep uncertainty of complex samples is itself a time-consuming computational activity, the rigor and prudence of decisions are dynamically adjusted according to the remaining time resources. For example, when a sample reaches this step, if there is still ample remaining decision-making time, a more conservative and rigorous judgment can be made; if the remaining decision-making time is very tight, a more decisive and potentially riskier strategy must be adopted to avoid exceeding the time limit. Dynamic management of the time budget enables the entire monitoring process to intelligently allocate the prudence of its decisions without violating the hard real-time requirements, thus obtaining the remaining allowable decision-making time as a real-time constraint.

[0086] The remaining allowable decision time expression is: ; in, For the remaining allowable decision-making time, The preset total processing time limit for a single automotive bearing. This represents the time currently consumed in the data acquisition and preprocessing steps for automotive bearings. This represents the time consumed by the current automotive bearing during the forward propagation step of the multi-task convolutional neural network. This represents the time that the current vehicle bearing has consumed in the first-level routing decision step.

[0087] S7.2 Based on the quantified value of cognitive uncertainty and real-time constraints, the final uncertainty decision threshold is dynamically determined through a preset trade-off strategy.

[0088] Furthermore, the cognitive uncertainty quantification value characterizes the degree of self-doubt of the multi-task convolutional neural network regarding the current preliminary diagnostic opinion of the automotive bearing. The real-time constraint, i.e., the remaining allowable decision time, represents the time resources available to complete subsequent decisions. The pre-defined trade-off strategy defines how these two factors jointly determine the final uncertainty decision threshold. A typical trade-off strategy is that the final uncertainty decision threshold is positively correlated with the real-time constraint and negatively correlated with the requirement of the cognitive uncertainty quantification value itself. The logic is twofold: on the one hand, when faced with a high cognitive uncertainty quantification value, the model should require a lower decision threshold to trigger more cautious processing (such as marking it as high uncertainty), because high uncertainty means unreliable judgment. On the other hand, when the real-time constraint is very tight, i.e., the remaining allowable decision time is very short, the process does not have enough time to tolerate or process the high uncertainty state, forcing the decision to converge faster and increasing the final uncertainty decision threshold, so that only extremely high uncertainty can trigger special processing.

[0089] Specifically, when real-time constraints are relaxed, a wider range of uncertainties can be conditionally tolerated. By setting a lower final uncertainty decision threshold, more samples with uncertainties can be identified and specially labeled or processed, thereby achieving higher final decision reliability. Instead of comparing the cognitive uncertainty quantification value with a fixed threshold, it is compared with a dynamic threshold that reflects the remaining time resources, thus achieving online optimization and balance between decision reliability and decision timeliness. It is recognized that in real-time monitoring, the pursuit of reliability may become unrealistic due to excessive time consumption. The optimal trade-off is made within a limited time. Through a preset trade-off strategy, the cognitive uncertainty quantification value and real-time constraints are comprehensively evaluated to obtain the final uncertainty decision threshold applicable to the current sample and the current moment.

[0090] S8. By combining the preliminary diagnostic opinions of the automotive bearings, generate a comprehensive report on the final monitoring results of the automotive bearings.

[0091] S8.1 Compare the quantified value of cognitive uncertainty with the final uncertainty decision threshold.

[0092] Furthermore, the cognitive uncertainty quantification value, obtained through multiple Monte Carlo forward propagations, quantifies the degree of self-doubt of the second output head of the multi-task convolutional neural network regarding the current judgment on the automobile bearing. The final uncertainty decision threshold, another scalar value dynamically determined based on the cognitive uncertainty quantification value and real-time constraints through a pre-defined trade-off strategy, represents the acceptable upper limit of uncertainty for the multi-task convolutional neural network under the current remaining allowable decision time pressure. The comparison operation involves performing a simple numerical judgment to determine whether the cognitive uncertainty quantification value is less than the final uncertainty decision threshold. This transforms the probabilistic cognitive uncertainty quantification value into a discrete binary decision signal. Through the comparison operation, it determines whether the reliability of the current automobile bearing model judgment meets the acceptable standard under the current time constraints, thereby deciding whether to directly adopt the preliminary diagnostic opinion or mark it as a high-uncertainty state requiring manual review. The cognitive uncertainty quantification value is then compared with the final uncertainty decision threshold. S8.2 When the quantified value of cognitive uncertainty is lower than the final uncertainty decision threshold, the preliminary diagnostic opinion of the automotive bearing shall be adopted as the final diagnostic opinion of the automotive bearing.

[0093] Furthermore, when the cognitive uncertainty quantification value is lower than the final uncertainty decision threshold, the preliminary diagnostic opinion of the automotive bearing is adopted as the final diagnostic opinion of the automotive bearing. If the self-uncertainty of the current sample is lower than the dynamically set acceptable upper limit, the preliminary diagnostic opinion of the automotive bearing is adopted as the final diagnostic opinion of the automotive bearing. The information such as the defect category and size deviation value initially determined by the second output head will be inherited without modification. The trust mechanism is not unconditionally trusted, but trust is only given when the self-evaluation of the multi-task convolutional neural network and external constraints are satisfied, thus avoiding unnecessary conservative processing.

[0094] S8.3 When the quantified value of cognitive uncertainty is not lower than the final uncertainty decision threshold, the final diagnostic opinion of the automotive bearing is marked as a high uncertainty state requiring manual review.

[0095] Furthermore, when the cognitive uncertainty quantification value is not lower than the final uncertainty decision threshold, the final diagnosis opinion of the automotive bearing is marked as a high uncertainty state requiring manual review. This condition triggers another processing logic. The fact that the cognitive uncertainty quantification value is not lower than the final uncertainty decision threshold indicates that the self-uncertainty of the multi-task convolutional neural network has reached or exceeded the tolerance limit in the current situation. Regardless of the specific content of the preliminary diagnosis opinion of the automotive bearing, its credibility is questionable. Marking the final diagnosis opinion of the automotive bearing as a high uncertainty state requiring manual review is a risk control operation. It does not overturn the preliminary diagnosis opinion.

[0096] S8.4 Integrate the final diagnostic opinion of the automotive bearing, the quantified value of cognitive uncertainty, and the specific information in the preliminary diagnostic opinion of the automotive bearing to generate a comprehensive report on the final monitoring results of the automotive bearing.

[0097] Furthermore, all key data items are structured, summarized, and formatted for output. The final diagnostic opinion for the automotive bearing is the core conclusion. The adopted preliminary diagnostic opinion may also be marked as a high-uncertainty state. The cognitive uncertainty quantification value is recorded as a key reliability indicator to provide a quantitative basis for the conclusion. Specific information in the preliminary diagnostic opinion for the automotive bearing, such as detailed defect category probability distribution and precise dimensional deviation values, is retained as reference details to generate a comprehensive report on the final monitoring results of the automotive bearing.

[0098] This embodiment also provides a product monitoring system based on machine vision and convolutional neural networks, including: a data processing module, which performs image acquisition and preprocessing on automobile bearings to obtain preprocessed two-dimensional image features and three-dimensional point cloud data of automobile bearings; The perception module inputs the pre-processed 2D image features of the car bearing and the pre-processed 3D point cloud data of the car bearing into a pre-trained multi-task convolutional neural network. The shared feature encoder extracts the fused features. The first output head generates the normal score of the car bearing, and the second output head generates the depth feature vector of the preliminary defect judgment and high-dimensional features. The decision module generates a first-level decision threshold based on the current real-time throughput rate of the production line and historical quality statistics. It compares the normal score of the automotive bearing with the first-level decision threshold and outputs the qualified and process end status and the in-depth verification status respectively. The evaluation module, based on deep feature vectors, performs forward propagation on the second output head with Monte Carlo drop-off enabled to obtain the cognitive uncertainty quantification value; The decision-making module determines the final uncertainty decision threshold based on the quantified value of cognitive uncertainty and real-time constraints. By combining the preliminary diagnostic opinions of the automotive bearings, it generates a comprehensive report on the final monitoring results of the automotive bearings.

[0099] This embodiment also provides a computer device applicable to product monitoring methods based on machine vision and convolutional neural networks, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the product monitoring method based on machine vision and convolutional neural networks as proposed in the above embodiment.

[0100] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0101] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the product monitoring method based on machine vision and convolutional neural networks as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0102] In summary, this invention, through a multimodal data fusion and multi-task collaborative processing architecture, employs a shared feature encoder to synchronously generate normal scores and deep feature vectors for automotive bearings, achieving a dual improvement in detection efficiency and feature reusability. It dynamically generates a first-level decision threshold based on the real-time throughput rate of the production line and historical quality statistics, constructing an intelligent diversion mechanism that adapts to production cycle time, thus resolving the contradiction between fixed computing resources and changing production demands. Based on the deep feature vector, it performs forward propagation on the Monte Carlo dropout network to obtain a quantified value of cognitive uncertainty, establishing an objective evaluation system for the reliability of model decisions. Furthermore, through a final uncertainty decision threshold linked to real-time constraints, it forms a dual guarantee mechanism that balances detection efficiency and result reliability. The output comprehensive report includes specific defect diagnoses and indicates the detection confidence level, providing a complete decision-making basis for quality control.

[0103] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A product monitoring method based on machine vision and convolutional neural networks, characterized in that: This includes acquiring and preprocessing images of automotive bearings to obtain preprocessed two-dimensional image features and three-dimensional point cloud data of automotive bearings. The pre-processed 2D image features of the automobile bearing and the pre-processed 3D point cloud data of the automobile bearing are input into a pre-trained multi-task convolutional neural network. The fused features are extracted through a shared feature encoder. The first output head generates the normal score of the automobile bearing, and the second output head generates the depth feature vector of the preliminary defect judgment and high-dimensional features. Based on the current real-time throughput rate of the production line and historical quality statistics, a first-level decision threshold is generated. The normal score of the automotive bearing is compared with the first-level decision threshold, and the qualified and process end status and the in-depth verification status are output respectively. Based on the deep feature vector, the second output head with Monte Carlo drop-off operation enabled is forward-propagated to obtain the cognitive uncertainty quantification value. Based on the quantification of cognitive uncertainty and real-time constraints, the final uncertainty decision threshold is determined. By combining the preliminary diagnostic opinions of the automotive bearings, a comprehensive report on the final monitoring results of the automotive bearings is generated. Based on the current real-time throughput rate of the production line and historical quality statistics, a first-level decision threshold is generated, including the following steps: Read the current real-time throughput rate of the production line from the data interface of the production line controller; Query the historical database to obtain historical quality statistics of normal scores for automotive bearings in historical batches that match the current time period; Based on the current real-time throughput rate of the production line and historical quality statistics of the normal score of automotive bearings, the first-level decision threshold is calculated through a predefined decision function. The expression for the first-level decision threshold is: ； in, This is the first-level decision threshold. Basic decision threshold, This is the throughput rate adjustment coefficient. To achieve the target production line throughput rate, This represents the current real-time throughput rate of the production line. This is the quality stability adjustment coefficient. The historical standard deviation of the normal score for automotive bearings; Based on the deep feature vector, the second output head with Monte Carlo dropout operation enabled is forward-propagated to obtain the cognitive uncertainty quantification value, including the following steps: The deep feature vector is input into the second output head with Monte Carlo dropout enabled. The second output head with Monte Carlo dropout enabled is forward propagated to obtain the probability distribution of multiple sets of second output head outputs generated by multiple forward propagations. Statistical calculations were performed on the probability distributions of multiple sets of second output heads to obtain the quantification value of cognitive uncertainty. The expression for the quantification of cognitive uncertainty is: ； in, To quantify cognitive uncertainty, For Monte Carlo forward propagation number, For the number of Monte Carlo forward propagations, The total number of defect categories. For the defect category, For the first The next forward propagation predicts the category to which the current sample belongs. The probability, For category The predicted probability is Mean of the next forward propagation The weighting coefficients are for the uncertainty components of the regression task. For the first Regression predictions of critical dimensions during the second forward propagation This is the set of regression predicted values.

2. The product monitoring method based on machine vision and convolutional neural networks as described in claim 1, characterized in that: Image acquisition and preprocessing of automotive bearings are performed to obtain preprocessed 2D image features and 3D point cloud data of automotive bearings, including the following steps: An industrial area array camera is used to acquire a two-dimensional grayscale image of the car bearing. Then, Canny edge detection is performed to extract the preprocessed two-dimensional image features of the car bearing. The same car bearing was scanned using a line laser 3D scanner to obtain a 3D point cloud of the car bearing, and a statistical outlier removal operation was performed to obtain a noise-removed 3D point cloud of the car bearing. The noise-removed 3D point cloud of automotive bearings is analyzed, and the surface orientation information of each point in the point cloud is estimated. This is the preprocessed 3D point cloud data of automotive bearings.

3. The product monitoring method based on machine vision and convolutional neural networks as described in claim 2, characterized in that: The preprocessed 2D image features of the automobile bearing and the preprocessed 3D point cloud data of the automobile bearing are input into a pre-trained multi-task convolutional neural network, including the following steps: A training set is constructed based on the preprocessed 2D image features of automobile bearings and the preprocessed 3D point cloud data of automobile bearings. The multi-task convolutional neural network is trained using the training set to obtain a pre-trained multi-task convolutional neural network. The preprocessed 2D image features of the automobile bearing are input into the 2D convolution branch of a pre-trained multi-task convolutional neural network to generate a 2D feature map. The pre-processed 3D point cloud data of the automobile bearing is input into the 3D convolution branch of a pre-trained multi-task convolutional neural network to generate a 3D feature vector.

4. The product monitoring method based on machine vision and convolutional neural networks as described in claim 3, characterized in that: The fusion features are extracted through a shared feature encoder. The first output head generates the normal score of the automotive bearing, and the second output head generates a deep feature vector with preliminary defect judgment and high-dimensional features. The process includes the following steps: The two-dimensional feature map and the three-dimensional feature vector are concatenated in the fusion layer of the shared feature encoder to obtain the fused features; The fused features are input into the first output head of a pre-trained multi-task convolutional neural network. The first output head is configured to perform a simplified binary classification task, extracting a global pattern from the fused features to determine whether the car bearing is qualified overall, and outputting a normal score for the car bearing. The fused features are input into the second output head of a pre-trained multi-task convolutional neural network. The second output head is configured to perform fine-grained multi-task learning, parse the specific semantic information for defect classification and the geometric features for size regression from the fused features, and output the preliminary defect judgment and the deep feature vector of high-dimensional features.

5. The product monitoring method based on machine vision and convolutional neural networks as described in claim 4, characterized in that: The normal score of the automotive bearing is compared with the first-level decision threshold, and the "qualified and process ended" and "deep inspection" statuses are output respectively, including the following steps: The normal score of the automotive bearing is numerically compared with the first-level decision threshold; When the normal score of the automotive bearing is higher than the first-level decision threshold, the automotive bearing is judged to be qualified and the monitoring process ends. When the normal score of the automotive bearing is not higher than the first-level decision threshold, the status of the monitoring process is marked as requiring in-depth verification.

6. The product monitoring method based on machine vision and convolutional neural networks as described in claim 5, characterized in that: Based on the quantification of cognitive uncertainty and real-time constraints, the final uncertainty decision threshold is determined, including the following steps: The remaining allowable decision time is calculated based on the preset total processing time limit and the time already consumed in the current process, serving as a real-time constraint; Based on the quantification of cognitive uncertainty and real-time constraints, the final uncertainty decision threshold is dynamically determined through a preset trade-off strategy.

7. The product monitoring method based on machine vision and convolutional neural networks as described in claim 6, characterized in that: By combining the preliminary diagnostic opinions of the automotive bearings, a comprehensive report on the final monitoring results of the automotive bearings is generated, including the following steps: Compare the quantified value of cognitive uncertainty with the final uncertainty decision threshold; When the quantified value of cognitive uncertainty is lower than the final uncertainty decision threshold, the preliminary diagnostic opinion of the automotive bearing is adopted as the final diagnostic opinion of the automotive bearing. When the cognitive uncertainty quantification value is not lower than the final uncertainty decision threshold, the final diagnosis opinion of the automotive bearing is marked as a high uncertainty state that requires manual review; By integrating the final diagnostic opinion of automotive bearings, the quantified value of cognitive uncertainty, and the specific information in the preliminary diagnostic opinion of automotive bearings, a comprehensive report on the final monitoring results of automotive bearings is generated.

8. A product monitoring system based on machine vision and convolutional neural networks, wherein the product monitoring method based on machine vision and convolutional neural networks according to any one of claims 1 to 7 is used to implement the system, characterized in that: This includes a data processing module that performs image acquisition and preprocessing on the automotive bearing to obtain preprocessed two-dimensional image features and three-dimensional point cloud data of the automotive bearing. The perception module inputs the pre-processed 2D image features of the car bearing and the pre-processed 3D point cloud data of the car bearing into a pre-trained multi-task convolutional neural network. The shared feature encoder extracts the fused features. The first output head generates the normal score of the car bearing, and the second output head generates the depth feature vector of the preliminary defect judgment and high-dimensional features. The decision module generates a first-level decision threshold based on the current real-time throughput rate of the production line and historical quality statistics. It compares the normal score of the automotive bearing with the first-level decision threshold and outputs the qualified and process end status and the in-depth verification status respectively. The evaluation module, based on deep feature vectors, performs forward propagation on the second output head with Monte Carlo drop-off enabled to obtain the cognitive uncertainty quantification value; The decision-making module determines the final uncertainty decision threshold based on the quantified value of cognitive uncertainty and real-time constraints. By combining the preliminary diagnostic opinions of the automotive bearings, it generates a comprehensive report on the final monitoring results of the automotive bearings.

Images