A new energy automobile valve body assembly monitoring platform based on machine vision

Abstract

The application discloses a new energy automobile valve body assembly monitoring platform based on machine vision, so as to realize intelligent assembly quality monitoring of the new energy automobile valve body. First, based on the process specification, the assembly stage is divided, a multi-source sensor network is deployed to collect monitoring data with a stage label, and a multi-source data set is generated after preprocessing. Secondly, the target detection network is pre-trained to identify the components, extract spatial and time sequence features, and construct a full-process feature set. Simultaneously, a three-dimensional model is constructed by using point cloud data to measure the contact surface fitting degree, stress field distribution and other derived features to generate an enhanced feature set. Then, a double-layer topological graph of the assembly stage and the features is constructed, and the quality score and the defect probability are analyzed by using a graph neural network. Finally, the substandard valve body is traced back to the abnormal stage, and the defect diagnosis is performed to determine whether it is an accidental assembly anomaly. If it is not an accidental assembly anomaly, control is performed. The quality control requirement of the new energy automobile valve body assembly is met, and the monitoring accuracy and intelligence of the automobile valve body assembly are improved.

Description

Technical Field

[0001] This invention relates to the field of automotive valve body assembly monitoring technology, and in particular to a new energy vehicle valve body assembly monitoring platform based on machine vision. Background Technology

[0002] As the core control unit of the thermal management system, the valve body in new energy vehicles directly affects the safety and energy efficiency of the entire vehicle. These components are typically assembled from multiple nested precision metal parts, involving numerous processes such as sealing ring press-fitting, valve core positioning, and bolt tightening, with dimensional tolerances needing to be controlled at the micrometer level. While mainstream manufacturing has introduced automated production lines, significant technical bottlenecks still exist in the quality monitoring stage.

[0003] Existing assembly monitoring technologies suffer from three major drawbacks: First, traditional machine vision solutions are affected by the reflective properties of metal parts, making it difficult to reliably acquire effective image features under strong lighting conditions, and dynamic occlusion caused by assembly tools can easily lead to the loss of key features. Second, when monitoring multiple processes collaboratively, there is a lack of effective data fusion mechanisms; visual data, mechanical parameters, and 3D geometric information are often analyzed in isolation, failing to capture the multi-physics coupling effects during assembly. Third, most systems can only achieve single-point anomaly detection. Furthermore, with the continuous acceleration of production line cycles, existing algorithms struggle to balance detection accuracy and real-time performance, and the computational latency of high-precision models has become a key obstacle restricting the technology's practical application.

[0004] Therefore, this invention provides a machine vision-based monitoring platform for the assembly of valve bodies in new energy vehicles to overcome the above-mentioned defects, meet the quality control requirements of high-precision assembly of valve bodies in new energy vehicles, and improve the accuracy and intelligence of valve body assembly monitoring. Summary of the Invention

[0005] This invention overcomes the shortcomings of the prior art and provides a machine vision-based monitoring platform for the assembly of valve bodies in new energy vehicles. Its main purpose is to improve the accuracy and intelligence of monitoring the assembly of automotive valve bodies.

[0006] To achieve the above objectives, the first aspect of this invention provides a machine vision-based method for monitoring the assembly of valve bodies in new energy vehicles, comprising:

[0007] The preset assembly process of the automotive valve body is obtained and the assembly stage is divided. The target assembly line is monitored by a set multi-source sensor monitoring network. The corresponding assembly stage labels are associated and pre-processed to generate a multi-source assembly monitoring dataset.

[0008] The multi-source assembly monitoring dataset is input into a pre-trained target detection network for component identification. Based on the component identification results, full-process assembly features are extracted to obtain the full-process assembly feature set.

[0009] Based on the multi-source assembly monitoring dataset, assembly monitoring point cloud data is extracted, and a three-dimensional point cloud model is constructed using the assembly monitoring point cloud data to perform derivative feature calculations on the target automobile valve body assembly process, ultimately obtaining an enhanced full-process assembly feature set.

[0010] Based on the enhanced full-process assembly feature set, a topology graph with assembly stages and assembly features as nodes is constructed. The assembly quality of the target automobile valve body is analyzed through the constructed topology graph to obtain assembly quality analysis information.

[0011] When the target vehicle valve body fails to meet the assembly quality standards of the current batch, the abnormal assembly stage is traced using the assembly quality analysis information, and a defect diagnosis is performed to determine whether it is an accidental assembly abnormality. If it is not an accidental assembly abnormality, control measures are implemented.

[0012] In this solution, the preset assembly process of the automotive valve body is obtained and divided into assembly stages. A multi-source sensor monitoring network is used to monitor the target assembly line, and corresponding assembly stage labels are associated and preprocessed to generate a multi-source assembly monitoring dataset. Specifically, this includes:

[0013] Obtain a preset automotive valve body assembly scheme for the target assembly line, extract the component assembly sequence and corresponding assembly sequence constraints of the target assembly line from the preset automotive valve body assembly scheme, divide the assembly stage through the assembly sequence constraints, and generate several assembly stage labels.

[0014] A multi-source sensor monitoring network is set up on the target assembly line. The multi-source sensor monitoring network is used to monitor the assembly of the target assembly line and obtain multi-source assembly monitoring data. The multi-source assembly data is mapped to a unified spatiotemporal coordinate system using timestamps, and each data unit is labeled with the corresponding assembly stage label.

[0015] After the label association is completed in the assembly stage, data preprocessing is performed. A polarization compensation algorithm is introduced for image sequence preprocessing. High dynamic range images are synthesized by weighted fusion of image groups with different polarization angles to eliminate specular reflection. After elimination, adaptive histogram equalization is performed to enhance image texture features.

[0016] The collected point cloud data is downsampled using voxel grids. The original point cloud is divided into several spatial units according to a preset spatial size. The geometric center point in each spatial unit is extracted and the local density distribution is calculated to filter out outliers.

[0017] Pressure sensor data is imported into a Butterworth low-pass filter for mechanical vibration noise elimination. The DBSCAN clustering algorithm is used to detect and remove transient interference outliers. Missing values ​​are filled in by linear interpolation. After preprocessing, a multi-source assembled monitoring dataset is output.

[0018] In this scheme, the step of inputting the multi-source assembly monitoring dataset into a pre-trained target detection network for component identification, and extracting full-process assembly features based on the component identification results to obtain a full-process assembly feature set, specifically includes:

[0019] A target detection network was built and pre-trained using YOLOv7 as the framework. A multi-source assembly monitoring dataset was obtained, and the multi-source assembly monitoring dataset was input into the pre-trained target detection network for component identification.

[0020] Multi-level convolutional layers are set in the backbone network to extract features at multiple scales. The shallow layers of the backbone network use a dense stacking structure of small-sized convolutional kernels to capture the micro-geometric features in the image to be detected, while the deep layers of the backbone network learn the overall assembly relationship of the valve body through stacking of large-sized convolutional kernels.

[0021] The feature pyramid fuses feature maps of different scales based on a preset cross-level skip connection mechanism. It generates multi-scale fused feature maps by embedding a self-attention weight calculation module for channel attention weighted fusion. The multi-scale fused feature maps are then input into the detection head for component detection and recognition, generating component recognition information.

[0022] Based on the component identification information, the assembly feature extraction is performed throughout the entire process. The component position feature sequence is extracted through the component identification information. The pixel blocks of the tool region between adjacent frames are extracted as analysis units using the assembly tool bounding box coordinates in the component position sequence.

[0023] The preset time sliding window divides the continuous operation process into overlapping time segments. Dense optical flow analysis is performed in each time segment to calculate the displacement vector of each pixel in the assembly tool area, generate the displacement vector field between consecutive frames, and fit the motion trajectory curve by the least squares method to output the motion trajectory key point sequence and timestamp.

[0024] Based on the sequence of key points of the motion trajectory, the ratio of displacement increment to time interval between adjacent key points is calculated to generate an instantaneous velocity sequence. The velocity sequence is then subjected to time difference operation to obtain the acceleration change curve. The turning characteristics of the trajectory turning point are analyzed through the curvature calculation formula, and finally the motion feature vector is obtained.

[0025] Pressure sensing data is obtained through a multi-source assembly monitoring dataset. The pressure change rate curve is extracted based on signal differentiation. Zero-crossing detection is used to capture torque mutation feature points to segment the assembly action stages. The deviation between the pressure change degree and duration and the preset standard is calculated to generate a mechanical feature vector. The motion feature vector is combined to form a full-process assembly feature set.

[0026] In this solution, the step of extracting assembly monitoring point cloud data from the multi-source assembly monitoring dataset, constructing a three-dimensional point cloud model using the assembly monitoring point cloud data, and performing derivative feature calculations on the target automobile valve body assembly process ultimately yields an enhanced full-process assembly feature set, specifically including:

[0027] Acquire multi-source assembly monitoring dataset and component identification information, extract assembly monitoring point cloud data from the multi-source assembly monitoring dataset, and spatially register the point cloud with the position features of each component in the component identification information using the world coordinate system of the robotic arm end effector as a reference, and output a calibration point cloud frame sequence of coordinate system one.

[0028] A three-dimensional point cloud model is constructed based on the calibration point cloud frame sequence. Derived feature calculations are performed on the three-dimensional point cloud model to generate multi-dimensional derived feature information, which includes contact surface feature vector, stress field feature vector, and sealing ring assembly feature vector.

[0029] For the contact surface feature vector, based on the component identification information, the valve core-valve seat assembly area is located in the three-dimensional point cloud model. The contact area point cloud cluster is segmented by the region growing algorithm and the normal vector direction of each point in the point cloud cluster is calculated. The dispersion of the normal vector angle is used as the fit score. The root mean square distance from the contact area point cloud to the reference plane is calculated simultaneously to characterize the parallelism deviation of the assembly surface. Finally, the contact surface feature vector is output.

[0030] For the stress field feature vector, the bolt hole coordinates are locked according to the component identification information and a cylindrical coordinate system is established with the hole center as the origin. Pressure sensing data is extracted from the multi-source assembly monitoring dataset and mapped to the bolt hole area. The pressure distribution field is generated by radial basis function interpolation. The gradient change features and peak position offset of the pressure distribution field are extracted to generate the stress field feature vector.

[0031] For the sealing ring assembly feature vector, the motion features of the assembly tool are extracted through the full-process assembly feature set to generate the motion path of the assembly tool. Point cloud slices are cut according to the preset interval, and the contour of each slice is fitted by the least squares ellipse fitting algorithm to generate the time series curve of the major axis of the ellipse.

[0032] The slope of the rising segment of the ellipse major axis time series curve is extracted as the deformation rate index. Then, the initial slice profile at the beginning of the assembly stage is selected as the benchmark. The point cloud residual between the subsequent slice profile and the initial slice profile is calculated to characterize the compression of the sealing ring. Finally, the sealing ring assembly feature vector is output.

[0033] The contact surface feature vector, stress field feature vector, and sealing ring assembly feature vector are spatiotemporally aligned to generate multidimensional derived feature information. This multidimensional derived feature information is then fused with the full-process assembly feature set as an enhancement feature to generate an enhanced full-process assembly feature set.

[0034] In this solution, a topology graph is constructed based on the enhanced full-process assembly feature set, with assembly stages and assembly features as nodes. The assembly quality of the target automotive valve body is analyzed through the constructed topology graph to obtain assembly quality analysis information. Specifically, this includes:

[0035] Obtain an enhanced full-process assembly feature set, extract assembly stage labels and feature type labels corresponding to each feature vector through the enhanced full-process assembly feature set, and construct an initial topology diagram with assembly stage as parent node and assembly features as child nodes.

[0036] By using the component assembly sequence of the target assembly line and the corresponding assembly sequence constraints, directed connections are made between each parent node in the initial topology diagram. The Pearson correlation coefficient between each child node and its corresponding parent node is calculated and used as the edge weight between the parent node and the child node to generate the final topology diagram.

[0037] An assembly quality analysis model is built based on a graph neural network framework. The node adjacency matrix and edge weight matrix are obtained based on the final topology graph and used as inputs to the assembly quality analysis model. The assembly quality of the target automobile valve body is analyzed through a message passing mechanism.

[0038] Local information transfer is performed at the feature sub-node layer. For each feature sub-node, the state offsets and edge weight parameters of adjacent sub-nodes are aggregated. The process correlation between features is learned through a gating aggregation mechanism to generate a local feature embedding vector.

[0039] The local feature embedding vector is passed to the stage parent node layer, and the process tolerance constraint is passed along the assembly process direction. Based on the actual state deviation fed back by the downstream child nodes, the positive constraint and the negative feedback are weighted and fused using a multi-head attention mechanism to output the state encoding vector representing the health of the stage.

[0040] The local feature embedding vector and the state encoding vector are imported into the multi-task decoding layer. The state encoding vector is mapped to the assembly quality score of the corresponding assembly stage through a fully connected network. Based on the local feature embedding vector, a softmax classifier is used to generate the probability distribution of various assembly defects, and finally the assembly quality analysis information is obtained.

[0041] In this solution, when the target automotive valve body fails to meet the assembly quality standards of the current batch, the assembly quality analysis information is used to trace the abnormal assembly stage and perform defect diagnosis to determine whether it is an accidental assembly abnormality. If it is not an accidental assembly abnormality, control measures are implemented, specifically including:

[0042] Obtain assembly quality analysis information, compare the assembly quality analysis information with the assembly quality standard of the current batch, and if it fails to meet the assembly quality standard of the current batch, calculate the deviation between the assembly quality standard and the current batch, perform defect diagnosis, and determine whether it is an accidental assembly abnormality.

[0043] Based on the assembly analysis information, extract all feature nodes corresponding to the assembly stage where the quality score is lower than the preset score threshold, simultaneously obtain the defect type identifiers that exceed the preset probability threshold in each defect probability distribution, and backtrack to the upstream associated parent node in the topology diagram corresponding to the target automobile valve body to generate defect diagnosis information.

[0044] The defect diagnosis information is used to extract the assembly monitoring feature sequence of the target vehicle valve body in the abnormal assembly stage. The assembly monitoring feature sequence of other vehicle valve bodies in the corresponding assembly stage within a preset time range is obtained with the assembly time of the target vehicle valve body as the center. The time sequence is aligned to generate the abnormal assembly stage monitoring feature sequence.

[0045] The abnormal assembly stage monitoring feature sequence is input into a pre-trained Bayesian inference network to obtain the transient prior distribution and probability likelihood function. The transient posterior distribution is output by fusing the prior distribution and likelihood function through Bayes' theorem.

[0046] Monte Carlo sampling is performed in a Bayesian inference network based on transient posterior distribution to simulate the evolution trajectory of monitoring features during the abnormal assembly stage within a future time window, and generate a sequence of changes in monitoring features at future time points.

[0047] The dynamic time warping distance between the future feature change sequence and the assembly monitoring feature sequence of the target automobile valve body during the abnormal assembly stage is calculated using a dynamic warping algorithm. If the distance is less than a preset distance threshold, it means that the current assembly abnormality time is not an accidental assembly abnormality. Then, the preset control strategy database is used to obtain the control strategy and adjust the control parameters corresponding to the abnormal assembly stage.

[0048] A second aspect of the present invention provides a machine vision-based monitoring platform for the assembly of valve bodies in new energy vehicles. The platform includes a memory, a processor, and a communication interface. The memory contains a machine vision-based method program for monitoring the assembly of valve bodies in new energy vehicles. When executed by the processor, the machine vision-based method program for monitoring the assembly of valve bodies in new energy vehicles performs the following steps:

[0049] The preset assembly process of the automotive valve body is obtained and the assembly stage is divided. The target assembly line is monitored by a set multi-source sensor monitoring network. The corresponding assembly stage labels are associated and pre-processed to generate a multi-source assembly monitoring dataset.

[0050] The multi-source assembly monitoring dataset is input into a pre-trained target detection network for component identification. Based on the component identification results, full-process assembly features are extracted to obtain the full-process assembly feature set.

[0051] Based on the multi-source assembly monitoring dataset, assembly monitoring point cloud data is extracted, and a three-dimensional point cloud model is constructed using the assembly monitoring point cloud data to perform derivative feature calculations on the target automobile valve body assembly process, ultimately obtaining an enhanced full-process assembly feature set.

[0052] Based on the enhanced full-process assembly feature set, a topology graph with assembly stages and assembly features as nodes is constructed. The assembly quality of the target automobile valve body is analyzed through the constructed topology graph to obtain assembly quality analysis information.

[0053] When the target vehicle valve body fails to meet the assembly quality standards of the current batch, the abnormal assembly stage is traced using the assembly quality analysis information, and a defect diagnosis is performed to determine whether it is an accidental assembly abnormality. If it is not an accidental assembly abnormality, control measures are implemented.

[0054] A third aspect of the present invention provides a computer-readable storage medium comprising a machine vision-based method program for monitoring the assembly of valve bodies in new energy vehicles. When the machine vision-based method program for monitoring the assembly of valve bodies in new energy vehicles is executed by a processor, it implements the steps of the machine vision-based method for monitoring the assembly of valve bodies in new energy vehicles as described in any of the preceding claims. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments or examples of the present invention, the accompanying drawings used in the embodiments or examples will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained according to these drawings without creative effort.

[0056] Figure 1The first method flow of a new energy vehicle valve body assembly monitoring method based on machine vision is provided in an embodiment of the present invention;

[0057] Figure 2 The second method flow of a new energy vehicle valve body assembly monitoring method based on machine vision is provided in an embodiment of the present invention;

[0058] Figure 3 A block diagram of a new energy vehicle valve body assembly monitoring platform based on machine vision is provided in one embodiment of the present invention;

[0059] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0060] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0061] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0062] Figure 1 The first method flow of a new energy vehicle valve body assembly monitoring method based on machine vision is provided in an embodiment of the present invention;

[0063] like Figure 1 As shown, the present invention provides a first method flow for a machine vision-based monitoring method for valve body assembly in new energy vehicles, comprising:

[0064] S102: Obtain the preset assembly process of the automotive valve body and divide the assembly stage. Use the set multi-source sensor monitoring network to monitor the assembly of the target assembly line, associate the corresponding assembly stage labels and preprocess them to generate a multi-source assembly monitoring dataset.

[0065] S104, The multi-source assembly monitoring dataset is input into a pre-trained target detection network for component identification, and the full-process assembly features are extracted based on the component identification results to obtain the full-process assembly feature set;

[0066] S106, Extract assembly monitoring point cloud data based on the multi-source assembly monitoring dataset, Construct a three-dimensional point cloud model using the assembly monitoring point cloud data to perform derivative feature calculations on the target automobile valve body assembly process, and finally obtain an enhanced full-process assembly feature set.

[0067] S108. Based on the enhanced full-process assembly feature set, a topology diagram with assembly stages and assembly features as nodes is constructed. The assembly quality of the target automobile valve body is analyzed through the constructed topology diagram to obtain assembly quality analysis information.

[0068] S110, when the target vehicle valve body fails to meet the assembly quality standard of the current batch, the abnormal assembly stage is traced using the assembly quality analysis information, and a defect diagnosis is performed to determine whether it is an accidental assembly abnormality. If it is not an accidental assembly abnormality, control measures are implemented.

[0069] Furthermore, in a preferred embodiment provided by the present invention, the step of obtaining the preset assembly process of the automotive valve body and dividing it into assembly stages, monitoring the target assembly line through a set multi-source sensor monitoring network, associating the corresponding assembly stage labels and performing preprocessing to generate a multi-source assembly monitoring dataset specifically includes:

[0070] Obtain a preset automotive valve body assembly scheme for the target assembly line, extract the component assembly sequence and corresponding assembly sequence constraints of the target assembly line from the preset automotive valve body assembly scheme, divide the assembly stage through the assembly sequence constraints, and generate several assembly stage labels.

[0071] A multi-source sensor monitoring network is set up on the target assembly line. The multi-source sensor monitoring network is used to monitor the assembly of the target assembly line and obtain multi-source assembly monitoring data. The multi-source assembly data is mapped to a unified spatiotemporal coordinate system using timestamps, and each data unit is labeled with the corresponding assembly stage label.

[0072] After the label association is completed in the assembly stage, data preprocessing is performed. A polarization compensation algorithm is introduced for image sequence preprocessing. High dynamic range images are synthesized by weighted fusion of image groups with different polarization angles to eliminate specular reflection. After elimination, adaptive histogram equalization is performed to enhance image texture features.

[0073] The collected point cloud data is downsampled using voxel grids. The original point cloud is divided into several spatial units according to a preset spatial size. The geometric center point in each spatial unit is extracted and the local density distribution is calculated to filter out outliers.

[0074] Pressure sensor data is imported into a Butterworth low-pass filter for mechanical vibration noise elimination. The DBSCAN clustering algorithm is used to detect and remove transient interference outliers. Missing values ​​are filled in by linear interpolation. After preprocessing, a multi-source assembled monitoring dataset is output.

[0075] It should be noted that, based on the pre-defined automotive valve body assembly scheme, the component assembly sequence and its constraints are analyzed. A topological sorting algorithm is used to break down the continuous process flow into mutually exclusive assembly stages such as seal ring pre-assembly and valve core positioning, generating a unique coded label for each stage. A monitoring network consisting of industrial cameras, laser displacement sensors, and six-axis force sensors is deployed at key workstations on the production line. A precise time protocol enables hardware-level synchronization of multi-source data, and timestamps are used to map all sensor data to a unified spatiotemporal coordinate system. Each data unit is dynamically labeled with the corresponding assembly stage label according to the process time period. For the image data stream after stage labeling, a polarization compensation mechanism is used to eliminate interference from metal surface reflections: multi-angle image groups are acquired by rotating polarization filters, and high dynamic range images are synthesized based on a weighted fusion algorithm to suppress overexposure in specular reflection areas. The processed image is input into an adaptive histogram equalization channel, dynamically expanding local contrast within 8×8 pixel blocks to enhance the recognizability of key features such as the rubber texture of the seal ring. Point cloud data is processed synchronously and in parallel, performing voxel downsampling and outlier removal: the original point cloud is divided into grid cells with fixed spatial dimensions, and the geometric center point of each cell is extracted to represent local surface features. Abnormal noise points deviating from the mean range are identified and removed through the statistical distribution of neighborhood point distances. For pressure data, a Butterworth low-pass filter is introduced to filter out mechanical vibration interference, and the DBSCAN clustering algorithm is used to detect transient pressure spikes and delete abnormal sampling points. Missing values ​​are filled by linear interpolation based on adjacent timestamp data. The final output is a preprocessed multi-source assembled monitoring dataset, whose feature dimensions include preprocessed images, downsampled point clouds, and filtered pressure waveforms, and all data cells carry encrypted pointers that can trace back to the original signal.

[0076] Furthermore, in a preferred embodiment provided by the present invention, the step of inputting the multi-source assembly monitoring dataset into a pre-trained target detection network for component identification, and extracting full-process assembly features based on the component identification results to obtain a full-process assembly feature set, specifically includes:

[0077] A target detection network was built and pre-trained using YOLOv7 as the framework. A multi-source assembly monitoring dataset was obtained, and the multi-source assembly monitoring dataset was input into the pre-trained target detection network for component identification.

[0078] Multi-level convolutional layers are set in the backbone network to extract features at multiple scales. The shallow layers of the backbone network use a dense stacking structure of small-sized convolutional kernels to capture the micro-geometric features in the image to be detected, while the deep layers of the backbone network learn the overall assembly relationship of the valve body through stacking of large-sized convolutional kernels.

[0079] The feature pyramid fuses feature maps of different scales based on a preset cross-level skip connection mechanism. It generates multi-scale fused feature maps by embedding a self-attention weight calculation module for channel attention weighted fusion. The multi-scale fused feature maps are then input into the detection head for component detection and recognition, generating component recognition information.

[0080] Based on the component identification information, the assembly feature extraction is performed throughout the entire process. The component position feature sequence is extracted through the component identification information. The pixel blocks of the tool region between adjacent frames are extracted as analysis units using the assembly tool bounding box coordinates in the component position sequence.

[0081] The preset time sliding window divides the continuous operation process into overlapping time segments. Dense optical flow analysis is performed in each time segment to calculate the displacement vector of each pixel in the assembly tool area, generate the displacement vector field between consecutive frames, and fit the motion trajectory curve by the least squares method to output the motion trajectory key point sequence and timestamp.

[0082] Based on the sequence of key points of the motion trajectory, the ratio of displacement increment to time interval between adjacent key points is calculated to generate an instantaneous velocity sequence. The velocity sequence is then subjected to time difference operation to obtain the acceleration change curve. The turning characteristics of the trajectory turning point are analyzed through the curvature calculation formula, and finally the motion feature vector is obtained.

[0083] Pressure sensing data is obtained through a multi-source assembly monitoring dataset. The pressure change rate curve is extracted based on signal differentiation. Zero-crossing detection is used to capture torque mutation feature points to segment the assembly action stages. The deviation between the pressure change degree and duration and the preset standard is calculated to generate a mechanical feature vector. The motion feature vector is combined to form a full-process assembly feature set.

[0084] It should be noted that an object detection network was constructed and pre-trained based on the YOLOv7 framework. Subsequently, temporal image data from a multi-source assembly monitoring dataset was input into this network to perform component recognition tasks. In the backbone network, shallow layers employ a densely stacked small-sized convolutional kernel structure to capture microscopic geometric features such as bolt threads and sealing ring indentations, while deeper layers learn the spatial assembly relationships of valve body components through large-sized convolutional kernel stacking. The feature pyramid integrates multi-scale feature maps through a pre-defined cross-level skip connection mechanism. An embedded self-attention module dynamically allocates feature channel weights, and after weighted fusion, a fused feature map containing multi-dimensional semantic information is generated. This feature map is input into the detection head to output structured recognition information such as component type and rotation bounding box coordinates. Furthermore, based on the assembly tool bounding box coordinate sequence in the component recognition information, pixel blocks in the tool operation area between consecutive frames are extracted as motion analysis units. The operation process is divided into overlapping temporal segments through a pre-defined time sliding window. Dense optical flow analysis is performed within each segment to calculate pixel displacement vectors. After generating a displacement vector field, the least squares method is used to fit the tool motion trajectory curve equation, outputting the timestamp sequence of trajectory turning points. Instantaneous velocity curves are generated by analyzing the displacement increments and time intervals between adjacent points based on trajectory key points. Time difference operations are performed on the velocity sequence to obtain the acceleration variation law. Combined with trajectory curvature features, the abrupt change characteristics of motion direction are quantified, ultimately outputting a motion feature vector representing the operational law. Subsequently, pressure sensor data streams are processed synchronously: first-order differential operations are performed on the original pressure waveform to extract the rate of change curve. A zero-crossing detection algorithm captures torque abrupt change points to segment action stages such as bolt tightening and valve core pressing. Based on the segmentation results, parameters such as peak pressure intensity, rate of rise, and stage duration are calculated. The absolute deviation value is calculated by comparing with preset process standards, generating feature vectors reflecting the assembly mechanical state. The mechanical features and motion feature vectors are aligned and spliced ​​on a unified time axis, fusing them to form a full-process assembly feature set containing kinematic laws and mechanical properties. This provides a purely dynamic behavioral feature base for subsequent quality analysis.

[0085] Furthermore, in a preferred embodiment provided by the present invention, the step of extracting assembly monitoring point cloud data based on the multi-source assembly monitoring dataset, constructing a three-dimensional point cloud model using the assembly monitoring point cloud data to perform derivative feature calculations on the target automobile valve body assembly process, and finally obtaining an enhanced full-process assembly feature set, specifically includes:

[0086] Acquire multi-source assembly monitoring dataset and component identification information, extract assembly monitoring point cloud data from the multi-source assembly monitoring dataset, and spatially register the point cloud with the position features of each component in the component identification information using the world coordinate system of the robotic arm end effector as a reference, and output a calibration point cloud frame sequence of coordinate system one.

[0087] A three-dimensional point cloud model is constructed based on the calibration point cloud frame sequence. Derived feature calculations are performed on the three-dimensional point cloud model to generate multi-dimensional derived feature information, which includes contact surface feature vector, stress field feature vector, and sealing ring assembly feature vector.

[0088] For the contact surface feature vector, based on the component identification information, the valve core-valve seat assembly area is located in the three-dimensional point cloud model. The contact area point cloud cluster is segmented by the region growing algorithm and the normal vector direction of each point in the point cloud cluster is calculated. The dispersion of the normal vector angle is used as the fit score. The root mean square distance from the contact area point cloud to the reference plane is calculated simultaneously to characterize the parallelism deviation of the assembly surface. Finally, the contact surface feature vector is output.

[0089] For the stress field feature vector, the bolt hole coordinates are locked according to the component identification information and a cylindrical coordinate system is established with the hole center as the origin. Pressure sensing data is extracted from the multi-source assembly monitoring dataset and mapped to the bolt hole area. The pressure distribution field is generated by radial basis function interpolation. The gradient change features and peak position offset of the pressure distribution field are extracted to generate the stress field feature vector.

[0090] For the sealing ring assembly feature vector, the motion features of the assembly tool are extracted through the full-process assembly feature set to generate the motion path of the assembly tool. Point cloud slices are cut according to the preset interval, and the contour of each slice is fitted by the least squares ellipse fitting algorithm to generate the time series curve of the major axis of the ellipse.

[0091] The slope of the rising segment of the ellipse major axis time series curve is extracted as the deformation rate index. Then, the initial slice profile at the beginning of the assembly stage is selected as the benchmark. The point cloud residual between the subsequent slice profile and the initial slice profile is calculated to characterize the compression of the sealing ring. Finally, the sealing ring assembly feature vector is output.

[0092] The contact surface feature vector, stress field feature vector, and sealing ring assembly feature vector are spatiotemporally aligned to generate multidimensional derived feature information. This multidimensional derived feature information is then fused with the full-process assembly feature set as an enhancement feature to generate an enhanced full-process assembly feature set.

[0093] It should be noted that, firstly, assembly monitoring point cloud data is extracted from the multi-source assembly monitoring dataset. Using the world coordinate system of the robotic arm's end effector as a reference, an iterative nearest-point algorithm is employed to spatially register the point cloud with the component position features in the identification information. This is achieved by aligning the surface normal distribution of the point cloud with the geometric center of the component bounding box frame by frame, solving for the optimal rigid body transformation matrix (including rotation and translation components), and outputting a standard point cloud frame sequence in coordinate system one. A complete 3D point cloud model is constructed based on the calibrated point cloud sequence, and derived features are calculated using this model. For contact surface feature extraction, the valve core-seat assembly area is located in the model, and a contact area point cloud cluster is generated from the seed point using a region growing algorithm. The normal vector of each point within the point cloud cluster is calculated, and the directional dispersion is statistically analyzed as a fitting score. A reference plane is simultaneously fitted, and the root mean square distance from all points to this plane is calculated, outputting a contact surface feature vector representing the parallelism deviation. For stress field features, a cylindrical coordinate system is established at the center of the bolt hole, and pressure sensor data is mapped to spatial grid points. Radial basis function interpolation is used to reconstruct the 3D pressure distribution field. The stress field gradient variation trend and peak offset are analyzed to generate a stress field feature vector reflecting the assembly stress state. For the sealing ring assembly state, the motion path trajectory of the assembly tool is extracted. Point cloud slices are extracted along the tool's motion direction at preset intervals. Least-squares ellipse fitting is performed on the contour of each slice, and the curve of the ellipse's major axis length changing over time is recorded. The average slope of the rising segment of the curve is calculated as a deformation rate index. Using the initial slice as a benchmark, the point cloud deformation of subsequent slice contours relative to the benchmark is analyzed to quantify the change in compression. A sealing ring assembly feature vector containing deformation rate and compression amount is output. Finally, the three types of derived feature vectors are spatiotemporally aligned: a spatial benchmark is bound through the robotic arm coordinate system, and dynamic feature synchronization is achieved using a timestamp sequence. The fusion results generate multi-dimensional derived feature information, which is then spliced ​​with the full-process assembly feature set at the feature level to form an enhanced full-process assembly feature set, providing multi-dimensional three-dimensional process parameter support for assembly quality analysis.

[0094] Furthermore, in a preferred embodiment provided by the present invention, the step of constructing a topology graph with assembly stages and assembly features as nodes based on the enhanced full-process assembly feature set, and analyzing the assembly quality of the target automobile valve body through the constructed topology graph to obtain assembly quality analysis information, specifically includes:

[0095] Obtain an enhanced full-process assembly feature set, extract assembly stage labels and feature type labels corresponding to each feature vector through the enhanced full-process assembly feature set, and construct an initial topology diagram with assembly stage as parent node and assembly features as child nodes.

[0096] By using the component assembly sequence of the target assembly line and the corresponding assembly sequence constraints, directed connections are made between each parent node in the initial topology diagram. The Pearson correlation coefficient between each child node and its corresponding parent node is calculated and used as the edge weight between the parent node and the child node to generate the final topology diagram.

[0097] An assembly quality analysis model is built based on a graph neural network framework. The node adjacency matrix and edge weight matrix are obtained based on the final topology graph and used as inputs to the assembly quality analysis model. The assembly quality of the target automobile valve body is analyzed through a message passing mechanism.

[0098] Local information transfer is performed at the feature sub-node layer. For each feature sub-node, the state offsets and edge weight parameters of adjacent sub-nodes are aggregated. The process correlation between features is learned through a gating aggregation mechanism to generate a local feature embedding vector.

[0099] The local feature embedding vector is passed to the stage parent node layer, and the process tolerance constraint is passed along the assembly process direction. Based on the actual state deviation fed back by the downstream child nodes, the positive constraint and the negative feedback are weighted and fused using a multi-head attention mechanism to output the state encoding vector representing the health of the stage.

[0100] The local feature embedding vector and the state encoding vector are imported into the multi-task decoding layer. The state encoding vector is mapped to the assembly quality score of the corresponding assembly stage through a fully connected network. Based on the local feature embedding vector, a softmax classifier is used to generate the probability distribution of various assembly defects, and finally the assembly quality analysis information is obtained.

[0101] It should be noted that, firstly, based on the enhanced end-to-end assembly feature set, the mapping relationship between feature vectors and process stages is analyzed, and the assembly stage label and feature type label corresponding to each feature vector are extracted. Using assembly stages as parent nodes and assembly features as child nodes, a tree-like topology framework is constructed in memory: parent nodes store process attributes such as stage start time and number, and child nodes bind feature values ​​to form an initial topology diagram with hierarchical labels. This stage completes the structured reorganization of feature data and establishes a framework for the subordinate relationship between processes and parameters. The connection structure is strengthened based on the assembly sequence constraints of the target production line. The logical dependencies of the processes are analyzed, and directed temporal edges are established between parent nodes: the direction points from the preceding stage to the subsequent stage, and the weight value is the deviation rate between the actual conversion time and the standard working hours. For feature child nodes under the same parent node, the Pearson correlation coefficient between each child node and the parent node is calculated, and this coefficient is used as the edge weight value between parent and child nodes. Simultaneously, the statistical correlation between feature child nodes is detected, and undirected collaborative edges are added when it is less than a threshold. After edge structure expansion, a final topology diagram with complete connection semantics is generated. Subsequently, the final topology graph is input into a pre-trained graph neural network model to analyze the assembly quality of the target automotive valve body. First, local message passing is performed at the feature sub-node layer: for each feature sub-node, the feature offsets and edge weight parameters of its neighboring sub-nodes are aggregated, and the process coupling rules between features are learned through a gated recurrent unit, outputting a feature embedding vector that integrates local correlations. This vector is then passed to the stage parent node layer, transmitting process tolerance threshold constraints along the assembly direction, while simultaneously receiving actual deviation data from downstream nodes. A multi-head attention mechanism is used to weightedly fuse positive constraints and negative deviation signals, generating a state encoding vector representing the stage's health. Next, joint analysis is performed using a multi-task decoding layer: the state encoding vector is mapped to quality scores for each assembly stage via a fully connected network, and the local feature embedding vector is input into a Softmax classifier to output the probability distribution of various defect types. Finally, assembly quality analysis information is formed, including a quality score matrix, defect probability vectors, and anomaly location markers, providing a foundation for subsequent control.

[0102] Figure 2 The second method flow of a new energy vehicle valve body assembly monitoring method based on machine vision is provided in an embodiment of the present invention;

[0103] like Figure 2 As shown, the present invention provides a second method flow for monitoring the assembly of valve bodies in new energy vehicles based on machine vision, including:

[0104] S202, Obtain assembly quality analysis information, compare the assembly quality analysis information with the assembly quality standard of the current batch, if it fails to meet the assembly quality standard of the current batch, calculate the deviation between the assembly quality standard of the current batch, perform defect diagnosis and determine whether it is an accidental assembly abnormality.

[0105] S204. Based on the assembly analysis information, extract all feature nodes corresponding to the assembly stage where the quality score is lower than the preset score threshold, simultaneously obtain the defect type identifiers that exceed the preset probability threshold in each defect probability distribution, and backtrack to the upstream associated parent node in the topology diagram corresponding to the target automobile valve body to generate defect diagnosis information.

[0106] S206, extract the assembly monitoring feature sequence of the target automobile valve body in the abnormal assembly stage through the defect diagnosis information, obtain the assembly monitoring feature sequence of other automobile valve bodies in the corresponding assembly stage within a preset time range with the assembly time of the target automobile valve body as the center, and perform time alignment to generate the abnormal assembly stage monitoring feature sequence.

[0107] S208, The abnormal assembly stage monitoring feature sequence is input into the pre-trained Bayesian inference network to obtain the transient prior distribution and probability likelihood function, and the transient posterior distribution is output by fusing the prior distribution and likelihood function through Bayes' theorem.

[0108] S210 performs Monte Carlo sampling in a Bayesian inference network based on transient posterior distribution to simulate the evolution trajectory of monitoring features during the abnormal assembly stage within a future time window and generate a sequence of changes in monitoring features at future times.

[0109] S212, the dynamic time warping distance between the future feature change sequence and the assembly monitoring feature sequence of the target automobile valve body in the abnormal assembly stage is calculated using a dynamic warping algorithm. If it is less than a preset distance threshold, it means that the current assembly abnormality time is not an accidental assembly abnormality. Then, the preset control strategy database is used to obtain the control strategy and control the control parameters corresponding to the abnormal assembly stage.

[0110] It should be noted that during the assembly of automotive valve bodies, due to equipment use and wear, even if the assembly tool control parameters at each assembly stage are at their optimal levels, control fluctuations may still occur, leading to substandard assembly quality. In such cases, it is necessary to identify whether the assembly anomaly is accidental, thus avoiding subsequent assembly quality and progress issues caused by blindly controlling and adjusting parameters. When the assembly quality analysis determines that the target valve body fails to meet the standards, the assembly stage nodes with quality scores below the threshold and their associated feature vectors are first extracted, and defect type identifiers with excessive defect probabilities are simultaneously screened. Backtracking along directed edges in the pre-constructed topology graph, the upstream process parent node affecting the abnormal indicators is located (e.g., tracing back to the sealing ring pre-assembly stage when the seal failure probability exceeds the standard), generating defect diagnosis information containing abnormal stage identifiers, a list of key features, and defect types. Based on the defect diagnosis information, the temporal feature sequence of the target valve body at the abnormal stage is extracted. Centered on the valve body assembly time, the feature sequences of other valve bodies on the same production line at the same stage within a preset time window are retrieved, and a multi-dimensional feature matrix is ​​generated after aligning the time axis. The feature matrix is ​​input into a pre-trained Bayesian inference network: the hidden layers of the network construct conditional probability relationships through a process knowledge base and calculate the likelihood function based on the input features. By fusing the transient prior distribution generated from historical data with the real-time likelihood function using Bayes' theorem, a posterior probability distribution representing the evolution trend of the anomaly is output. The network performs Markov chain Monte Carlo sampling to simulate the evolution trajectory of feature parameters within a future time window, generating a feature change sequence with probability confidence intervals. A dynamic time warping algorithm is used to quantify the similarity between the predicted sequence and the current anomaly sequence: the minimum cumulative distance is calculated through a curved path; when the distance value is less than a preset threshold, the current anomaly is determined to have continuous characteristics and is not a random fluctuation. For systemic anomalies, the system automatically retrieves the control strategy database to obtain parameter compensation schemes, which are then distributed in real-time to the corresponding workstation controllers via the industrial bus, thereby achieving precise monitoring and intelligent control of automotive valve body assembly anomalies and improving production efficiency.

[0111] Furthermore, the invention provides a machine vision-based method for monitoring the assembly of valve bodies in new energy vehicles, which also includes the following steps:

[0112] After adjusting the assembly stage where abnormalities occur on the target assembly line, the adjustment effect is monitored. By continuously monitoring the assembly quality of automotive valves within the preset assembly quantity, it is determined whether the assembly defects have been eliminated.

[0113] If an assembly abnormality still exists in the corresponding assembly stage, it indicates that there is an abnormality in the assembly equipment in the corresponding assembly stage. A maintenance early warning report is generated to prompt that the corresponding assembly equipment needs to be maintained, and the on-duty information of maintenance personnel is obtained to analyze whether maintenance can be carried out in a timely manner.

[0114] Based on the on-duty information of maintenance personnel, available maintenance personnel are extracted and compared with the maintenance tasks pending in the target production workshop. If there are available maintenance personnel, it means that maintenance can be carried out in a timely manner.

[0115] If there are no available maintenance personnel, the maintenance type and start time of the task to be maintained are obtained. The duration of the same maintenance type is obtained through similarity calculation in the historical maintenance database. The fastest maintenance time is predicted to obtain the waiting time prediction information.

[0116] Based on the assembly stage type where there is an assembly anomaly, determine whether semi-finished product assembly can be carried out. If semi-finished product assembly can be carried out, analyze the semi-finished product production volume by combining the waiting time for maintenance prediction information.

[0117] The assembly of the semi-finished products is divided into Class I assembly and Class II assembly. Class I assembly is where the assembly sequence during the assembly abnormality stage can be adjusted to the end, while Class II assembly is where the assembly sequence during the assembly abnormality stage cannot be adjusted to the end.

[0118] The assembly time of a single automotive valve body under normal assembly is obtained. Based on the semi-finished product assembly type and the waiting time prediction information for maintenance, the number of semi-finished products that can be produced during the waiting time for maintenance is analyzed to obtain semi-finished product production prediction information.

[0119] Obtain the production tasks of each assembly line in the target workshop, calculate the redundancy of the production tasks of each assembly line and generate a sorting table, select several assembly lines for assembling semi-finished products through the sorting table, and use a genetic algorithm to solve the allocation problem based on the semi-finished product production prediction information to generate a semi-finished product allocation scheme.

[0120] It should be noted that after implementing abnormal assembly stage control, the assembly quality data of a preset number of products (e.g., 30 consecutive pieces) is collected in real time to analyze whether the key characteristic indicators of the abnormal stage have returned to the qualified range. If the quality score of this stage is still below the threshold and the defect probability continues to exceed the standard during the monitoring period, it is determined that there is a hardware failure in the assembly equipment, and a maintenance warning report is automatically generated. Simultaneously, the status information of maintenance personnel is acquired, and the list of currently available certified maintenance personnel is retrieved from the scheduling database and matched with the queue of tasks awaiting maintenance in the workshop: if there are available personnel with matching skills, they are marked as ready for immediate maintenance; if there are no available personnel, the historical maintenance database is retrieved according to the maintenance type of the task awaiting maintenance, and the shortest maintenance time for the same type is predicted through similarity calculation. Combined with the task queuing status, the fastest response time is estimated, and the predicted waiting time for maintenance is output. The feasibility of semi-finished product assembly is assessed based on the abnormal stage type: when the abnormal process is a non-critical path that can be independently postponed (e.g., final bolt tightening), it is classified as a Class I assembly, allowing the process to be adjusted to the end of the production line; if the abnormal process involves core assembly dependencies (e.g., valve core positioning), it is classified as a Class II assembly and the order cannot be adjusted. Based on the semi-finished product type and waiting time for maintenance, the theoretical production capacity is calculated: The assembly time of a single normal valve body is obtained, and combined with the maintenance waiting time and production line cycle time, after deducting efficiency losses due to process adjustments, the predicted production quantity of semi-finished products is output. Subsequently, the real-time task load of each assembly line in the workshop is scanned, and the task redundancy of each line (planned capacity - work-in-process quantity) is calculated. A production line capacity ranking table is generated in descending order of redundancy, and the top N high-redundancy production lines are selected as semi-finished product receiving units. Combining the distribution of semi-finished product types and the predicted quantity, a genetic algorithm is used to solve for the optimal allocation scheme: A random allocation population is initialized, and the fitness function is based on the production line utilization rate and process switching cost. Through selection, crossover, and mutation operations, iterative optimization is performed, ultimately outputting the semi-finished product allocation quantity and process adjustment instructions for each production line, forming a dynamic scheduling scheme that minimizes downtime losses, thereby ensuring quality while avoiding production losses.

[0121] Figure 3 A machine vision-based monitoring platform 3 for valve body assembly of new energy vehicles is provided in one embodiment of the present invention. The platform includes: a memory 301, a processor 302, and a communication interface 303. The memory 301 contains a machine vision-based monitoring method program for valve body assembly of new energy vehicles. When the machine vision-based monitoring method program for valve body assembly of new energy vehicles is executed by the processor 302, it performs the following steps:

[0122] The preset assembly process of the automotive valve body is obtained and the assembly stage is divided. The target assembly line is monitored by a set multi-source sensor monitoring network. The corresponding assembly stage labels are associated and pre-processed to generate a multi-source assembly monitoring dataset.

[0123] The multi-source assembly monitoring dataset is input into a pre-trained target detection network for component identification. Based on the component identification results, full-process assembly features are extracted to obtain the full-process assembly feature set.

[0124] Based on the multi-source assembly monitoring dataset, assembly monitoring point cloud data is extracted, and a three-dimensional point cloud model is constructed using the assembly monitoring point cloud data to perform derivative feature calculations on the target automobile valve body assembly process, ultimately obtaining an enhanced full-process assembly feature set.

[0125] Based on the enhanced full-process assembly feature set, a topology graph with assembly stages and assembly features as nodes is constructed. The assembly quality of the target automobile valve body is analyzed through the constructed topology graph to obtain assembly quality analysis information.

[0126] When the target vehicle valve body fails to meet the assembly quality standards of the current batch, the abnormal assembly stage is traced using the assembly quality analysis information, and a defect diagnosis is performed to determine whether it is an accidental assembly abnormality. If it is not an accidental assembly abnormality, control measures are implemented.

[0127] In another aspect, the present invention provides a computer-readable storage medium including a machine vision-based method program for monitoring the assembly of valve bodies in new energy vehicles. When the machine vision-based method program for monitoring the assembly of valve bodies in new energy vehicles is executed by a processor, it implements the steps of the machine vision-based method for monitoring the assembly of valve bodies in new energy vehicles as described in any of the preceding claims.

[0128] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0129] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0130] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0131] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0132] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0133] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A machine vision-based method for monitoring assembly of a valve body of a new energy vehicle, characterized in that, The method comprises the following steps: acquiring a preset assembly process of an automobile valve body and performing assembly stage division, monitoring the target assembly line through a set of multi-source sensor monitoring network, associating the corresponding assembly stage label and preprocessing to generate a multi-source assembly monitoring dataset; inputting the multi-source assembly monitoring dataset into a pre-trained target detection network for component recognition, performing full-process assembly feature extraction according to the component recognition result, and obtaining a full-process assembly feature set; extracting assembly monitoring point cloud data from the multi-source assembly monitoring dataset, constructing a three-dimensional point cloud model using the assembly monitoring point cloud data, and deriving feature calculation for the target automobile valve body assembly process, and finally obtaining an enhanced full-process assembly feature set; constructing a topological structure diagram with assembly stages and assembly features as nodes based on the enhanced full-process assembly feature set, analyzing the assembly quality of the target automobile valve body through the constructed topological structure diagram, and obtaining assembly quality analysis information; when the target automobile valve body fails to meet the assembly quality standard of the current batch, using the assembly quality analysis information to trace the abnormal assembly stage, and performing defect diagnosis to determine whether it is an accidental assembly abnormality, if not, performing control; wherein the extraction of assembly monitoring point cloud data from the multi-source assembly monitoring dataset, the construction of a three-dimensional point cloud model using the assembly monitoring point cloud data, the derivation of feature calculation for the target automobile valve body assembly process, and the final obtaining of an enhanced full-process assembly feature set, specifically include: acquiring a multi-source assembly monitoring dataset and component recognition information, extracting assembly monitoring point cloud data from the multi-source assembly monitoring dataset, taking the world coordinate system of the end effector of the robot as the reference, and performing spatial registration of the point cloud and the position features of each component in the component recognition information through the iterative closest point algorithm, and outputting a calibrated point cloud frame sequence with the same coordinate system; constructing a three-dimensional point cloud model based on the calibrated point cloud frame sequence, performing derivative feature calculation through the three-dimensional point cloud model, and generating multi-dimensional derivative feature information, wherein the multi-dimensional derivative feature information includes contact surface feature vectors, stress field feature vectors, and sealing ring assembly feature vectors; for the contact surface feature vector, based on the component recognition information, the valve core-valve seat assembly area is located in the three-dimensional point cloud model, the contact area point cloud cluster is segmented through the region growing algorithm, and the normal vector direction of each point in the point cloud cluster is calculated, the angle dispersion degree of the normal vector is counted as the fitting degree score, the root mean square distance of the contact area point cloud to the reference plane is calculated synchronously to represent the parallelism deviation of the assembly surface, and finally the contact surface feature vector is outputted; for the stress field feature vector, according to the component recognition information, the bolt hole coordinate is locked and a cylindrical coordinate system is established with the hole center as the origin, the pressure sensing data is extracted from the multi-source assembly monitoring dataset and mapped to the bolt hole area, the radial basis function interpolation is used to generate a pressure distribution field, the gradient change feature and peak position offset of the pressure distribution field are extracted, and the stress field feature vector is generated. For the sealing ring assembly feature vector, the assembly tool motion characteristics are extracted by the full-process assembly feature set to generate the assembly tool motion path, and the point cloud is sliced according to the preset interval, the least square ellipse fitting algorithm is used to fit the profile of each slice to generate the time sequence curve of the ellipse major axis; The slope value of the rising section of the time sequence curve of the ellipse major axis is extracted as the deformation rate index, then the initial slice profile at the initial stage of assembly is selected as the reference, the point cloud residual between the subsequent slice profile and the initial slice profile is calculated to represent the compression amount of the sealing ring, and finally the sealing ring assembly feature vector is output; The contact surface feature vector, the stress field feature vector and the sealing ring assembly feature vector are spatiotemporally aligned to generate multi-dimensional derived feature information, and the multi-dimensional derived feature information is fused as an enhanced feature with the full-process assembly feature set to generate an enhanced full-process assembly feature set. 2.The machine vision-based new energy vehicle valve body assembly monitoring method according to claim 1, wherein The preset assembly process of the automobile valve body is obtained, and the assembly stage is divided, the target assembly line is monitored by the set multi-source sensor monitoring network, the corresponding assembly stage label is associated and preprocessed to generate a multi-source assembly monitoring data set, specifically including: A preset automobile valve body assembly scheme of a target assembly line is obtained, a part assembly sequence and a corresponding assembly sequence constraint of the target assembly line are extracted from the preset automobile valve body assembly scheme, the assembly stage is divided by the assembly sequence constraint, and a plurality of assembly stage labels are generated; A multi-source sensor monitoring network is set on the target assembly line, the multi-source sensor monitoring network is used to monitor the target assembly line to obtain multi-source assembly monitoring data, and the multi-source assembly data is mapped to a unified space-time coordinate system by using a time stamp, and the corresponding assembly stage label is labeled for each data unit; After completing the association of the assembly stage label, the data is preprocessed, a polarized light compensation algorithm is introduced for image sequence preprocessing, high dynamic range images are synthesized by using a weighted fusion method for image groups with different polarization angles to eliminate mirror reflection, and after the elimination is completed, adaptive histogram equalization is performed to enhance the image texture features; The collected point cloud data is subjected to voxel grid downsampling processing, the original point cloud is divided into a plurality of space units according to a preset space size, the geometric center point in each space unit is extracted, and the local density distribution is calculated for outlier removal; The pressure sensing data is imported into a Butterworth low-pass filter to eliminate mechanical vibration noise, a DBSCAN clustering algorithm is used to detect and remove transient interference outliers, missing values are supplemented by linear interpolation, and the multi-source assembly monitoring data set is output after the preprocessing is completed. 3.The machine vision-based new energy vehicle valve body assembly monitoring method according to claim 1, wherein, The multi-source assembly monitoring data set is input into a pre-trained target detection network for part recognition, and the full-process assembly feature set is obtained by full-process assembly feature extraction according to the part recognition result, specifically including: A target detection network is built based on YOLOv7 and pre-trained, a multi-source assembly monitoring data set is obtained, and the multi-source assembly monitoring data set is input into the pre-trained target detection network for part recognition. A plurality of convolutional layers are set in the backbone network to extract multi-scale features, wherein a small-size convolutional kernel dense stacking structure is used in the shallow layer of the backbone network to capture microscopic geometric features in the image to be detected, and a large-size convolutional kernel stacking is used in the deep layer of the backbone network to learn the overall assembly relationship of the valve body; The feature pyramid fuses feature maps of different scales based on a preset cross-level skip connection mechanism, generates a multi-scale fusion feature map through channel attention weight calculation module, and inputs the multi-scale fusion feature map into a detection head for component detection and recognition to generate component recognition information; Based on the component recognition information, full-process assembly feature extraction is performed, the component position feature sequence is extracted through the component recognition information, and the pixel blocks of the tool region between adjacent frames are extracted as analysis units using the assembly tool bounding box coordinates in the component position sequence; A preset time sliding window divides the continuous operation process into overlapping time segments, dense optical flow analysis is performed in each time segment, the displacement vector of each pixel point in the assembly tool region is calculated, the displacement vector field between consecutive frames is generated, and the motion trajectory curve is fitted through the least square method to output the motion trajectory key point sequence and the time stamp; Based on the motion trajectory key point sequence, the displacement increment and time interval ratio between adjacent key points are calculated to generate an instantaneous speed sequence, the speed sequence is subjected to time difference operation to obtain an acceleration change curve, the turning features of the trajectory turning points are analyzed through the curvature calculation formula, and finally the motion feature vector is obtained. Pressure sensing data is obtained through a multi-source assembly monitoring data set, a pressure change rate curve is extracted based on signal differential operation, torque mutation feature points are captured using zero-crossing detection to segment the assembly action stage, and a mechanical feature vector is generated by calculating the deviation between the pressure change degree and duration and the preset standard, and the motion feature vector is combined to form a full-process assembly feature set. 4.The new energy vehicle valve body assembly monitoring method based on machine vision of claim 1, wherein, The enhanced full-process assembly feature set is used to construct a topological structure graph with assembly stages and assembly features as nodes, the assembly quality of the target automobile valve body is analyzed through the constructed topological structure graph, and assembly quality analysis information is obtained, specifically including: An enhanced full-process assembly feature set is obtained, assembly stage labels and feature type labels corresponding to each feature vector are extracted through the enhanced full-process assembly feature set, an initial topological structure graph is constructed by taking assembly stages as parent nodes and assembly features as child nodes; Each parent node in the initial topological structure graph is directionally connected through the component assembly sequence of the target assembly line and the corresponding assembly sequence constraint, the Pearson correlation coefficient between each child node and the corresponding parent node is calculated as the edge weight between the parent node and the child node, and a final topological structure graph is generated; An assembly quality analysis model is built based on a graph neural network framework, a node adjacency matrix and an edge weight matrix are obtained based on the final topological structure graph as inputs of the assembly quality analysis model, and the assembly quality of the target automobile valve body is analyzed through a message passing mechanism. Local information transmission is performed at a feature sub-node layer, state offsets and edge weight parameters of adjacent sub-nodes are aggregated for each feature sub-node, process correlation between features is learned through a gating aggregation mechanism to generate a local feature embedding vector; The local feature embedding vector is transmitted to a stage parent node layer, process tolerance constraints are transmitted in the direction of an assembly process, and a state encoding vector representing a stage health degree is output by using a multi-head attention mechanism to weight and fuse forward constraints and reverse feedback based on actual state deviations fed back by downstream sub-nodes; The local feature embedding vector and the state encoding vector are imported into a multi-task decoding layer, the state encoding vector is mapped to an assembly quality score of the corresponding assembly stage through a fully connected network, and a probability distribution of each type of assembly defect is generated based on the local feature embedding vector using a softmax classifier, so that assembly quality analysis information is finally obtained. 5.The new energy vehicle valve body assembly monitoring method based on machine vision of claim 1, wherein, When the target automobile valve body fails to meet the assembly quality standard of the current batch, the assembly quality analysis information is used to trace the abnormal assembly stage, and defect diagnosis is performed to determine whether it is an accidental assembly abnormality. If it is not an accidental assembly abnormality, control is performed, specifically including: Obtaining assembly quality analysis information, judging the assembly quality analysis information and the assembly quality standard of the current batch, if it fails to meet the assembly quality standard of the current batch, calculating the deviation between the assembly quality standard of the current batch, performing defect diagnosis and determining whether it is an accidental assembly abnormality; According to the assembly quality analysis information, all feature nodes corresponding to the assembly stage with a quality score lower than a preset score threshold are extracted, the defect type identifiers exceeding a preset probability threshold in each defect probability distribution are synchronously obtained, the upstream associated parent nodes are traced back in the topological structure graph corresponding to the target automobile valve body, and defect diagnosis information is generated; Through the defect diagnosis information, the assembly monitoring feature sequence of the target automobile valve body in the abnormal assembly stage is extracted, the assembly monitoring feature sequences of the remaining automobile valve bodies in the corresponding assembly stage within a preset time range are obtained with the assembly time of the target automobile valve body as the center, time sequence alignment is performed to generate an abnormal assembly stage monitoring feature sequence; The abnormal assembly stage monitoring feature sequence is input into a pre-trained Bayesian inference network to obtain a transient prior distribution and a probability likelihood function, and a transient posterior distribution is output by fusing the prior distribution and the likelihood function through Bayesian theorem; Based on the transient posterior distribution, Monte Carlo sampling is performed in the Bayesian inference network to simulate the evolution trajectory of the monitoring feature of the abnormal assembly stage within a future time window, and a future time monitoring feature change sequence is generated; A dynamic time warping distance between the future time monitoring feature change sequence and the assembly monitoring feature sequence of the target automobile valve body in the abnormal assembly stage is calculated using a dynamic warping algorithm. If it is less than a preset distance threshold, it represents that the current assembly abnormal event is not an accidental assembly abnormality, and a control strategy is obtained from a preset control strategy database to control the control parameters corresponding to the abnormal assembly stage.

6. A new energy vehicle valve body assembly monitoring platform based on machine vision, characterized in that, The platform comprises a memory, a processor, and a communication interface, the memory contains a machine vision-based new energy vehicle valve body assembly monitoring method program, and the machine vision-based new energy vehicle valve body assembly monitoring method program is implemented when the processor is executed to realize the following steps: A preset assembly process of an automobile valve body is obtained, and an assembly stage is divided; a target assembly line is monitored through a set of multi-source sensor monitoring network, a corresponding assembly stage label is associated, and preprocessing is performed, and a multi-source assembly monitoring data set is generated; The multi-source assembly monitoring data set is input into a pre-trained target detection network for component identification, full-process assembly feature extraction is performed according to the component identification result, and a full-process assembly feature set is obtained; Assembly monitoring point cloud data is extracted from the multi-source assembly monitoring data set, a three-dimensional point cloud model is constructed using the assembly monitoring point cloud data, derivative feature calculation is performed on the target automobile valve body assembly process, and finally an enhanced full-process assembly feature set is obtained; A topology structure diagram with assembly stages and assembly features as nodes is constructed based on the enhanced full-process assembly feature set, the assembly quality of the target automobile valve body is analyzed through the constructed topology structure diagram, and assembly quality analysis information is obtained; When the target automobile valve body fails to meet the assembly quality standard of the current batch, the assembly quality analysis information is used to trace the abnormal assembly stage, and defect diagnosis is performed to determine whether it is an accidental assembly abnormality, if it is not an accidental assembly abnormality, control is performed; The method for extracting assembly monitoring point cloud data from the multi-source assembly monitoring data set, constructing a three-dimensional point cloud model using the assembly monitoring point cloud data, performing derivative feature calculation on the target automobile valve body assembly process, and finally obtaining an enhanced full-process assembly feature set, specifically includes: Multi-source assembly monitoring data set and component identification information are obtained, assembly monitoring point cloud data is extracted from the multi-source assembly monitoring data set, a world coordinate system of an end effector of a mechanical arm is taken as a reference, point clouds are spatially registered with position features of each component in the component identification information through an iterative closest point algorithm, and a calibrated point cloud frame sequence with a coordinate system is output; A three-dimensional point cloud model is constructed based on the calibrated point cloud frame sequence, derivative feature calculation is performed through the three-dimensional point cloud model, multi-dimensional derivative feature information is generated, and the multi-dimensional derivative feature information includes a contact surface feature vector, a stress field feature vector, and a sealing ring assembly feature vector; For the contact surface feature vector, a valve core-valve seat assembly area is located in the three-dimensional point cloud model based on the component identification information, a contact area point cloud cluster is segmented through a region growing algorithm, a normal vector direction of each point in the point cloud cluster is calculated, a normal vector angle dispersion degree is counted as a fit degree score, a root mean square distance of the contact area point cloud to a reference plane is calculated to represent a parallelism deviation of the assembly surface, and finally the contact surface feature vector is output. For the stress field feature vector, lock the bolt hole coordinate according to the component identification information and establish a cylindrical coordinate system with the hole center as the origin, extract pressure sensing data from the multi-source assembly monitoring data set and map it to the bolt hole area, generate a pressure distribution field using radial basis function interpolation, extract the gradient change characteristics and peak position offset of the pressure distribution field, and generate the stress field feature vector; For the sealing ring assembly feature vector, extract the assembly tool motion characteristics from the full-process assembly feature set to generate the assembly tool motion path, and perform point cloud slicing according to the preset interval. An ellipse fitting algorithm is used to fit the contours of each slice to generate an ellipse major axis time series curve. The slope value of the rising segment of the ellipse major axis time series curve is extracted as the deformation rate indicator. Then, the initial slice profile at the beginning of the assembly is selected as the reference, and the point cloud residual between the subsequent slice profiles and the initial slice profile is calculated to represent the compression amount of the sealing ring. Finally, the sealing ring assembly feature vector is output. The contact surface feature vector, stress field feature vector and sealing ring assembly feature vector are spatiotemporally aligned to generate multi-dimensional derived feature information. The multi-dimensional derived feature information is fused as an enhanced feature with the full-process assembly feature set to generate an enhanced full-process assembly feature set. 7.The new energy vehicle valve body assembly monitoring platform based on machine vision of claim 6, wherein, The preset assembly process of the automobile valve body is obtained and the assembly stages are divided. The target assembly line is monitored by a set of multi-source sensor monitoring network, the corresponding assembly stage labels are associated and preprocessed to generate a multi-source assembly monitoring data set, which specifically includes: The preset automobile valve body assembly scheme of the target assembly line is obtained, the component assembly sequence and the corresponding assembly sequence constraint of the target assembly line are extracted from the preset automobile valve body assembly scheme, the assembly stages are divided by the assembly sequence constraint, and a plurality of assembly stage labels are generated; A multi-source sensor monitoring network is set up on the target assembly line. The multi-source sensor monitoring network is used to monitor the target assembly line to obtain multi-source assembly monitoring data, and the multi-source assembly data is mapped to a unified spatiotemporal coordinate system using a timestamp. Each data unit is labeled with the corresponding assembly stage label; After completing the association of the assembly stage labels, the data is preprocessed. A polarized light compensation algorithm is introduced for image sequence preprocessing. High dynamic range images are synthesized by using weighted fusion of images with different polarization angles to eliminate specular reflection. After elimination, adaptive histogram equalization is performed to enhance the image texture features; The collected point cloud data is processed by voxel grid downsampling. The original point cloud is divided into a plurality of spatial units according to a predetermined spatial size. The geometric center point in each spatial unit is extracted and the local density distribution is calculated for outlier removal. The pressure sensing data is imported into a Butterworth low-pass filter to eliminate mechanical vibration noise. DBSCAN clustering algorithm is used to detect and remove transient interference outliers. Linear interpolation is used to fill in missing values. When the preprocessing is completed, the multi-source assembly monitoring data set is output. 8.The new energy vehicle valve body assembly monitoring platform based on machine vision of claim 6, wherein, The multi-source assembly monitoring data set is input into a pre-trained target detection network for component identification, and full-process assembly feature extraction is performed according to the component identification result to obtain a full-process assembly feature set, specifically including: YOLOv7 is taken as a framework to build a target detection network and pre-training is performed, a multi-source assembly monitoring data set is obtained, and the multi-source assembly monitoring data set is input into the pre-trained target detection network for component identification; A plurality of convolution layers are set in the backbone network to extract multi-scale features, wherein a small-size convolution kernel dense stacking structure is used in the shallow layer of the backbone network to capture microscopic geometric features in the image to be detected, and a large-size convolution kernel stacking is used in the deep layer of the backbone network to learn the overall assembly relationship of the valve body; The feature pyramid fuses feature maps of different scales based on a preset cross-level jump connection mechanism, performs channel attention weight calculation through an embedded self-attention weight calculation module, generates a multi-scale fusion feature map, and inputs the multi-scale fusion feature map into a detection head for component detection and identification to generate component identification information; Full-process assembly feature extraction is performed based on the component identification information, a component position feature sequence is extracted through the component identification information, and pixel blocks of tool regions between adjacent frames are extracted as analysis units using the assembly tool bounding box coordinates in the component position sequence; A preset time sliding window divides a continuous operation process into overlapping time segments, dense optical flow analysis is performed in each time segment, displacement vectors of each pixel point in the assembly tool region are calculated, a displacement vector field between consecutive frames is generated, and a motion trajectory curve is fitted through a least square method to output a motion trajectory key point sequence and a timestamp; An instantaneous speed sequence is generated based on the motion trajectory key point sequence by calculating the displacement increment and the time interval ratio between adjacent key points, a time difference operation is performed on the speed sequence to obtain an acceleration change curve, the turning features of the trajectory turning points are analyzed through a curvature calculation formula, and finally a motion feature vector is obtained; Pressure sensing data is obtained through the multi-source assembly monitoring data set, a pressure change rate curve is extracted based on signal differential operation, torque mutation feature points are captured using zero-crossing detection to segment the assembly action stage, and a deviation between the pressure change degree and duration and a preset standard is calculated to generate a mechanical feature vector, and the motion feature vector is combined to form a full-process assembly feature set.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium includes a new energy automobile valve body assembly monitoring method program based on machine vision, and when the new energy automobile valve body assembly monitoring method program based on machine vision is executed by the processor, the steps of the new energy automobile valve body assembly monitoring method based on machine vision in any one of claims 1 to 5 are realized.

Images