Assembly state intelligent monitoring method and system based on AI

Abstract

The invention provides an AI-based assembly state intelligent monitoring method and system, and relates to the technical field of artificial intelligence, and the method comprises the steps: collecting a multi-angle real-time assembly process image of a target product, constructing an assembly digital twin model, inputting the image into a hybrid neural network in combination with an assembly knowledge graph, analyzing the assembly node coordinate and torque data deviation, and obtaining an assembly state of the target product. And obtaining an optimal assembly path and torque control parameters, and finally generating an assembly quality evaluation matrix. Through fault feature extraction and mode recognition, potential faults are predicted, and a dynamic risk coefficient is calculated. And when the fault occurrence probability exceeds a preset threshold value, the position and torque compensation parameters are calculated, an execution instruction sequence is generated to control the assembly execution mechanism to perform adjustment, closed-loop optimization control in the assembly process is achieved, the assembly precision and efficiency are improved, and the fault rate is reduced.

Description

Technical Field

[0001] The present invention relates to artificial intelligence technology, and in particular to an intelligent monitoring method and system for assembly status based on AI. Background Art

[0002] With the rapid development of intelligent manufacturing, the intelligent monitoring and control of assembly quality have become an important development direction in the manufacturing industry. Traditional assembly quality monitoring mainly relies on manual experience judgment and offline detection methods, which have problems such as poor real-time performance, low accuracy, and weak adaptability, and are difficult to meet the monitoring requirements of high-precision and high-efficiency assembly processes in modern manufacturing.

[0003] Currently, although some enterprises have begun to use technical means such as visual inspection and sensor monitoring for assembly quality control, these methods are often single-dimensional detections, lacking the ability to fuse and analyze multi-source data, and unable to achieve full-range monitoring and intelligent control of the assembly process. At the same time, the response mechanism of existing assembly monitoring systems to abnormal states is relatively lagging, and it is difficult to detect assembly deviations in a timely manner and make dynamic adjustments.

[0004] During the assembly process, due to the complex mating relationships between mechanical parts, even minor changes in assembly parameters may lead to product quality problems. Existing technologies are difficult to establish a digital model of the assembly process, unable to achieve real-time monitoring and intelligent analysis of the assembly status, and also lacking the ability of fault prediction and optimization control based on historical data. Therefore, there is an urgent need for an intelligent monitoring method for assembly status based on artificial intelligence to achieve real-time monitoring, fault warning, and closed-loop optimization of the assembly process. Summary of the Invention

[0005] Embodiments of the present invention provide an intelligent monitoring method and system for assembly status based on AI, which can solve the problems in the prior art.

[0006] In the first aspect of the embodiments of the present invention, There is provided an intelligent monitoring method for assembly status based on AI, including: Obtaining the assembly drawing and assembly process parameters of the target product, extracting the assembly reference feature points and assembly accuracy thresholds based on the assembly drawing, extracting the assembly process information and torque control thresholds based on the assembly process parameters, constructing an assembly knowledge graph with the assembly reference feature points, assembly accuracy thresholds, assembly process information, and torque control thresholds, and at the same time collecting multi-angle real-time assembly process images of the target product through multiple cameras on the production line, registering and mapping the multi-angle real-time assembly process images with the assembly reference feature points, extracting the assembly node coordinates and assembly torque data in the real-time assembly state, and generating an assembly digital twin model; Input the assembled digital twin model and the assembled knowledge graph into a hybrid neural network. The hybrid neural network constructs a position constraint matrix based on the assembly reference feature points, constructs a torque constraint matrix based on the torque control threshold, performs deviation analysis on the assembly node coordinates and the assembly accuracy threshold based on the position constraint matrix to obtain the optimal assembly path, performs deviation analysis on the assembly torque data and the torque control threshold based on the torque constraint matrix to obtain the torque control parameters, and calculates the assembly quality evaluation matrix by calculating the optimal assembly path and the torque control parameters according to the assembly process information; Based on the assembly quality evaluation matrix, perform fault feature extraction and pattern recognition to obtain the fault type, fault location, and fault occurrence probability. When the fault occurrence probability exceeds the preset threshold, establish a fault scenario library containing multi-level fault scenarios according to the assembly process information and the fault type, calculate the dynamic risk coefficient of the fault location based on the fault scenario library and the assembly torque data, construct a compensation optimization objective function based on the dynamic risk coefficient and the assembly quality evaluation matrix, use the compensation optimization objective function to calculate the position compensation parameter and the torque compensation parameter, convert the position compensation parameter and the torque compensation parameter into an execution instruction sequence according to the assembly process information, control the assembly execution mechanism to adjust according to the priority order of the execution instruction sequence, and feedback the adjusted data to the assembled digital twin model to dynamically update the assembly quality evaluation matrix, realizing the closed-loop optimization control of the assembly process.

[0007] In an alternative embodiment, Collect multi-angle real-time assembly process images of the target product through multiple cameras on the production line, register and map the multi-angle real-time assembly process images with the assembly reference feature points, and extract the assembly node coordinates and assembly torque data in the real-time assembly state. The generated assembled digital twin model includes: Collect multi-angle real-time assembly process images of the target product through multiple cameras on the production line, preprocess the multi-angle real-time assembly process images to obtain preprocessed images, extract the local feature description information of the preprocessed images and perform feature matching, calculate the relative position and attitude parameters between multiple cameras based on the matching results, and construct a multi-view space constraint model according to the relative position and attitude parameters; Project the pre-acquired assembly reference feature points into the image space of the preprocessed image to obtain the assembly initial points, perform real-time tracking on the assembly initial points to obtain the tracking feature points, establish a motion prediction model according to the assembly process information, input the tracking feature points into the motion prediction model to obtain the predicted feature points, and perform spatial reconstruction on the predicted feature points using the multi-view space constraint model to obtain the assembly node coordinates; Collect the torque sensor data during the assembly process as assembly torque data, align the assembly node coordinates and the assembly torque data in time series to obtain assembly state data, extract motion information and force information from the assembly state data respectively, construct a motion state vector according to the motion information, construct a force state vector according to the force information, and combine the motion state vector and the force state vector to obtain a state observation vector; Determine the state observation weight according to the assembly process information and establish a state prediction equation, filter and optimize the state observation vector to obtain real-time assembly state information, obtain standard assembly parameters based on the assembly process information, and compare the real-time assembly state information with the standard assembly parameters to obtain assembly deviation data; Generate an assembly quality evaluation result according to the assembly deviation data, fuse the assembly quality evaluation result, real-time assembly state information and assembly process information, and generate an assembly digital twin model including assembly node coordinates and assembly torque data.

[0008] In an alternative embodiment, Input the assembly digital twin model and the assembly knowledge graph into a hybrid neural network, and the hybrid neural network constructs a position constraint matrix according to the assembly reference feature points and constructs a torque constraint matrix according to the torque control threshold, including: Input the assembly digital twin model and the assembly knowledge graph into a hybrid neural network. The hybrid neural network includes a graph attention neural network module, a long short-term memory neural network module, and a recurrent neural network module. The assembly digital twin model contains a coordinate sequence and a torque data sequence of assembly reference feature points, and the assembly knowledge graph contains the topological relationship of assembly processes; Extract the process node features and edge connection features in the assembly knowledge graph, and construct a process topology graph including a vertex set and an edge set. The vertex set represents the assembly process nodes, and the edge set represents the connection relationship between the process nodes; Input the process topology graph into the graph attention neural network module, calculate the multi-head attention weight coefficients between the process nodes, perform attention weighted calculation on the process node features through a multi-layer feature transformation network, and output an assembly process feature vector representing the constraint relationship between the processes; Extract the coordinate data of adjacent feature points from the coordinate sequence of the assembly reference feature points, calculate the position deviation value between the adjacent feature points, combine the position deviation value and the assembly process feature vector in time series order to form a position feature sequence, input the position feature sequence into the long short-term memory neural network module, and process the position feature sequence through the gating unit of the long short-term memory neural network module to output a position constraint matrix; Extract the torque data within a continuous time window from the torque data sequence, calculate the torque change rate between adjacent assembly process nodes, and combine the torque change rate with the assembly process feature vector to form a torque feature sequence; Input the torque feature sequence into the recurrent neural network module, calculate the torque change trend between assembly process nodes based on the hidden layer state of the recurrent neural network module, generate a torque control threshold, calculate the ratio of the torque change rate between adjacent assembly process nodes to the torque control threshold, and obtain the torque constraint coefficient between process nodes; Construct a torque constraint matrix based on the number of process nodes. When there is a direct constraint relationship between process nodes, set the corresponding matrix element value to the torque constraint coefficient. When there is no direct constraint relationship between process nodes, set the corresponding matrix element value to zero. When it is the same process node, set the corresponding matrix element value to one, and finally obtain the torque constraint matrix.

[0009] In an alternative embodiment, Perform deviation analysis on the assembly node coordinates and the assembly accuracy threshold based on the position constraint matrix to obtain the optimal assembly path. Perform deviation analysis on the assembly torque data and the torque control threshold based on the torque constraint matrix to obtain the torque control parameters. Calculate the optimal assembly path and the torque control parameters according to the assembly process information to obtain an assembly quality evaluation matrix, including: Obtain the position constraint matrix, which contains the three-dimensional coordinate information and the assembly accuracy threshold information of the assembly nodes. Extract the coordinate information of adjacent assembly nodes, calculate the spatial distance between nodes to obtain the actual position data, perform a difference operation on the actual position data and the corresponding assembly accuracy threshold to obtain the position deviation amount, and construct the position constraint relationship between nodes based on the position deviation amount; Perform normalization processing on the position constraint relationship to obtain the standardized position constraint coefficient. Use the standardized position constraint coefficient as the path evaluation index, and adopt a heuristic search strategy to calculate the cumulative position constraint coefficient of each assembly path. Select the assembly path with the smallest cumulative position constraint coefficient as the optimal assembly path; Obtain the torque constraint matrix, which contains the torque data and the torque control threshold during the assembly process. Extract the torque data at adjacent sampling times, calculate the torque change amount to obtain the actual torque data, perform a difference operation on the actual torque data and the corresponding torque control threshold to obtain the torque deviation amount, and construct the time-series torque constraint relationship based on the torque deviation amount; Normalize the time-sequence torque constraint relationship to obtain a standardized torque constraint coefficient. Based on the standardized torque constraint coefficient, set the initial values of the proportional parameter, integral parameter, and differential parameter. Under the constraints of the torque change rate limit and the system stability index, by comparing the parameter adjustment amounts in adjacent iteration cycles, update the proportional parameter, integral parameter, and differential parameter in sequence. When the parameter adjustment amount is less than the preset adjustment threshold, obtain the final torque control parameter; Determine the connection relationship of the assembly nodes according to the node access sequence of the optimal assembly path, extract the position constraint coefficient and torque constraint coefficient corresponding to adjacent assembly nodes, and use an adaptive weight coefficient to perform weighted combination on the position constraint coefficient and torque constraint coefficient to obtain a comprehensive score. The adaptive weight coefficient is dynamically adjusted according to the importance of the position constraint and torque constraint; Determine the matrix dimension according to the total number of assembly nodes, construct an assembly quality evaluation matrix, fill the comprehensive score between adjacent assembly nodes into the corresponding row and column positions of the nodes in the assembly quality evaluation matrix, set the diagonal elements of the assembly quality evaluation matrix to unity values to represent the self-constraint of the nodes, and set the matrix elements corresponding to non-adjacent nodes to zero values to represent no constraint relationship. Perform normalization processing on the filled assembly quality evaluation matrix to obtain a standardized assembly quality evaluation matrix.

[0010] In an alternative embodiment, Based on the assembly quality evaluation matrix, perform fault feature extraction and pattern recognition to obtain the fault type, fault location, and fault occurrence probability. When the fault occurrence probability exceeds the preset threshold, establish a fault scenario library containing multi-level fault scenarios according to the assembly process information and fault type. Calculate the dynamic risk coefficient of the fault location based on the fault scenario library and assembly torque data, including: Perform multi-dimensional feature decomposition on the assembly quality evaluation matrix, construct a deep neural network containing multiple parallel convolutional branches. Each convolutional branch uses convolutional kernels of different sizes for feature extraction, and perform feature fusion on the outputs of each convolutional branch to obtain a feature matrix; use a multi-head attention mechanism to perform feature weighting on the feature matrix in the time sequence and spatial dimensions, and combine the assembly process time sequence information to obtain a fault feature vector and a process correlation matrix; Based on the fault feature vector and the process correlation matrix, construct a multi-task learning framework. Among them, extract the fault type features and fault type probability distribution through the fault classification branch, determine the fault location coordinates through the position regression branch, input the fault type features, fault type probability distribution, fault location coordinates, and process correlation matrix into a dynamic Bayesian network, and calculate the fault occurrence probability of each process node; When the failure occurrence probability of any process node exceeds a preset threshold, transform the process correlation matrix into a directed acyclic graph. Starting from the process node corresponding to the failure location coordinates, calculate the propagation influence coefficient between adjacent process nodes in combination with the failure type characteristics and the failure type probability distribution. The propagation influence coefficient represents the diffusion probability of the failure between processes. Construct a multi-level failure scenario library based on the topological structure of the directed acyclic graph and the propagation influence coefficient. Each scenario node includes a process identifier, failure type characteristics, failure occurrence probability, and propagation influence coefficient; Transform the relationship of scenario nodes in the failure scenario library into a state transition probability matrix. Combine the real-time assembly torque data to construct a Markov decision process model. Generate a random perturbation sample set by using Latin hypercube sampling based on the state transition probability matrix. Input the random perturbation sample set and the assembly torque data into the Markov decision process model to obtain a process state transition sequence. Combine the process state transition sequence, propagation influence coefficient, and failure occurrence probability, and calculate the dynamic risk coefficient of the process node corresponding to the failure location coordinates through a dynamic programming algorithm.

[0011] In an alternative embodiment, Generating a random perturbation sample set by using Latin hypercube sampling based on the state transition probability matrix, inputting the random perturbation sample set and the assembly torque data into the Markov decision process model to obtain a process state transition sequence, and calculating the dynamic risk coefficient of the process node corresponding to the failure location coordinates through a dynamic programming algorithm in combination with the process state transition sequence, propagation influence coefficient, and failure occurrence probability includes: Uniformly divide the probability value range in the state transition probability matrix to obtain multiple sub-intervals. Randomly extract multiple perturbation amounts in the multiple sub-intervals and perform a random permutation to form a perturbation sequence. Use the perturbation sequence to perturb the probability values in the state transition probability matrix to generate multiple groups of perturbed state transition probability matrices; Input multiple groups of perturbed state transition probability matrices and assembly torque data into the Markov decision process model. Determine the system state at the next moment according to the state transition equation, construct a policy function based on the immediate reward function and the state value function, determine the optimal action through the policy function, and perform iterative calculations based on the optimal action to obtain a process state transition sequence; Obtain the propagation influence coefficient and the failure occurrence probability. Input the process state transition sequence, propagation influence coefficient, and failure occurrence probability into the risk value function, where the risk value function is used to combine the failure occurrence probability of the current state, the propagation influence coefficient of the current state, and the maximum risk value of the next state. Construct a Bellman equation based on the risk value function, and perform backward recursive solution on the Bellman equation through a dynamic programming algorithm to obtain the dynamic risk coefficient of the process node corresponding to the failure location coordinates.

[0012] In an alternative embodiment, Constructing a compensation optimization objective function based on the dynamic risk coefficient and the assembly quality evaluation matrix, and calculating the position compensation parameter and the torque compensation parameter by using the compensation optimization objective function includes: Constructing a compensation optimization objective function based on the dynamic risk coefficient and the assembly quality evaluation matrix, where the compensation optimization objective function includes a deviation term of the assembly quality evaluation matrix before and after compensation, a weighted adjustment term of the dynamic risk coefficient for the position compensation parameter and the torque compensation parameter, and a constraint term for the position compensation parameter and the torque compensation parameter; Performing Taylor expansion of the position compensation parameter and the torque compensation parameter at the current position to obtain the objective gradient vector of the compensation optimization objective function at the current position; respectively calculating the second-order derivatives of each item in the compensation optimization objective function to form an objective matrix, and constructing an optimization sub-objective function with a compensation range constraint based on the objective gradient vector and the objective matrix; Taking the negative direction of the objective gradient vector as the initial search direction, solving the optimization sub-objective function in combination with the objective matrix to obtain the search direction vector of the compensation parameter; updating the current compensation parameter along the search direction vector and substituting it into the compensation optimization objective function to calculate the actual decrease of the objective function value; substituting the search direction vector into the optimization sub-objective function to calculate the predicted decrease of the objective function value; calculating the ratio of the actual decrease to the predicted decrease; When the ratio is greater than a preset acceptance threshold, updating the position compensation parameter and the torque compensation parameter based on the search direction vector, and simultaneously adjusting the adjustable ranges of the position compensation parameter and the torque compensation parameter according to the ratio; when the ratio is less than the preset acceptance threshold, keeping the position compensation parameter and the torque compensation parameter unchanged, and simultaneously reducing the adjustable ranges of the position compensation parameter and the torque compensation parameter, and repeating the iteration until the change amounts of the position compensation parameter and the torque compensation parameter are less than the convergence threshold to obtain the final position compensation parameter and torque compensation parameter.

[0013] In the second aspect of the embodiments of the present invention, an AI-based intelligent monitoring system for assembly status is provided, including: The first unit is used to obtain the assembly drawings and assembly process parameters of the target product, extract the assembly reference feature points and assembly accuracy thresholds based on the assembly drawings, extract the assembly process information and torque control thresholds based on the assembly process parameters, construct an assembly knowledge graph with the assembly reference feature points, assembly accuracy thresholds, assembly process information and torque control thresholds. Meanwhile, collect multi-angle real-time assembly process images of the target product through multiple cameras on the production line, register and map the multi-angle real-time assembly process images with the assembly reference feature points, extract the assembly node coordinates and assembly torque data in the real-time assembly state, and generate an assembly digital twin model; The second unit is used to input the assembly digital twin model and the assembly knowledge graph into a hybrid neural network. The hybrid neural network constructs a position constraint matrix according to the assembly reference feature points, constructs a torque constraint matrix according to the torque control thresholds, performs deviation analysis on the assembly node coordinates and the assembly accuracy thresholds based on the position constraint matrix to obtain the optimal assembly path, performs deviation analysis on the assembly torque data and the torque control thresholds based on the torque constraint matrix to obtain the torque control parameters, and calculates the assembly quality evaluation matrix by calculating the optimal assembly path and the torque control parameters according to the assembly process information; The third unit is used to extract fault features and perform pattern recognition based on the assembly quality evaluation matrix to obtain the fault type, fault location and fault occurrence probability. When the fault occurrence probability exceeds the preset threshold, establish a fault scenario library containing multi-level fault scenarios according to the assembly process information and the fault type, calculate the dynamic risk coefficient of the fault location based on the fault scenario library and the assembly torque data, construct a compensation optimization objective function based on the dynamic risk coefficient and the assembly quality evaluation matrix, calculate the position compensation parameter and the torque compensation parameter by using the compensation optimization objective function, convert the position compensation parameter and the torque compensation parameter into an execution instruction sequence according to the assembly process information, control the assembly execution mechanism to adjust according to the priority order of the execution instruction sequence, and feedback the adjusted data to the assembly digital twin model to dynamically update the assembly quality evaluation matrix, realizing the closed-loop optimization control of the assembly process.

[0014] The third aspect of the embodiments of the present invention provides an electronic device, including: a processor; a memory for storing instructions executable by the processor; wherein, the processor is configured to call the instructions stored in the memory to execute the method described above.

[0015] The fourth aspect of the embodiments of the present invention Provided is a computer-readable storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the foregoing method is implemented.

[0016] In this embodiment, by constructing an assembly knowledge graph and a digital twin model, and combining a hybrid neural network to monitor and analyze the assembly process in real time, assembly deviations can be accurately identified, and optimized according to the optimal assembly path and torque control parameters, effectively improving the assembly accuracy and quality. It is possible to extract features and perform pattern recognition on the faults during the assembly process, predict potential faults, and perform compensation optimization according to the fault scenario library and the dynamic risk coefficient, thereby effectively reducing the probability of faults and improving the reliability of the assembly process. Based on the assembly quality evaluation matrix and the compensation optimization objective function, an execution instruction sequence is automatically generated to control the assembly execution mechanism to make adjustments, realizing the closed-loop optimization control of the assembly process and improving the assembly efficiency and automation level. Description of the Drawings

[0017] Figure 1 It is a schematic flowchart of the AI-based intelligent monitoring method for assembly status according to an embodiment of the present invention; Figure 2 It is a schematic structural diagram of the AI-based intelligent monitoring system for assembly status according to an embodiment of the present invention. Detailed Embodiments

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0019] The technical solutions of the present invention will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0020] Figure 1 It is a schematic flowchart of the AI-based intelligent monitoring method for assembly status according to an embodiment of the present invention, as Figure 1 shown, the method includes: S101. Obtain the assembly drawings and assembly process parameters of the target product. Extract the assembly reference feature points and assembly accuracy thresholds based on the assembly drawings, and extract the assembly process information and torque control thresholds based on the assembly process parameters. Construct an assembly knowledge graph with the assembly reference feature points, assembly accuracy thresholds, assembly process information, and torque control thresholds. At the same time, collect multi-angle real-time assembly process images of the target product through multiple cameras on the production line, register and map the multi-angle real-time assembly process images with the assembly reference feature points, extract the assembly node coordinates and assembly torque data in the real-time assembly state, and generate an assembly digital twin model; S102. Input the assembly digital twin model and the assembly knowledge graph into a hybrid neural network. The hybrid neural network constructs a position constraint matrix based on the assembly reference feature points, constructs a torque constraint matrix based on the torque control thresholds, performs deviation analysis on the assembly node coordinates and the assembly accuracy thresholds based on the position constraint matrix to obtain the optimal assembly path, performs deviation analysis on the assembly torque data and the torque control thresholds based on the torque constraint matrix to obtain the torque control parameters, and calculates the assembly quality evaluation matrix by computing the optimal assembly path and the torque control parameters according to the assembly process information; S103. Based on the assembly quality evaluation matrix, perform fault feature extraction and pattern recognition to obtain the fault type, fault location, and fault occurrence probability. When the fault occurrence probability exceeds the preset threshold, establish a fault scenario library containing multi-level fault scenarios according to the assembly process information and the fault type. Calculate the dynamic risk coefficient of the fault location based on the fault scenario library and the assembly torque data. Construct a compensation optimization objective function based on the dynamic risk coefficient and the assembly quality evaluation matrix. Use the compensation optimization objective function to calculate the position compensation parameters and torque compensation parameters. Convert the position compensation parameters and torque compensation parameters into an execution instruction sequence according to the assembly process information, control the assembly execution mechanism to adjust according to the priority order of the execution instruction sequence, and feedback the adjusted data to the assembly digital twin model to dynamically update the assembly quality evaluation matrix, realizing the closed-loop optimization control of the assembly process.

[0021] Among them, the assembly reference feature points refer to the key structural points used to determine the position and angle during the assembly process, such as the center of the hole position, the edge of the mating surface, etc.; the assembly accuracy threshold refers to the allowable error range during the assembly process, such as the allowable deviation range of the hole position. The assembly process parameters include the operation steps, tool specifications, assembly sequence, etc. of each assembly process, as well as the key data for controlling the assembly process such as the torque control threshold.

[0022] Based on the above assembly drawings and assembly process parameters, extract the assembly process information and torque control thresholds, and construct an assembly knowledge graph with the assembly reference feature points, assembly precision thresholds, assembly process information, and torque control thresholds. The assembly knowledge graph is a data model centered on knowledge association, showing the associations between assembly reference feature points and assembly precision thresholds, and between assembly process information and torque control thresholds. Through this knowledge graph, the dependency relationships and constraint conditions of various elements in the assembly process of the target product can be comprehensively described.

[0023] The assembly digital twin model is a virtual dynamic mapping model of the product assembly process, which synchronizes the assembly state of the target product in real time and combines with the assembly knowledge graph to provide a basis for assembly quality analysis and optimization.

[0024] In an optional implementation, collect multi-angle real-time assembly process images of the target product through multiple cameras on the production line, register and map the multi-angle real-time assembly process images with the assembly reference feature points, extract the assembly node coordinates and assembly torque data in the real-time assembly state, and generate the assembly digital twin model including: Collect multi-angle real-time assembly process images of the target product through multiple cameras on the production line, preprocess the multi-angle real-time assembly process images to obtain preprocessed images, extract local feature description information of the preprocessed images and perform feature matching, calculate the relative position and attitude parameters between multiple cameras based on the matching results, and construct a multi-view space constraint model according to the relative position and attitude parameters; Project the pre-acquired assembly reference feature points into the image space of the preprocessed image to obtain assembly initial points, perform real-time tracking on the assembly initial points to obtain tracking feature points, establish a motion prediction model according to the assembly process information, input the tracking feature points into the motion prediction model to obtain predicted feature points, and use the multi-view space constraint model to perform spatial reconstruction on the predicted feature points to obtain assembly node coordinates; Collect the torque sensor data during the assembly process as the assembly torque data, align the assembly node coordinates and the assembly torque data in time series to obtain assembly state data, respectively extract motion information and force information from the assembly state data, construct a motion state vector according to the motion information, construct a force state vector according to the force information, and combine the motion state vector and the force state vector to obtain a state observation vector; Determine the state observation weight according to the assembly process information and establish a state prediction equation, filter and optimize the state observation vector to obtain real-time assembly state information, obtain standard assembly parameters based on the assembly process information, and compare the real-time assembly state information with the standard assembly parameters to obtain assembly deviation data; Generate an assembly quality assessment result based on the described assembly deviation data, and perform data fusion on the assembly quality assessment result, real-time assembly status information, and assembly process information to generate an assembly digital twin model containing assembly node coordinates and assembly torque data.

[0025] Exemplarily, first, collect multi-angle real-time assembly process images of the target product through multiple cameras arranged on the production line. Suppose there are four cameras shooting from the front, back, left, and right of the product respectively, collecting color images with a resolution of 1920×1080 pixels and a frame rate of 30 frames per second. Then preprocess the collected images, including operations such as image denoising, distortion correction, and color correction. For example, use Gaussian filtering to remove image noise, use camera calibration parameters for distortion correction, and perform white balance processing to ensure color consistency.

[0026] Next, extract local feature description information from the preprocessed images. For example, use the SIFT algorithm to extract image feature points and their descriptors. Then perform feature matching. For example, use the FLANN algorithm for fast nearest neighbor search to find the matching relationship of corresponding feature points between images from different perspectives. Suppose 1000 feature points are found in two images respectively, and 500 pairs of feature points are successfully matched. Based on these matching results, calculate the relative position and attitude parameters between multiple cameras. For example, use the RANSAC algorithm to estimate the rotation and translation matrices between cameras. According to these relative position and attitude parameters, construct a multi-view space constraint model, which describes the geometric relationship between images from different perspectives.

[0027] Then, project the pre-acquired assembly reference feature points into the image space of the preprocessed images to obtain assembly initial points. Suppose the reference feature point is a screw hole on the product, and its coordinates in the world coordinate system are known. Project it into the image coordinate system through the camera projection matrix. Perform real-time tracking on the assembly initial points to obtain tracked feature points. For example, use the optical flow method or Kalman filter to track the feature points. According to the assembly process information, establish a motion prediction model. For example, use a linear or non-linear model to predict the motion trajectory of the feature points. Input the tracked feature points into the motion prediction model to obtain predicted feature points. Use the multi-view space constraint model to perform spatial reconstruction on the predicted feature points to obtain assembly node coordinates.

[0028] Meanwhile, collect the data of the torque sensor during the assembly process as the assembly torque data. For example, use the torque sensor fixed on the assembly tool to collect the torque information during the assembly process, and the sampling frequency is 100Hz. Align the assembly node coordinates and the assembly torque data in time series to obtain the assembly state data. For example, synchronize the image acquisition timestamp and the torque data timestamp, and correlate the corresponding coordinates and torque data. Extract the motion information and the force information from the assembly state data respectively. For example, calculate the motion information such as speed and acceleration from the coordinate data, and extract the force information such as the magnitude and direction of the torque from the torque data. Construct the motion state vector according to the motion information, and construct the force state vector according to the force information. For example, compose the speed, acceleration, etc. into the motion state vector, and compose the magnitude and direction of the torque, etc. into the force state vector. Combine the motion state vector and the force state vector to obtain the state observation vector.

[0029] Determine the state observation weight according to the assembly process information and establish the state prediction equation. For example, assign different weights according to the requirements of motion and force in different processes. Filter and optimize the state observation vector. For example, use the Kalman filter to filter the state observation vector to obtain the real-time assembly state information. Obtain the standard assembly parameters based on the assembly process information. For example, obtain the standard coordinates, torques and other parameters from the assembly process document. Compare the real-time assembly state information with the standard assembly parameters to obtain the assembly deviation data. For example, calculate the difference between the real-time coordinates and the standard coordinates, and the difference between the real-time torque and the standard torque.

[0030] Generate the assembly quality evaluation result according to the assembly deviation data. For example, judge whether the assembly quality is qualified according to the deviation magnitude. Perform data fusion on the assembly quality evaluation result, the real-time assembly state information and the assembly process information to generate an assembly digital twin model containing the assembly node coordinates and the assembly torque data. This model can intuitively display the state changes and quality information during the assembly process.

[0031] In this embodiment, through the fusion of multi-camera vision and torque sensor data, the state information during the assembly process can be obtained more comprehensively, thereby improving the accuracy of assembly quality monitoring. At the same time, the digital twin model can display the assembly process in real time, which is convenient for operators to monitor and analyze, and improves the monitoring efficiency. This method records the complete assembly process data, including the assembly node coordinates, the assembly torque data and the assembly quality evaluation result, which can facilitate the traceability and analysis of the assembly process, find out the cause of the problem and make improvements. By analyzing the data in the digital twin model, the deficiencies in the assembly process and flow can be found and optimized, thereby improving the assembly efficiency and product quality.

[0032] In an alternative embodiment, the assembled digital twin model and the assembly knowledge graph are input into a hybrid neural network. The hybrid neural network constructs a position constraint matrix based on the assembly reference feature points and constructs a torque constraint matrix based on the torque control threshold, including: The assembled digital twin model and the assembly knowledge graph are input into a hybrid neural network. The hybrid neural network includes a graph attention neural network module, a long short-term memory neural network module, and a recurrent neural network module. The assembled digital twin model contains a coordinate sequence and a torque data sequence of assembly reference feature points, and the assembly knowledge graph contains the topological relationship of assembly processes; Extract the process node features and edge connection features in the assembly knowledge graph, and construct a process topology graph including a vertex set and an edge set. The vertex set represents the assembly process nodes, and the edge set represents the connection relationship between the process nodes; Input the process topology graph into the graph attention neural network module, calculate the multi-head attention weight coefficients between the process nodes, perform attention-weighted calculation on the process node features through a multi-layer feature transformation network, and output an assembly process feature vector representing the constraint relationship between processes; Extract the coordinate data of adjacent feature points from the coordinate sequence of the assembly reference feature points, calculate the position deviation value between the adjacent feature points, combine the position deviation value and the assembly process feature vector in chronological order into a position feature sequence, input the position feature sequence into the long short-term memory neural network module, and process the position feature sequence through the gating unit of the long short-term memory neural network module to output a position constraint matrix; Extract the torque data within a continuous time window from the torque data sequence, calculate the torque change rate between adjacent assembly process nodes, and combine the torque change rate and the assembly process feature vector into a torque feature sequence; Input the torque feature sequence into the recurrent neural network module, calculate the torque change trend between the assembly process nodes based on the hidden layer state of the recurrent neural network module to generate a torque control threshold, calculate the ratio of the torque change rate between adjacent assembly process nodes to the torque control threshold to obtain the torque constraint coefficient between the process nodes; Construct a torque constraint matrix based on the number of process nodes. When there is a direct constraint relationship between process nodes, the corresponding matrix element value is set to the torque constraint coefficient. When there is no direct constraint relationship between process nodes, the corresponding matrix element value is set to zero. When it is the same process node, the corresponding matrix element value is set to one, and finally a torque constraint matrix is obtained.

[0033] Exemplarily, first, prepare to assemble the digital twin model and the assembly knowledge graph. The digital twin model contains the three-dimensional coordinate sequence of the assembly reference feature points and the torque data sequence at each time step. For example, for the assembly of a mobile phone case, the reference feature points can be the positions of the screw holes, and the coordinate sequence records the spatial positions of these screw holes at different assembly stages. The torque data sequence records the magnitude of the torque applied to the assembly components at each time step. The assembly knowledge graph describes the topological relationship between the assembly processes in the form of a graph. For example, install component A first, then install component B, and there is a connection relationship between A and B. The knowledge graph can be stored using the graph database Neo4j, where the nodes represent the processes and the edges represent the dependency relationships between the processes. For example, there is a directed edge between the "install screen" node and the "install battery" node, indicating that the screen must be installed before the battery.

[0034] Then, extract the process node features and edge connection features from the assembly knowledge graph to construct a process topology graph. The node features can be the name, type, etc. of the process, and the edge connection features can be the type of dependency relationship between the processes, such as strong dependency or weak dependency. Assume that the mobile phone case assembly contains three processes: install the screen, install the battery, and install the back cover. Then the process topology graph contains three nodes, representing these three processes respectively, and the directed edges connecting these nodes represent their assembly order.

[0035] Next, input the process topology graph into the graph attention neural network module. This module calculates the multi-head attention weight coefficients between the process nodes and performs attention-weighted calculation on the process node features through a multi-layer feature transformation network, and outputs the assembly process feature vector representing the constraint relationship between the processes. For example, if the "install screen" process has a greater impact on the "install battery" process, the attention weight coefficient between them will be relatively high. Assume that the output assembly process feature vector is a three-dimensional vector, representing the features of the three processes respectively.

[0036] Subsequently, extract the coordinate data of adjacent feature points from the coordinate sequence of the assembly reference feature points, and calculate the position deviation value between the adjacent feature points. For example, after installing the screen, the position of the screw hole may shift slightly, and this shift value is the position deviation value. Combine the position deviation value with the assembly process feature vector output from the graph attention neural network module in chronological order to form a position feature sequence, and input it into the long short-term memory neural network module. This module processes the position feature sequence through the gating unit and outputs a position constraint matrix. The position constraint matrix reflects the mutual influence relationship of the position deviation of the reference feature points between different processes. Assume that the position constraint matrix is a 3x3 matrix, representing the pairwise position constraint relationships between the three processes.

[0037] Meanwhile, torque data within a continuous time window are extracted from the torque data sequence, and the torque change rate between adjacent assembly process nodes is calculated. For example, when installing a battery, the torque may suddenly increase, and this change rate reflects the torque change between processes. The torque change rate and the assembly process feature vector are combined into a torque feature sequence and input into the recurrent neural network module. The recurrent neural network module calculates the torque change trend between assembly process nodes based on its hidden layer state and generates a torque control threshold. Then, the ratio of the torque change rate between adjacent assembly process nodes to the torque control threshold is calculated to obtain the torque constraint coefficient between process nodes.

[0038] Finally, a torque constraint matrix is constructed based on the number of process nodes. When there is a direct constraint relationship between process nodes, the corresponding matrix element value is set to the torque constraint coefficient; when there is no direct constraint relationship between process nodes, the corresponding matrix element value is set to zero; when it is the same process node, the corresponding matrix element value is set to one. The finally obtained torque constraint matrix reflects the mutual influence and constraint relationship of torque changes between different processes. Assume the torque constraint matrix is also a 3x3 matrix, representing the pairwise torque constraint relationships between three processes.

[0039] In this embodiment, by considering the constraint relationship between processes, the robot can perform assembly operations more precisely, thereby improving the assembly accuracy and reducing assembly errors. Through the torque constraint matrix, the magnitude of the torque applied by the robot can be optimized to avoid assembly failures caused by excessive or insufficient torque, thereby improving the assembly efficiency. By considering the topological relationship and torque change trend between processes, the robustness of the assembly process can be enhanced, enabling it to adapt to more complex assembly environments and working conditions.

[0040] In an alternative embodiment, a deviation analysis is performed on the assembly node coordinates and the assembly accuracy threshold based on the position constraint matrix to obtain an optimal assembly path, a deviation analysis is performed on the assembly torque data and the torque control threshold based on the torque constraint matrix to obtain torque control parameters, and the optimal assembly path and the torque control parameters are calculated according to the assembly process information to obtain an assembly quality evaluation matrix, including: Obtain the position constraint matrix, which contains the three-dimensional coordinate information and the assembly accuracy threshold information of the assembly nodes, extract the coordinate information of adjacent assembly nodes, calculate the spatial distance between nodes to obtain the actual position data, perform a difference operation on the actual position data and the corresponding assembly accuracy threshold to obtain the position deviation amount, and construct the position constraint relationship between nodes based on the position deviation amount; Normalize the position constraint relationship to obtain a standardized position constraint coefficient. Use the standardized position constraint coefficient as a path evaluation index, and adopt a heuristic search strategy to calculate the cumulative position constraint coefficient of each assembly path. Select the assembly path with the smallest cumulative position constraint coefficient as the optimal assembly path; Obtain a torque constraint matrix, which contains torque data and torque control thresholds during the assembly process. Extract the torque data at adjacent sampling moments, calculate the torque change amount to obtain the actual torque data, perform a difference operation between the actual torque data and the corresponding torque control threshold to obtain a torque deviation amount, and construct a time-series torque constraint relationship based on the torque deviation amount; Normalize the time-series torque constraint relationship to obtain a standardized torque constraint coefficient. Based on the standardized torque constraint coefficient, set the initial values of the proportional parameter, integral parameter, and differential parameter. Under the constraints of the torque change rate limit and the system stability index, by comparing the parameter adjustment amounts in adjacent iteration cycles, update the proportional parameter, integral parameter, and differential parameter in sequence. When the parameter adjustment amount is less than the preset adjustment threshold, obtain the final torque control parameter; Determine the connection relationship of the assembly nodes according to the node access sequence of the optimal assembly path. Extract the position constraint coefficient and torque constraint coefficient corresponding to adjacent assembly nodes, and use an adaptive weight coefficient to perform a weighted combination of the position constraint coefficient and the torque constraint coefficient to obtain a comprehensive score. The adaptive weight coefficient is dynamically adjusted according to the importance of the position constraint and the torque constraint; Determine the matrix dimension according to the total number of assembly nodes, construct an assembly quality evaluation matrix, fill the comprehensive score between adjacent assembly nodes into the corresponding row and column positions of the nodes in the assembly quality evaluation matrix, set the diagonal elements of the assembly quality evaluation matrix to unit values to represent the self-constraint of the nodes, and set the matrix elements corresponding to non-adjacent nodes to zero values to represent no constraint relationship. Normalize the filled assembly quality evaluation matrix to obtain a standardized assembly quality evaluation matrix.

[0041] Exemplarily, first, obtain a position constraint matrix, which contains the three-dimensional coordinate information and assembly precision threshold information of the assembly nodes. Extract the three-dimensional coordinates of adjacent assembly nodes in the assembly path, and calculate the actual spatial distance between these nodes to obtain the actual position data during the assembly process. Compare the actual position data of each pair of nodes with the corresponding assembly precision threshold, and calculate the difference between the two. The result is the position deviation amount. The position deviation amount reflects the error size of the spatial alignment in the assembly path and is used to further construct the position constraint relationship between the nodes. The position constraint relationship is constructed by analyzing the relative position precision and allowable error of the nodes and represents the spatial coupling degree between the nodes during the assembly process.

[0042] Normalize the position constraint relationship, unify the deviation between different node pairs into a standardized position constraint coefficient, which is convenient for comparison during the path optimization process. The standardized position constraint coefficient is used as a path evaluation index. By analyzing each assembly path one by one, calculate the cumulative value of the position constraint coefficients of all adjacent nodes in each path. The smaller the cumulative value, the higher the overall assembly accuracy of the path. Use a heuristic search strategy to screen all possible assembly paths, select the path with the smallest cumulative position constraint coefficient, and determine it as the optimal assembly path. This path optimization method combines the spatial accuracy requirements between assembly nodes and optimizes the assembly path from a global perspective to ensure the final assembly quality.

[0043] Obtain the torque constraint matrix, which contains the torque data and torque control thresholds at different stages of the assembly process. Extract the torque data at adjacent sampling moments during the assembly process, calculate the torque change amount between these moments, and form the actual torque data sequence. Perform a difference operation between the actual torque data at each moment and the corresponding torque control threshold to obtain the torque deviation amount. The torque deviation amount is used to analyze the rationality of the torque change during the assembly process and its impact on the assembly quality, and further construct the time-series torque constraint relationship. The torque constraint relationship describes the dynamic change characteristics of the torque during the assembly process.

[0044] Normalize the time-series torque constraint relationship to obtain the standardized torque constraint coefficient. Based on the standardized torque constraint coefficient, set the initial values of the proportional parameter, integral parameter, and differential parameter, and perform dynamic adjustment in combination with the torque change rate limit and the system stability index. By gradually optimizing these parameters, compare the magnitudes of the adjustment amounts in each iteration cycle to ensure that the parameter changes tend to be stable. When the parameter adjustment amount is less than the preset adjustment threshold, determine the final torque control parameters. The final parameter settings are used to control the smoothness of the torque during the assembly process to prevent the assembly accuracy from decreasing or the components from being damaged due to excessive torque fluctuations.

[0045] According to the node access order of the optimal assembly path, determine the connection relationship between the assembly nodes. Extract the corresponding position constraint coefficient and torque constraint coefficient of adjacent assembly nodes, and use an adaptive weight coefficient to perform a weighted combination of these two types of constraint coefficients. The adaptive weight coefficient is dynamically adjusted based on the importance of the position constraint and torque constraint during the assembly process. For example, when the alignment accuracy requirement of the assembly node is high, the weight of the position constraint coefficient can be increased, while when the torque control is more important for the component safety, the weight of the torque constraint coefficient is increased. The weight dynamic adjustment mechanism can meet different optimization requirements at different assembly stages.

[0046] Determine the total number of assembly nodes, and construct the initial structure of the assembly quality evaluation matrix according to the number of nodes. Each row and each column of the matrix correspond to an assembly node respectively, which is used to represent the comprehensive constraint relationship between assembly nodes. Fill the comprehensive score obtained by weighted combination between adjacent assembly nodes into the corresponding row and column positions in the matrix. For the constraint relationship of the node itself, set the elements on the diagonal of the matrix to the unit value to represent self-constraint; for non-adjacent nodes, set the elements at the corresponding positions in the matrix to zero to represent the absence of direct constraint relationship.

[0047] Normalize the constructed assembly quality evaluation matrix to form a standardized assembly quality evaluation matrix. The normalized matrix comprehensively reflects factors such as spatial accuracy and torque smoothness in the assembly path and assembly process, providing a quantitative basis for subsequent assembly optimization and quality evaluation.

[0048] In this embodiment, the optimization of the assembly path can significantly improve the spatial accuracy of node alignment and reduce assembly problems caused by cumulative errors. The dynamic adjustment of the torque control parameters ensures the torque smoothness during the assembly process, improving the safety and quality consistency of the assembly. By constructing a standardized assembly quality evaluation matrix, a global quantitative evaluation of the assembly process is realized, providing a scientific basis for subsequent assembly optimization. Generally speaking, this technical solution can meet the requirements of high-precision and high-quality assembly, and is applicable to path planning and quality management in complex assembly systems.

[0049] In an alternative implementation, based on the assembly quality evaluation matrix, fault feature extraction and pattern recognition are performed to obtain the fault type, fault location, and fault occurrence probability. When the fault occurrence probability exceeds a preset threshold, a fault scenario library containing multi-level fault scenarios is established according to the assembly process information and the fault type. Calculating the dynamic risk coefficient of the fault location based on the fault scenario library and the assembly torque data includes: Perform multi-dimensional feature decomposition on the assembly quality evaluation matrix, construct a deep neural network containing multiple parallel convolutional branches, each convolutional branch uses convolutional kernels of different sizes for feature extraction, and fuse the outputs of each convolutional branch to obtain a feature matrix; use the multi-head attention mechanism to perform feature weighting on the feature matrix in the time series and spatial dimensions, and combine the assembly process time series information to obtain a fault feature vector and a process association matrix; Based on the fault feature vector and the process association matrix, construct a multi-task learning framework. Among them, extract the fault type features and the fault type probability distribution through the fault classification branch, determine the fault location coordinates through the position regression branch, and input the fault type features, the fault type probability distribution, the fault location coordinates, and the process association matrix into the dynamic Bayesian network to calculate the fault occurrence probability of each process node; When the failure occurrence probability of any process node exceeds a preset threshold, convert the process correlation matrix into a directed acyclic graph. Starting from the process node corresponding to the failure location coordinates, calculate the propagation influence coefficient between adjacent process nodes in combination with the failure type characteristics and the failure type probability distribution. The propagation influence coefficient represents the diffusion probability of the failure between processes. Based on the topological structure of the directed acyclic graph and the propagation influence coefficient, construct a multi-level failure scenario library, where each scenario node includes a process identifier, failure type characteristics, failure occurrence probability, and propagation influence coefficient; Convert the relationship of scenario nodes in the failure scenario library into a state transition probability matrix, construct a Markov decision process model in combination with real-time assembly torque data, generate a random perturbation sample set using Latin hypercube sampling based on the state transition probability matrix, input the random perturbation sample set and the assembly torque data into the Markov decision process model to obtain a process state transition sequence, and combine the process state transition sequence, propagation influence coefficient, and failure occurrence probability to calculate the dynamic risk coefficient of the process node corresponding to the failure location coordinates through a dynamic programming algorithm.

[0050] Exemplarily, first, perform feature extraction on the assembly quality evaluation matrix. Decompose the evaluation matrix into multiple feature dimensions, such as torque, displacement, angle, etc. Then, construct a deep neural network that includes multiple parallel convolutional branches. Each branch uses convolutional kernels of different sizes to extract features of different scales. For example, one branch uses a 3x3 convolutional kernel to extract local detail features, and another branch uses a 5x5 convolutional kernel to extract features in a larger range. Fuse the outputs of each branch to obtain a feature matrix. Using the multi-head attention mechanism, weight the feature matrix in the temporal and spatial dimensions according to the temporal information of the assembly process, and finally obtain a failure feature vector and a process correlation matrix. For example, if the torque feature at a certain time point appears abnormal in multiple processes, this torque feature will be assigned a higher weight.

[0051] Next, based on the failure feature vector and the process correlation matrix, construct a multi-task learning framework. This framework includes a failure classification branch and a location regression branch. The failure classification branch is used to extract failure type characteristics and the failure type probability distribution. For example, it can identify that the failure type is "screw loose" or "part missing" and give the probability of each failure type. The location regression branch is used to determine the coordinates of the failure location. For example, it can determine which specific location on the product the failure occurs. Input the failure type characteristics, the failure type probability distribution, the failure location coordinates, and the process correlation matrix into a dynamic Bayesian network to calculate the failure occurrence probability of each process node.

[0052] If the failure occurrence probability of any process node exceeds the preset threshold, the process association matrix is transformed into a directed acyclic graph. Taking the process node corresponding to the failure location coordinates as the starting point, combining the failure type characteristics and the failure type probability distribution, the propagation influence coefficient between adjacent process nodes is calculated. The propagation influence coefficient represents the probability of failure spreading between processes. For example, if a "screw loose" failure occurs in process A, it may affect subsequent processes B and C, and the propagation influence coefficient can quantify this influence. Based on the topological structure of the directed acyclic graph and the propagation influence coefficient, a multi-level failure scenario library is constructed. Each scenario node contains process identification, failure type characteristics, failure occurrence probability, and propagation influence coefficient.

[0053] Finally, the relationship between scenario nodes in the failure scenario library is transformed into a state transition probability matrix. Combining the real-time assembly torque data, a Markov decision process model is constructed. Based on the state transition probability matrix, a random perturbation sample set is generated using Latin hypercube sampling. The random perturbation sample set and the real-time assembly torque data are input into the Markov decision process model to obtain the process state transition sequence. Combining the process state transition sequence, the propagation influence coefficient, and the failure occurrence probability, the dynamic risk coefficient of the process node corresponding to the failure location coordinates is calculated through the dynamic programming algorithm. For example, assuming that the torque data shows that the torque value of process B is abnormally high, combining the previous failure scenario library and the state transition probability, the dynamic risk coefficient of process B can be calculated to evaluate the risk degree of failure occurrence in this process.

[0054] Suppose in an assembly process involving three processes (A, B, C), the collected assembly quality evaluation matrix contains data in three dimensions: torque, displacement, and angle. Through feature extraction and a multi-task learning framework, it is identified that there is a "screw loose" failure in process B with a probability of 80% and the failure location coordinates are (10, 20). The failure occurrence probability of process B exceeds the preset threshold (e.g., 70%). A failure scenario library is constructed, and one of the scenarios is that the "screw loose" failure propagates from process B to process C with a propagation influence coefficient of 60%. Combining the real-time assembly torque data and the Markov decision process model, the dynamic risk coefficient of process B is calculated to be 0.9.

[0055] In this embodiment, through multi-dimensional feature decomposition and a deep learning model, failure features can be extracted more accurately, and the failure type and location can be identified. By constructing a failure scenario library and a Markov decision process model, the failure risk can be dynamically evaluated according to real-time data, so as to take preventive measures in a timely manner. By analyzing the failure occurrence probability and the propagation influence coefficient, the weak links in the assembly process can be identified and optimized to improve product quality and production efficiency.

[0056] In an alternative embodiment, it includes: generating a set of random perturbation samples by using Latin hypercube sampling based on the state transition probability matrix, inputting the set of random perturbation samples and the assembly torque data into the Markov decision process model to obtain a process state transition sequence, and calculating the dynamic risk coefficient of the process node corresponding to the fault location coordinate by means of a dynamic programming algorithm in combination with the process state transition sequence, the propagation influence coefficient, and the fault occurrence probability, including: Uniformly divide the probability value range in the state transition probability matrix into multiple sub - intervals, randomly extract multiple perturbation amounts in the multiple sub - intervals and randomly arrange them to form a perturbation sequence, and use the perturbation sequence to perturb the probability values in the state transition probability matrix to generate multiple groups of perturbed state transition probability matrices; Input multiple groups of perturbed state transition probability matrices and assembly torque data into the Markov decision process model, determine the system state at the next moment according to the state transition equation, construct a policy function based on the immediate reward function and the state value function, determine the optimal action through the policy function, and perform iterative calculation based on the optimal action to obtain a process state transition sequence; Obtain the propagation influence coefficient and the fault occurrence probability, input the process state transition sequence, the propagation influence coefficient, and the fault occurrence probability into the risk value function, where the risk value function is used to combine the fault occurrence probability of the current state, the propagation influence coefficient of the current state, and the maximum risk value of the next state; construct a Bellman equation based on the risk value function, and perform backward recursive solution on the Bellman equation through a dynamic programming algorithm to obtain the dynamic risk coefficient of the process node corresponding to the fault location coordinate.

[0057] Exemplarily, first, obtain the process state transition probability matrix of the assembly process. This matrix describes the probability of transitioning from one process state to another during the assembly process. Next, uniformly divide the value range of each probability value in the state transition probability matrix into several sub - intervals to ensure that the length of each interval is the same. For each sub - interval, randomly generate multiple perturbation amounts and arrange them in a random order to generate a perturbation sequence. These perturbation sequences are used to adjust the probability values in the state transition probability matrix. For example, for an interval with an initial probability value range from 0.2 to 0.4, it can be divided into five sub - intervals, each with a length of 0.04. Randomly generate perturbation amounts in each sub - interval, such as 0.03, - 0.02, etc., and apply them to the original probability value one by one. After processing, a series of perturbed state transition probability matrices are generated. These matrices are used to simulate various random perturbations that may occur during the assembly process.

[0058] Input the perturbed state transition probability matrix and assembly torque data into the Markov decision process model. The model determines the state of the system at the next moment according to the state transition equation. The assembly torque data contains torque values of multiple process nodes at different time points, reflecting the mechanical interaction and force changes between nodes during the assembly process. The immediate reward function is used to evaluate the benefit or cost of each state, and the state value function provides an estimate of the cumulative effect from the current state to the final state. The policy function combines the immediate reward function and the state value function, and determines the optimal decision for the current state by selecting the action that can maximize the cumulative benefit. For example, when the torque of a node exceeds the threshold, the policy function may suggest reducing the torque or adjusting the assembly sequence to minimize the failure risk. Through iterative calculation, the current state and policy are continuously updated, and finally a sequence of process state transitions is generated.

[0059] Combined with the process state transition sequence, obtain the propagation influence coefficient and the probability of failure occurrence of the assembly system. The propagation influence coefficient represents the coupling strength between assembly nodes, and the higher the value, the greater the mutual influence between nodes. The probability of failure occurrence is estimated based on the physical characteristics of nodes, environmental conditions, and assembly historical data during the assembly process. For example, a node may have a higher probability of failure due to frequent high-intensity force operations. Input the process state transition sequence, propagation influence coefficient, and probability of failure occurrence into the risk value function, which calculates by integrating the probability of failure occurrence, propagation influence coefficient of the current state, and the maximum risk value of the next state. Through this function, the potential impact on the overall system when a failure occurs at a certain node can be evaluated.

[0060] Construct the Bellman equation according to the calculation result of the risk value function. The Bellman equation describes the dynamic optimization process of the system in a recursive manner and solves it backward from the terminal state to the initial state. The dynamic programming algorithm is used to gradually optimize the risk value of each node, combining the failure risk of the current state with the cumulative risk that the next state may bring, and recursively solving the dynamic risk coefficient of each node. For example, when the risk coefficient of an assembly node is high and the propagation influence coefficient is large, the dynamic programming algorithm will give priority to adjusting the assembly path or strategy to reduce the overall risk of the node. Finally, when the dynamic programming process is completed, the dynamic risk coefficients of each process node during the assembly process can be obtained, which are used to guide the further optimization of the assembly process.

[0061] In this embodiment, by generating multiple groups of perturbed state transition probability matrices, the influence of random perturbations on the state of the assembly system can be flexibly simulated, providing richer process analysis data. The Markov decision process model, combined with the dynamic optimization of the state transition sequence, ensures the global optimality of the assembly path. The introduction of the propagation influence coefficient and the failure occurrence probability makes the risk assessment more targeted, while the combination of the Bellman equation and the dynamic programming algorithm significantly improves the accuracy and efficiency of risk calculation. The overall solution can not only effectively reduce the failure risk during the assembly process, but also improve the stability and assembly quality of the system.

[0062] In an alternative embodiment, a compensation optimization objective function is constructed based on the dynamic risk coefficient and the assembly quality evaluation matrix. Calculating the position compensation parameter and the torque compensation parameter using the compensation optimization objective function includes: A compensation optimization objective function is constructed based on the dynamic risk coefficient and the assembly quality evaluation matrix. The compensation optimization objective function includes a deviation term of the assembly quality evaluation matrix before and after compensation, a weighted adjustment term of the dynamic risk coefficient for the position compensation parameter and the torque compensation parameter, and a constraint term for the position compensation parameter and the torque compensation parameter; The position compensation parameter and the torque compensation parameter are Taylor-expanded at the current position to obtain the target gradient vector of the compensation optimization objective function at the current position; the second-order derivatives of each term in the compensation optimization objective function are calculated respectively to form a target matrix, and an optimization sub-objective function with a compensation range constraint is constructed based on the target gradient vector and the target matrix; The negative direction of the target gradient vector is used as the initial search direction, and the optimization sub-objective function is solved in combination with the target matrix to obtain the search direction vector of the compensation parameter; the current compensation parameter is updated along the search direction vector and then substituted into the compensation optimization objective function to calculate the actual decrease amount of the objective function value; the search direction vector is substituted into the optimization sub-objective function to calculate the predicted decrease amount of the objective function value; calculate the ratio of the actual decrease amount to the predicted decrease amount; When the ratio is greater than the preset acceptance threshold, the position compensation parameter and the torque compensation parameter are updated based on the search direction vector, and the adjustable ranges of the position compensation parameter and the torque compensation parameter are adjusted simultaneously according to the ratio; when the ratio is less than the preset acceptance threshold, the position compensation parameter and the torque compensation parameter are kept unchanged, and the adjustable ranges of the position compensation parameter and the torque compensation parameter are reduced simultaneously. Repeat the iteration until the change amounts of the position compensation parameter and the torque compensation parameter are less than the convergence threshold to obtain the final position compensation parameter and torque compensation parameter.

[0063] Exemplarily, first, obtain the dynamic risk coefficient of the assembly process and the assembly quality evaluation matrix. The dynamic risk coefficient can be evaluated based on factors such as the current assembly environment and the robot state. For example, when the robot speed is relatively high and the load is relatively large, the dynamic risk coefficient is relatively high. The assembly quality evaluation matrix is used to quantify the assembly quality. For example, it can be constructed by measuring indicators such as the gap and position deviation between the assembled parts. Assume that the dynamic risk coefficient is 0.8 and the assembly quality evaluation matrix is a 3x3 matrix, where each element represents the score of an assembly quality indicator.

[0064] Then, construct the compensation optimization objective function. This objective function consists of three parts: the deviation term of the assembly quality evaluation matrix before and after compensation, the weighted adjustment term of the dynamic risk coefficient for the position compensation parameter and the torque compensation parameter, and the constraint term for the position compensation parameter and the torque compensation parameter. The deviation term is used to measure the compensation effect, the weighted adjustment term is used to balance the risk and the compensation effect, and the constraint term is used to limit the value range of the compensation parameters to ensure that the compensation parameters are within a reasonable range. For example, the objective function can be set to minimize the deviation of the assembly quality evaluation matrix before and after compensation and to constrain the position and torque compensation parameters so that they do not exceed the maximum allowable value.

[0065] Next, perform a Taylor expansion of the position compensation parameter and the torque compensation parameter at the current position to obtain the objective gradient vector of the compensation optimization objective function at the current position. The objective gradient vector represents the change direction and magnitude of the objective function value at the current position. And calculate the second-order derivatives of each term in the compensation optimization objective function to form the objective matrix. The objective matrix describes the curvature information of the objective function. Based on the objective gradient vector and the objective matrix, construct an optimization sub-objective function with compensation range constraints. The optimization sub-objective function is used to find the optimal compensation parameters near the current position.

[0066] Take the negative direction of the objective gradient vector as the initial search direction, and solve the optimization sub-objective function in combination with the objective matrix to obtain the search direction vector of the compensation parameters. The search direction vector indicates the update direction of the compensation parameters. Substitute the updated current compensation parameters along the search direction vector into the compensation optimization objective function to calculate the actual decrease in the objective function value. Substitute the search direction vector into the optimization sub-objective function to calculate the predicted decrease in the objective function value. Calculate the ratio of the actual decrease to the predicted decrease.

[0067] If the ratio of the actual decrease to the predicted decrease is greater than the preset acceptance threshold, it indicates that the current search direction is effective. Based on the search direction vector, update the position compensation parameter and the torque compensation parameter, and adjust the adjustable ranges of the position compensation parameter and the torque compensation parameter according to the ratio. For example, if the ratio is large, it means that the current search direction has a good effect, and the adjustable range can be appropriately increased to speed up the search. If the ratio of the actual decrease to the predicted decrease is less than the preset acceptance threshold, it indicates that the current search direction is ineffective. Keep the position compensation parameter and the torque compensation parameter unchanged, and at the same time reduce the adjustable ranges of the position compensation parameter and the torque compensation parameter to perform a more refined search. Repeat the iteration until the change amounts of the position compensation parameter and the torque compensation parameter are less than the convergence threshold to obtain the final position compensation parameter and torque compensation parameter. For example, when the change amounts of the position and torque compensation parameters are less than 0.001, the algorithm is considered to converge.

[0068] In this embodiment, by optimizing the compensation parameters, the assembly error can be effectively reduced and the assembly accuracy can be improved. Through the weighted adjustment of the dynamic risk coefficient, the assembly risk can be effectively controlled, and the assembly failure caused by too large compensation parameters can be avoided. Through the iterative optimization algorithm, the optimal compensation parameters can be quickly solved to improve the assembly efficiency.

[0069] Figure 2 The following is a schematic structural diagram of an intelligent monitoring system for assembly status based on AI according to an embodiment of the present invention, as Figure 2 shown, the system includes: A first unit, configured to obtain the assembly drawing and assembly process parameters of the target product, extract the assembly reference feature points and assembly accuracy thresholds based on the assembly drawing, extract the assembly process information and torque control thresholds based on the assembly process parameters, construct an assembly knowledge graph with the assembly reference feature points, assembly accuracy thresholds, assembly process information and torque control thresholds, and at the same time collect multi-angle real-time assembly process images of the target product through multiple cameras on the production line, register and map the multi-angle real-time assembly process images with the assembly reference feature points, extract the assembly node coordinates and assembly torque data in the real-time assembly state, and generate an assembly digital twin model; A second unit, configured to input the assembly digital twin model and the assembly knowledge graph into a hybrid neural network. The hybrid neural network constructs a position constraint matrix according to the assembly reference feature points, constructs a torque constraint matrix according to the torque control threshold, performs deviation analysis on the assembly node coordinates and the assembly accuracy threshold based on the position constraint matrix to obtain the optimal assembly path, performs deviation analysis on the assembly torque data and the torque control threshold based on the torque constraint matrix to obtain the torque control parameter, and calculates the optimal assembly path and the torque control parameter according to the assembly process information to obtain an assembly quality evaluation matrix; A third unit is used to extract fault features and perform pattern recognition based on the assembly quality assessment matrix to obtain the fault type, fault location, and fault occurrence probability. When the fault occurrence probability exceeds a preset threshold, a fault scenario library containing multi-level fault scenarios is established according to the assembly process information and the fault type. The dynamic risk coefficient of the fault location is calculated based on the fault scenario library and the assembly torque data. A compensation optimization objective function is constructed based on the dynamic risk coefficient and the assembly quality assessment matrix. The position compensation parameter and the torque compensation parameter are calculated using the compensation optimization objective function. The position compensation parameter and the torque compensation parameter are converted into an execution instruction sequence according to the assembly process information. The assembly execution mechanism is controlled to make adjustments according to the priority order of the execution instruction sequence, and the adjusted data is fed back to the assembly digital twin model to dynamically update the assembly quality assessment matrix, realizing the closed-loop optimization control of the assembly process.

[0070] In the third aspect of the embodiments of the present invention, a kind of electronic device is provided, including: a processor; a memory for storing instructions executable by the processor; wherein, the processor is configured to call the instructions stored in the memory to execute the method described above.

[0071] In the fourth aspect of the embodiments of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the method described above is implemented.

[0072] The present invention can be a method, a device, a system, and / or a computer program product. The computer program product can include a computer-readable storage medium on which computer-readable program instructions for executing various aspects of the present invention are uploaded.

[0073] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An AI-based intelligent monitoring method for assembly status, characterized in that: include: Obtain assembly drawings and assembly process parameters of the target product, extract assembly reference feature points and assembly accuracy thresholds based on the assembly drawings, extract assembly process information and torque control thresholds based on the assembly process parameters, construct the assembly reference feature points, assembly accuracy thresholds, assembly process information and torque control thresholds into an assembly knowledge graph, and simultaneously collect multi-angle real-time assembly process images of the target product through multiple cameras on the production line, align and map the multi-angle real-time assembly process images with the assembly reference feature points, extract assembly node coordinates and assembly torque data in the real-time assembly state, and generate an assembly digital twin model; The assembly digital twin model and the assembly knowledge graph are input into a hybrid neural network, the hybrid neural network constructs a position constraint matrix according to the assembly reference feature points, constructs a torque constraint matrix according to the torque control threshold, performs deviation analysis on the assembly node coordinates and the assembly accuracy threshold based on the position constraint matrix to obtain an optimal assembly path, performs deviation analysis on the assembly torque data and the torque control threshold based on the torque constraint matrix to obtain a torque control parameter, and calculates the optimal assembly path and the torque control parameter according to the assembly process information to obtain an assembly quality evaluation matrix; Fault feature extraction and pattern recognition are performed based on the assembly quality assessment matrix to obtain the fault type, fault location and fault probability. When the fault probability exceeds a preset threshold, a fault scenario library containing multi-level fault scenarios is established according to the assembly process information and the fault type. The dynamic risk coefficient of the fault location is calculated based on the fault scenario library and the assembly torque data. A compensation optimization objective function is constructed based on the dynamic risk coefficient and the assembly quality assessment matrix. The position compensation parameters and torque compensation parameters are calculated using the compensation optimization objective function. The position compensation parameters and torque compensation parameters are converted into an execution instruction sequence according to the assembly process information. The assembly actuator is controlled to be adjusted according to the priority order of the execution instruction sequence, and the adjusted data is fed back to the assembly digital twin model. The assembly quality assessment matrix is ​​dynamically updated to achieve closed-loop optimization control of the assembly process.

2. The method according to claim 1, characterized in that Collecting multi-angle real-time assembly process images of target products through multiple cameras on the production line, registering and mapping the multi-angle real-time assembly process images with the assembly reference feature points, extracting assembly node coordinates and assembly torque data in the real-time assembly state, and generating an assembly digital twin model includes: Collecting multi-angle real-time assembly process images of target products through multiple cameras on the production line, preprocessing the multi-angle real-time assembly process images to obtain preprocessed images, extracting local feature description information of the preprocessed images and performing feature matching, calculating relative position and posture parameters between multiple cameras based on the matching results, and constructing a multi-view spatial constraint model according to the relative position and posture parameters; Projecting the pre-acquired assembly reference feature points to the image space of the pre-processed image to obtain an assembly initial point, performing real-time tracking on the assembly initial point to obtain a tracking feature point, establishing a motion prediction model according to assembly process information, inputting the tracking feature point into the motion prediction model to obtain a predicted feature point, and using the multi-view spatial constraint model to spatially reconstruct the predicted feature point to obtain assembly node coordinates; Collecting torque sensor data in the assembly process as assembly torque data, aligning the assembly node coordinates with the assembly torque data in time sequence to obtain assembly state data, extracting motion information and force information from the assembly state data, respectively, constructing a motion state vector according to the motion information, constructing a force state vector according to the force information, and combining the motion state vector and the force state vector to obtain a state observation vector; Determine the state observation weight according to the assembly process information and establish the state prediction equation, filter and optimize the state observation vector to obtain real-time assembly state information, obtain standard assembly parameters based on the assembly process information, and compare the real-time assembly state information with the standard assembly parameters to obtain assembly deviation data; An assembly quality assessment result is generated according to the assembly deviation data, and the assembly quality assessment result, real-time assembly status information and assembly process information are fused to generate an assembly digital twin model including assembly node coordinates and assembly torque data.

3. The method according to claim 1, characterized in that Inputting the assembly digital twin model and the assembly knowledge graph into a hybrid neural network, wherein the hybrid neural network constructs a position constraint matrix according to assembly reference feature points, and constructs a torque constraint matrix according to a torque control threshold, including: Inputting the assembly digital twin model and the assembly knowledge graph into a hybrid neural network, wherein the hybrid neural network includes a graph attention neural network module, a long short-term memory neural network module, and a recurrent neural network module, wherein the assembly digital twin model includes a coordinate sequence and a moment data sequence of assembly reference feature points, and the assembly knowledge graph includes an assembly process topological relationship; Extracting process node features and edge connection features in the assembly knowledge graph, and constructing a process topology graph including a vertex set and an edge set, wherein the vertex set represents the assembly process nodes, and the edge set represents the connection relationship between the process nodes; Input the process topology graph into the graph attention neural network module, calculate the multi-head attention weight coefficients between process nodes, perform attention weighted calculation on the process node features through a multi-layer feature transformation network, and output an assembly process feature vector representing the constraint relationship between processes; Extract coordinate data of adjacent feature points from the assembly reference feature point coordinate sequence, calculate position deviation values ​​between the adjacent feature points, combine the position deviation values ​​with the assembly process feature vector in a time series order into a position feature sequence, input the position feature sequence into the long short-term memory neural network module, process the position feature sequence through the gating unit of the long short-term memory neural network module, and output a position constraint matrix; Extracting the moment data in a continuous time window from the moment data sequence, calculating the moment change rate between adjacent assembly process nodes, and combining the moment change rate with the assembly process feature vector into a moment feature sequence; The torque feature sequence is input into the recurrent neural network module, and the torque change trend between the assembly process nodes is calculated based on the hidden layer state of the recurrent neural network module to generate a torque control threshold, and the ratio of the torque change rate between adjacent assembly process nodes to the torque control threshold is calculated to obtain the torque constraint coefficient between the process nodes; A torque constraint matrix is ​​constructed based on the number of process nodes. When there is a direct constraint relationship between process nodes, the corresponding matrix element value is set to the torque constraint coefficient. When there is no direct constraint relationship between process nodes, the corresponding matrix element value is set to zero. When they are the same process nodes, the corresponding matrix element value is set to one, and finally a torque constraint matrix is ​​obtained.

4. The method according to claim 1, characterized in that: Based on the position constraint matrix, deviation analysis is performed on the assembly node coordinates and the assembly accuracy threshold to obtain the optimal assembly path; based on the torque constraint matrix, deviation analysis is performed on the assembly torque data and the torque control threshold to obtain the torque control parameter; and according to the assembly process information, the optimal assembly path and the torque control parameter are calculated to obtain the assembly quality evaluation matrix, including: Acquire a position constraint matrix, wherein the position constraint matrix includes three-dimensional coordinate information of assembly nodes and assembly accuracy threshold information, extract coordinate information of adjacent assembly nodes, calculate the spatial distance between nodes to obtain actual position data, perform difference operation between the actual position data and the corresponding assembly accuracy threshold to obtain position deviation, and construct a position constraint relationship between nodes based on the position deviation; The position constraint relationship is normalized to obtain a standardized position constraint coefficient, the standardized position constraint coefficient is used as a path evaluation index, a heuristic search strategy is adopted to calculate the cumulative position constraint coefficient of each assembly path, and the assembly path with the smallest cumulative position constraint coefficient is selected as the optimal assembly path; Obtain a torque constraint matrix, wherein the torque constraint matrix includes torque data and torque control thresholds of the assembly process, extract torque data at adjacent sampling moments, calculate the torque change to obtain actual torque data, perform a difference operation between the actual torque data and the corresponding torque control threshold to obtain a torque deviation, and construct a time-series torque constraint relationship based on the torque deviation; The time sequence torque constraint relationship is normalized to obtain a standardized torque constraint coefficient, initial values ​​of a proportional parameter, an integral parameter and a differential parameter are set based on the standardized torque constraint coefficient, and under the constraints of a torque change rate limit and a system stability index, the proportional parameter, the integral parameter and the differential parameter are updated in sequence by comparing the parameter adjustment amounts of adjacent iteration cycles, and a final torque control parameter is obtained when the parameter adjustment amount is less than a preset adjustment threshold; Determine the connection relationship of the assembly nodes according to the node access sequence of the optimal assembly path, extract the position constraint coefficients and moment constraint coefficients corresponding to the adjacent assembly nodes, and use adaptive weight coefficients to weightedly combine the position constraint coefficients and the moment constraint coefficients to obtain a comprehensive score, wherein the adaptive weight coefficients are dynamically adjusted according to the importance of the position constraint and the moment constraint; The matrix dimension is determined according to the total number of assembly nodes, an assembly quality assessment matrix is ​​constructed, the comprehensive scores between adjacent assembly nodes are filled into the row and column positions of the corresponding nodes in the assembly quality assessment matrix, the diagonal elements of the assembly quality assessment matrix are set to unit values ​​to represent the constraints of the nodes themselves, the matrix elements corresponding to non-adjacent nodes are set to zero values ​​to represent unconstrained relationships, and the filled assembly quality assessment matrix is ​​normalized to obtain a standardized assembly quality assessment matrix.

5. The method according to claim 1, characterized in that Based on the assembly quality assessment matrix, fault feature extraction and pattern recognition are performed to obtain the fault type, fault location and fault probability. When the fault probability exceeds a preset threshold, a fault scenario library containing multi-level fault scenarios is established according to the assembly process information and the fault type. The dynamic risk coefficient of the fault location is calculated based on the fault scenario library and the assembly torque data, including: The assembly quality assessment matrix is ​​decomposed into multiple dimensions, and a deep neural network containing multiple parallel convolution branches is constructed. Each convolution branch uses convolution kernels of different sizes for feature extraction, and the outputs of each convolution branch are fused to obtain a feature matrix. The feature matrix is ​​weighted in terms of time and space dimensions using a multi-head attention mechanism, and the fault feature vector and process association matrix are obtained by combining the assembly process timing information. A multi-task learning framework is constructed based on the fault feature vector and the process association matrix, wherein the fault type features and the fault type probability distribution are extracted through the fault classification branch, the fault location coordinates are determined through the position regression branch, the fault type features, the fault type probability distribution, the fault location coordinates and the process association matrix are input into a dynamic Bayesian network, and the fault occurrence probability of each process node is calculated; When the fault occurrence probability of any process node exceeds a preset threshold, the process association matrix is ​​converted into a directed acyclic graph, and the process node corresponding to the fault location coordinate is taken as the starting point. The propagation influence coefficient between adjacent process nodes is calculated in combination with the fault type characteristics and the fault type probability distribution. The propagation influence coefficient represents the probability of fault diffusion between processes. A multi-level fault scenario library is constructed based on the topological structure of the directed acyclic graph and the propagation influence coefficient. Each scenario node includes a process identifier, a fault type characteristic, a fault occurrence probability and a propagation influence coefficient. The scene node relationship in the fault scenario library is converted into a state transition probability matrix, and a Markov decision process model is constructed in combination with real-time assembly torque data. A random disturbance sample set is generated based on the state transition probability matrix using Latin hypercube sampling. The random disturbance sample set and the assembly torque data are input into the Markov decision process model to obtain a process state transition sequence. In combination with the process state transition sequence, the propagation influence coefficient and the probability of failure, the dynamic risk coefficient of the process node corresponding to the fault location coordinate is calculated through a dynamic programming algorithm.

6. The method according to claim 5, characterized in that include: Based on the state transition probability matrix, Latin hypercube sampling is used to generate a random disturbance sample set, the random disturbance sample set and the assembly moment data are input into the Markov decision process model to obtain a process state transition sequence, and the dynamic risk coefficient of the process node corresponding to the fault position coordinate is calculated by a dynamic programming algorithm in combination with the process state transition sequence, the propagation influence coefficient and the probability of failure. The calculation includes: The probability value interval in the state transfer probability matrix is ​​evenly divided into a plurality of sub-intervals, a plurality of disturbance values ​​are randomly extracted in the plurality of sub-intervals and randomly arranged to form a disturbance sequence, and the probability values ​​in the state transfer probability matrix are disturbed by using the disturbance sequence to generate a plurality of groups of disturbed state transfer probability matrices; Input multiple groups of disturbed state transition probability matrices and assembly torque data into the Markov decision process model, determine the system state at the next moment according to the state transition equation, and construct a strategy function based on the immediate reward function and the state value function, determine the optimal action through the strategy function, and iterate based on the optimal action to obtain the process state transition sequence; Acquire the propagation influence coefficient and the probability of failure, input the process state transfer sequence, the propagation influence coefficient and the probability of failure into the risk value function, wherein the risk value function is used to combine the probability of failure in the current state, the propagation influence coefficient in the current state and the maximum risk value in the next state; construct the Bellman equation based on the risk value function, perform reverse recursion and solve the Bellman equation through a dynamic programming algorithm, and obtain the dynamic risk coefficient of the process node corresponding to the fault location coordinate.

7. The method according to claim 1, characterized in that Constructing a compensation optimization objective function based on the dynamic risk coefficient and the assembly quality assessment matrix, and calculating position compensation parameters and torque compensation parameters using the compensation optimization objective function includes: Constructing a compensation optimization objective function based on the dynamic risk coefficient and the assembly quality assessment matrix, the compensation optimization objective function comprising a deviation term of the assembly quality assessment matrix before and after compensation, a weighted adjustment term of the dynamic risk coefficient on the position compensation parameter and the moment compensation parameter, and a constraint term on the position compensation parameter and the moment compensation parameter; Taylor expansion is performed on the position compensation parameter and the torque compensation parameter at the current position to obtain the target gradient vector of the compensation optimization objective function at the current position; the second-order derivatives of each item in the compensation optimization objective function are respectively calculated to form a target matrix, and an optimization sub-objective function with a compensation range constraint is constructed based on the target gradient vector and the target matrix; The negative direction of the target gradient vector is used as the initial search direction, and the optimization sub-objective function is solved in combination with the target matrix to obtain the search direction vector of the compensation parameter; the current compensation parameter is updated along the search direction vector and then inserted into the compensation optimization objective function to calculate the actual decrease of the objective function value; the search direction vector is substituted into the optimization sub-objective function to calculate the predicted decrease of the objective function value; and the ratio of the actual decrease to the predicted decrease is calculated; When the ratio is greater than a preset acceptance threshold, the position compensation parameters and the torque compensation parameters are updated based on the search direction vector, and the adjustable ranges of the position compensation parameters and the torque compensation parameters are adjusted simultaneously according to the ratio; when the ratio is less than the preset acceptance threshold, the position compensation parameters and the torque compensation parameters are kept unchanged, and the adjustable ranges of the position compensation parameters and the torque compensation parameters are reduced, and the iteration is repeated until the changes in the position compensation parameters and the torque compensation parameters are less than the convergence threshold, so as to obtain the final position compensation parameters and the torque compensation parameters.

8. An AI-based assembly status intelligent monitoring system, used to implement the method described in any one of claims 1 to 7, characterized in that: include: The first unit is used to obtain the assembly drawings and assembly process parameters of the target product, extract assembly reference feature points and assembly accuracy thresholds based on the assembly drawings, extract assembly process information and torque control thresholds based on the assembly process parameters, construct the assembly reference feature points, assembly accuracy thresholds, assembly process information and torque control thresholds into an assembly knowledge graph, and simultaneously collect multi-angle real-time assembly process images of the target product through multiple cameras on the production line, align and map the multi-angle real-time assembly process images with the assembly reference feature points, extract assembly node coordinates and assembly torque data in the real-time assembly state, and generate an assembly digital twin model; The second unit is used to input the assembly digital twin model and the assembly knowledge graph into a hybrid neural network, the hybrid neural network constructs a position constraint matrix according to the assembly reference feature points, constructs a torque constraint matrix according to the torque control threshold, performs deviation analysis on the assembly node coordinates and the assembly accuracy threshold based on the position constraint matrix to obtain an optimal assembly path, performs deviation analysis on the assembly torque data and the torque control threshold based on the torque constraint matrix to obtain a torque control parameter, and calculates the optimal assembly path and the torque control parameter according to the assembly process information to obtain an assembly quality evaluation matrix; The third unit is used to perform fault feature extraction and pattern recognition based on the assembly quality assessment matrix to obtain the fault type, fault location and fault probability. When the fault probability exceeds a preset threshold, a fault scenario library containing multi-level fault scenarios is established according to the assembly process information and the fault type, the dynamic risk coefficient of the fault location is calculated based on the fault scenario library and the assembly torque data, a compensation optimization objective function is constructed based on the dynamic risk coefficient and the assembly quality assessment matrix, the position compensation parameters and the torque compensation parameters are calculated using the compensation optimization objective function, the position compensation parameters and the torque compensation parameters are converted into an execution instruction sequence according to the assembly process information, the assembly actuator is controlled to be adjusted according to the priority order of the execution instruction sequence, the adjusted data is fed back to the assembly digital twin model, the assembly quality assessment matrix is ​​dynamically updated, and closed-loop optimization control of the assembly process is realized.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images