Visual guidance method and system for intelligent auxiliary assembly

Abstract

The invention relates to the technical field of intelligent equipment, and provides a visual guidance method and system for intelligent auxiliary assembly, and the method comprises the steps: collecting an original image sequence, depth coordinate data and three-axis torque data of a current assembly object, and generating spatial configuration feature information; and carrying out stress state evaluation analysis on the spatial configuration feature information by utilizing the three-axis torque data to obtain a strategy candidate set and a plurality of assembly state transfer path sets, and carrying out strategy matching on all the assembly state transfer path sets by utilizing the strategy candidate set to obtain an assembly guide strategy. Through the strategy candidate set and the assembly state transfer path set, a high-adaptability assembly guide strategy is generated, the defects of the prior art in the aspects of coping with assembly step changes, error deviation and real-time regulation and control are overcome, and the defect that when assembly steps change and assembly deviation or error occurs to an assembly object, the assembly accuracy is improved. The problems that in the prior art, an adaptive dynamic guiding strategy is lacked, and an assembly scheme and a guiding mode cannot be adjusted in real time are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent devices, and particularly to a visual guidance method and system for intelligent auxiliary assembly. BACKGROUND

[0002] With the continuous development of intelligent manufacturing, assembly operations in industrial production are increasingly complex and precise. Traditional manual assembly mode gradually fails to meet the requirements of efficient, precise and flexible production. Therefore, intelligent assembly technology emerges as the times require and is widely used in high-precision manufacturing fields such as electronics, automobiles and aerospace. Intelligent assembly system gradually realizes real-time monitoring, analysis and optimization of the assembly process by combining robots, sensors, visual technology and other frontier technologies.

[0003] In related technical means, industrial cameras and depth sensors installed on workstations are used to obtain image information and spatial position information of assembly objects, and real-time monitoring and analysis of the assembly process are performed through image recognition and visual algorithms. The position, direction and state of the assembly components can be judged in real time, and guidance information such as next operation prompt, component position and installation direction is provided to the operator through the display device, which effectively improves the assembly efficiency and reduces human operation errors.

[0004] For the above technical solution, although the existing technology can realize basic monitoring and guidance of the assembly process by using visual guidance, reduce operation errors and improve efficiency, when the assembly steps change or the assembly object has assembly deviation or error, the existing technology lacks adaptive dynamic guidance strategy and cannot adjust the assembly scheme and guidance mode in real time. SUMMARY

[0005] In order to improve the problem that the existing technology lacks adaptive dynamic guidance strategy and cannot adjust the assembly scheme and guidance mode in real time when the assembly steps change or the assembly object has assembly deviation or error, the present application provides a visual guidance method and system for intelligent auxiliary assembly.

[0006] The present invention provides a visual guidance method for intelligent assisted assembly, comprising: collecting an original image sequence, depth coordinate data, and three-axis torque data of a current assembly object on an assembly station to construct a set of key assembly areas; generating spatial configuration feature information based on the set of key assembly areas, performing stress state evaluation and analysis on the spatial configuration feature information using the three-axis torque data, and obtaining stability index information and an assembly strain dynamic change map; performing feature extraction on the stability index information to obtain a stability feature vector, performing contact pattern recognition on the assembly strain dynamic change map using the stability feature vector, and obtaining a contact area state sequence; inputting the contact area state sequence into a preset assembly anomaly recognition model to obtain an assembly anomaly risk assessment result, performing a hierarchical analysis on the assembly anomaly risk assessment result, and obtaining risk level information and a strategy candidate set; obtaining dynamic evolution data of the spatial configuration feature information according to the risk level information, classifying the dynamic evolution data, and obtaining a plurality of assembly state transition path sets; performing strategy matching on all the assembly state transition path sets using the strategy candidate set, and obtaining an assembly guidance strategy.

[0007] As a preferred solution, the step of collecting the original image sequence, depth coordinate data and three-axis torque data of the current assembly object on the assembly station to construct a set of key assembly areas includes: using an industrial camera, a depth perception module and a force sensor installed at the assembly station to synchronously collect the original image sequence, depth coordinate data and three-axis torque data of the contact surface force applied to the current assembly object; calibrating and registering the original image sequence and the depth coordinate data, combining the calibrated and registered original image sequence and depth coordinate data with the station reference coordinate system to generate a three-dimensional visual model and real-time posture information of the assembly component, fusing the three-dimensional visual model of the assembly component with the real-time posture information to obtain a posture mapping field; performing time series analysis on the three-axis torque data to obtain instantaneous stress fluctuation characteristics, jointly calculating the posture mapping field and the instantaneous stress fluctuation characteristics to generate assembly action influence domain information, and using the assembly action influence domain information to dynamically annotate the three-dimensional visual model to obtain a set of key assembly areas.

[0008] As a preferred solution, the step of generating spatial configuration feature information based on the set of key assembly regions, performing stress state evaluation analysis on the spatial configuration feature information by using the three-axis moment data to obtain stability index information and assembly strain dynamic change map, comprises: constructing an assembly configuration contour surface based on the set of key assembly regions, and performing topology modeling on the assembly configuration contour surface combined with three-dimensional pose information to obtain spatial configuration feature information; wherein the three-dimensional pose information refers to position coordinate information and orientation attitude information of the current assembly object in the work station coordinate system; performing multi-region loading simulation analysis on the spatial configuration feature information by using the three-axis moment data to obtain stress uniformity distribution map and contact stiffness change characteristics of each configuration region, and cross-matching the stress uniformity distribution map and the contact stiffness change characteristics to obtain a high-risk contact sub-region and a low-stability feature point set; matching the real-time contact pressure distribution of the high-risk contact sub-region in the current assembly process with the stress mode of the same contact region in the historical task database, calculating the deviation degree score of each sub-region, and constructing stability index information according to the deviation degree score; extracting the three-dimensional displacement time series of the low-stability feature point set in the whole assembly process, calculating the strain rate change trend and the contact area change trend of the three-dimensional displacement time series, and applying the strain rate change trend and the contact area change trend to construct contact region change information; and constructing an assembly strain dynamic change map based on the contact region change information and the stability index information.

[0009] As a preferred solution, the step of performing multi-region loading simulation analysis on the spatial configuration feature information by using the three-axis moment data to obtain stress uniformity distribution map and contact stiffness change characteristics of each configuration region, and cross-matching the stress uniformity distribution map and the contact stiffness change characteristics to obtain a high-risk contact sub-region and a low-stability feature point set, comprises: dividing each key assembly region in the spatial configuration feature information according to the three-axis moment data to obtain a plurality of local regions, and performing multi-physical field coupling simulation on each local region to obtain loading simulation results; calculating the stress distribution of each local region according to the loading simulation results to generate a stress uniformity distribution map, extracting features from the stress uniformity distribution map, identifying the contact stiffness change characteristics of each assembly region to obtain a contact stiffness change map; cross-matching the stress uniformity distribution map and the contact stiffness change characteristics to analyze the correlation of stress and stiffness change of the local region, and identifying the existing high-risk contact sub-region and low-stability feature point set.

[0010] As a preferred solution, the step of performing feature extraction on the stability index information to obtain a stability feature vector, and performing contact mode recognition on the assembly strain dynamic change map using the stability feature vector to obtain a contact area state sequence, comprises: performing principal component analysis and dynamic time warping on the stability index information to extract a steady-state feature distribution and a change rate, and constructing a stability feature vector using the steady-state feature distribution and the change rate; performing interval segmentation and dynamic clustering on the assembly strain dynamic change map to obtain a stress response mode, embedding the stability feature vector and the stress response mode in a unified feature space to obtain a fusion stability feature vector; identifying a key contact event according to the fusion stability feature vector, and constructing a time sequence contact map using the key contact event, extracting a contact point state transition path of the time sequence contact map, and performing structural coding processing on the contact point state transition path to obtain a complete contact area state sequence.

[0011] As a preferred solution, the step of inputting the contact area state sequence into a preset assembly abnormality recognition model to obtain an assembly abnormality risk assessment result, and performing hierarchical analysis on the assembly abnormality risk assessment result to obtain risk level information and a strategy candidate set, comprises: encoding and vectorizing the contact area state sequence according to a time window and a state node to generate a dynamic contact behavior sequence, inputting the dynamic contact behavior sequence into a preset assembly abnormality recognition model to output an assembly stability deviation degree and an abnormal mode label; generating an assembly abnormality risk assessment result according to the assembly stability deviation degree and the abnormal mode label, performing similarity comparison on the assembly abnormality risk assessment result using a preset assembly database to obtain risk level information, and matching a preset strategy template library based on the risk level information to obtain a strategy candidate set.

[0012] As a preferred solution, the step of obtaining the dynamic evolution data of the spatial configuration feature information according to the risk level information, classifying the dynamic evolution data to obtain a plurality of assembly state transition path sets, and matching all the assembly state transition path sets with the strategy candidate set to obtain the assembly guidance strategy comprises: selecting a corresponding key spatial configuration region in the spatial configuration feature information based on the risk level information, and extracting a spatial feature evolution trajectory of the key spatial configuration region changing over time to obtain dynamic evolution data, performing multi-dimensional feature mapping and path structure analysis on the dynamic evolution data to obtain an analysis result, and dividing the analysis result into a plurality of assembly state transition units; clustering all the assembly state transition units into assembly state transition path sets according to time sequence order and evolution trend, matching each strategy in the strategy candidate set with each path in the assembly state transition path set to score, and constructing a strategy-path adaptation matrix; performing intervention effectiveness simulation prediction and response time evaluation on each strategy-path combination in the strategy-path adaptation matrix to obtain a test evaluation result, and selecting an optimal strategy-path combination based on the test evaluation result to generate an assembly guidance strategy.

[0013] The application also provides a visual guidance system for intelligent auxiliary assembly, comprising: an acquisition module configured to acquire an original image sequence, depth coordinate data, and three-axis torque data of a current assembly object at an assembly station to construct a key assembly region set; an analysis module configured to generate spatial configuration feature information based on the key assembly region set, and perform stress state evaluation analysis on the spatial configuration feature information using the three-axis torque data to obtain stability index information and an assembly strain dynamic change map; an extraction module configured to extract features from the stability index information to obtain a stability feature vector, and perform contact mode recognition on the assembly strain dynamic change map using the stability feature vector to obtain a contact region state sequence; an input module configured to input the contact region state sequence into a preset assembly abnormality recognition model to obtain an assembly abnormality risk evaluation result, and perform hierarchical analysis on the assembly abnormality risk evaluation result to obtain risk level information and a strategy candidate set; and a matching module configured to obtain dynamic evolution data of the spatial configuration feature information according to the risk level information, classify the dynamic evolution data to obtain a plurality of assembly state transition path sets, and match all the assembly state transition path sets with the strategy candidate set to obtain an assembly guidance strategy.

[0014] Compared with the prior art, the application has the following beneficial effects: high flexibility and accurate guidance. By combining multi-modal perception of the original image sequence, depth coordinate data and three-axis moment data, a set of key assembly regions is constructed, and stability index information and assembly strain dynamic change atlas are generated through spatial configuration feature information and multi-physical coupling simulation analysis; then, feature extraction and pattern recognition are performed on the stability index information, a contact region state sequence is generated based on the contact mode, and abnormal states generated in the assembly process are dynamically monitored; an assembly abnormality recognition model based on deep learning is used to perform hierarchical analysis on the assembly abnormality, and a high-adaptation assembly guidance strategy is generated by combining dynamic evolution data and a strategy candidate set, which can effectively solve the deficiencies of the prior art in dealing with assembly step changes, error deviations and system real-time regulation, greatly improving the assembly efficiency and the self-adaptability and intelligent level of the assembly process, and solving the problem that the prior art lacks adaptive dynamic guidance strategy when the assembly step changes, the assembly object has assembly deviation or error, and the assembly scheme and guidance mode cannot be adjusted in real time. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0016] The structures, proportions, sizes, etc. shown in the drawings of the present specification are only used to cooperate with the content disclosed in the present specification, to enable those skilled in the art to understand and read, and are not used to limit the defined conditions under which the present application can be implemented, so they do not have technical significance. Any modification of structure, change of proportion relationship or adjustment of size, which does not affect the effects that can be produced by the present application and the purposes that can be achieved, should still fall within the scope of the technical content disclosed by the present application.

[0017] Figure 1 is a flowchart of the visual guidance method of intelligent auxiliary assembly provided by the embodiments of the present application; Figure 2 is a structural schematic block diagram of the visual guidance system of intelligent auxiliary assembly provided by the embodiments of the present application.

[0018] Explanation of reference numerals: 10, visual guidance system of intelligent auxiliary assembly; 11, acquisition module; 12, analysis module; 13, extraction module; 14, input module; 15, matching module. DETAILED DESCRIPTION

[0019] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0020] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0021] It should also be understood that the terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0022] It should be further understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0023] The technical solution of the present invention will be further described below with reference to the accompanying drawings and through specific implementation methods.

[0024] Example 1: like Figure 1 As shown, the present application provides a visual guidance method for intelligent assisted assembly, including steps S100 to S500.

[0025] Step S100: collecting original image sequences, depth coordinate data, and three-axis torque data of the current assembly object at the assembly station to construct a set of key assembly areas.

[0026] In this step, the original image sequence of the current assembly object is acquired by the industrial camera installed on the assembly station, the depth coordinate data of the assembly object is acquired by the depth perception module, and the three-axis moment data of the current assembly object applied on the contact surface is collected by the force sensor. By calibrating and registering the original image sequence and the depth coordinate data, the calibrated image data and the depth coordinate data are combined with the station reference coordinate system to generate a three-dimensional visual model and real-time pose information of the assembly component. Then, the three-dimensional visual model and real-time pose information of the assembly component are jointly calculated to obtain a pose mapping field. On this basis, by performing time series analysis and dynamic region labeling on the pose mapping field and the three-axis moment data, a set of key assembly regions meeting the requirements is finally identified. Specifically, when calibrating and registering the image data and the depth coordinate data, a camera calibration technology based on a checkerboard calibration method is used to perform coordinate mapping and geometric alignment of the sampling space of the industrial camera and the depth sensor. The generation of the pose mapping field uses a multi-resolution pyramid model to perform layered calculation on the three-dimensional visual model, and obtains the pose state of each spatial point according to the gradient distribution of the depth coordinates and the pose information. In the time series analysis, the instantaneous stress fluctuation characteristics of the three-axis moment data are extracted by analyzing the frequency domain characteristics of the three-axis moment data, and the regions with obvious stress peaks or non-uniform distribution are identified as key assembly regions by combining the dynamic region labeling algorithm of the assembly component.

[0027] For example, in actual operation, when it is detected that the edge of a certain assembly component appears in a tilted state through the image sequence collected by the industrial camera, the depth perception module records the depth position information of the object in space. At the same time, the force sensor detects that the three-dimensional moment of this assembly component at the contact point changes (for example, the instantaneous torque increases to [M x =2.0, M y =1.8, M z =2.3] N·m), and the joint pose mapping field can quickly identify this place as an important contact key region in the assembly step, and label it as part of the set of key assembly regions.

[0028] Step S200, generate spatial configuration feature information based on the set of key assembly regions, and perform stress state evaluation analysis on the spatial configuration feature information by using the three-axis moment data to obtain stability index information and assembly strain dynamic change map.

[0029] In this step, the spatial configuration feature information is generated by constructing the assembly configuration profile surface for the set of key assembly regions, combining real-time pose information for topological modeling; and the simulation is performed for each key region using a load analysis algorithm, and the simulation results are processed using three-axis torque data to complete multi-field fusion analysis of the stress state, obtaining the stress uniformity distribution and contact stiffness variation characteristics of each key local region. Further, by analyzing the unevenness of the feature distribution between the overall regions, combined with multi-point strain analysis, the assembly strain dynamic change map reflecting the internal strain evolution relationship of the reaction region is generated. Specifically, in the topological modeling, the three-dimensional Delaunay triangulation algorithm is used to divide the assembly configuration profile surface into grids, and the boundary conditions conforming to the continuum mechanics model are set during the simulation analysis, and the stress distribution is simulated by using finite element software (such as ANSYS). The three-axis torque data is used as the load input in the multi-field fusion analysis, and the information of the local region pressure concentration point is obtained by analyzing the weight of the time series and spatial distribution of the torque, and the mechanical state distribution and deformation characteristics of the overall configuration are extracted.

[0030] For example, in a certain assembly station, when the pressure of the connection part of the key assembly region is suddenly increased (three-axis torque [M x = 5.5, M y = 3.6, M z = 4.7] N·m), the finite element simulation shows that the stress concentration area of the corresponding region gradually expands, and combined with the simulation results, the contact stiffness change map can further reflect the trend of the local region of the assembly component tending to instability.

[0031] Step S300, feature extraction is performed on the stability index information to obtain a stability feature vector, and the stability feature vector is used for contact mode recognition of the assembly strain dynamic change map to obtain a contact region state sequence.

[0032] In this step, the stability index information is analyzed in time and frequency domains, and the steady-state feature distribution and change rate are extracted by a principal component analysis (PCA) algorithm to construct a stability feature vector; the feature vector is combined with the assembly strain dynamic change map, and the stress response mode in different time windows during the assembly process is labeled by a dynamic clustering algorithm, so that the mode recognition of the contact region of the assembly component is completed, and finally a time-sequenced contact region state sequence is generated.

[0033] Specifically, the PCA algorithm is used for dimensionality reduction of high-dimensional data in the main feature extraction stage, and in this process, all input variables (such as data points of stress distribution and stiffness feature curve) are normalized, and principal components with a cumulative variance contribution rate of more than 95% are extracted. The dynamic clustering algorithm adopts the K-means++ method, and the stress fluctuation mode is classified and analyzed by setting the number of cluster center points.

[0034] For example, in a certain assembly step, the dynamic change map of assembly strain shows that the stiffness change slope of the key contact point is as high as 0.9. Combined with the setting of the stiffness slope threshold of <-0.5 in the normal assembly process, the dynamic clustering algorithm is used to identify the current state as "slip transition".

[0035] Step S400: Input the contact area state sequence into a preset assembly anomaly recognition model to obtain an assembly anomaly risk assessment result, perform hierarchical analysis on the assembly anomaly risk assessment result, and obtain risk level information and a strategy candidate set.

[0036] In this step, the contact area state sequence is used as the input feature, and anomaly pattern matching is performed through a deep learning-based anomaly recognition model combined with the historical database of the assembly process. The identified anomaly risks are graded and analyzed according to the loss probability and severity, and the corresponding strategy candidate set is matched from the built-in strategy template library for the subsequent generation of assembly guidance strategies.

[0037] Specifically, the anomaly recognition model uses a conventional convolutional neural network (CNN) model. The input state sequence is fed into a 10-frame time window. After model training, the model calculates the classification results output by the softmax layer to obtain anomaly labels, such as "assembly component position deviation" or "stress concentration occurrence." The grading analysis is based on the likelihood of an anomaly occurring and the magnitude of its impact, with a grading scale of 1 to 5.

[0038] For example, after a state sequence is input, the CNN model classifies it as "stress mutation anomaly". The historical database matching shows that the probability of occurrence of this state in the past position deviation is 70%, so it is classified as risk level 4 and outputs a set of strategy candidates such as "posture adjustment".

[0039] Step S500: Acquire dynamic evolution data of spatial configuration feature information according to risk level information, classify the dynamic evolution data to obtain several assembly state transition path sets, perform strategy matching on all assembly state transition path sets using strategy candidate sets, and obtain assembly guidance strategies.

[0040] In this step, based on the acquired risk level information, the dynamic evolution trajectory of the spatial configuration features over time is extracted. This dynamic evolution data is then classified using a time series classification algorithm to form a set of different state transition paths. Simultaneously, the candidate strategy set is combined and the degree of fit between each strategy and path is ranked by matching score, ultimately generating the optimal assembly guidance strategy. Specifically, the classification algorithm utilizes a state-of-the-art time series classification method based on dynamic time warping (DTW) to perform cluster analysis on the characteristic points in the evolution data. Each state transition path corresponds to a characteristic of an assembly step. When matching strategies, the scores are ranked based on historical success rates and expected response times.

[0041] For example, a dynamic evolution trajectory reflects that the deviation angle of the assembly component position gradually increases, which belongs to the state transition path type A; combined with the strategy candidate set including the "guide the operator to reposition" strategy applicable to the "Type A path", the assembly guidance strategy is finally generated to prompt the operator to make real-time adjustments.

[0042] In this embodiment, the original image sequence, depth coordinate data and three-axis torque data of the current assembly object on the assembly station are collected, and a key assembly area set is constructed in combination with these data; then, spatial configuration feature information is generated based on the key assembly area set, and the stress state evaluation and analysis of the spatial configuration feature information is performed using the three-axis torque data to obtain stability index information and an assembly strain dynamic change map; then, feature extraction is performed on the stability index information to obtain a stability feature vector, and the stability feature vector is used to perform contact pattern recognition on the assembly strain dynamic change map to obtain a contact area state sequence; the contact area state sequence is input into a preset assembly anomaly recognition model to obtain an assembly anomaly risk assessment result, and the assembly anomaly risk assessment result is graded and analyzed to obtain risk level information and a strategy candidate set; finally, the dynamic evolution data of the spatial configuration feature information is obtained according to the risk level information, and the dynamic evolution data is classified to obtain several assembly state transfer path sets, and the strategy candidate set is used to perform strategy matching on all assembly state transfer path sets to generate an assembly guidance strategy. To solve the problem that the existing technology cannot adjust the assembly plan and guidance method in real time, the original image sequence, depth coordinate data and three-axis torque data of the assembly object are collected in real time, and a set of key assembly areas is dynamically constructed to generate spatial configuration feature information. The stress state of the assembly is comprehensively evaluated to obtain stability index information and a dynamic change map of the assembly strain; feature extraction and contact pattern recognition technology are further used to monitor and evaluate abnormal risks in the assembly process in real time, and risk level information and strategy candidate sets are generated through hierarchical analysis. Dynamic evolution data classification and assembly state transfer path matching are achieved according to changes in the assembly environment. Finally, an assembly guidance strategy that can optimize assembly efficiency and quality is generated to ensure accuracy and flexibility in the assembly process and improve the adaptability of complex assembly tasks.

[0043] Example 2: In step S100, the steps of collecting the original image sequence, depth coordinate data, and three-axis torque data of the current assembly object at the assembly station to construct a set of key assembly areas specifically include: The industrial camera, depth perception module and force sensor installed at the assembly station are used to synchronously collect the original image sequence, depth coordinate data and three-axis torque data of the force applied on the contact surface of the current assembly object.

[0044] By configuring the hardware synchronization of the industrial camera, the depth perception module, and the force sensor, a unified timestamp is used to establish the data correspondence between the sensors. The industrial camera uses a high-speed imaging mode to continuously capture a sequence of raw images at a rate of 60 frames per second and captures the surface texture information of the assembly object through an appropriate optical lens. The depth perception module uses structured light or laser radar technology to obtain the depth coordinate data of the assembly object. This process generates a three-dimensional point cloud data of the assembly object by actively emitting a light beam and recording the time of the reflected information. The force sensor records the three-axis torque data in real time through the contact surface and stores the torque values as a time series. All data is transmitted in real time to the control system and is preliminarily calibrated in combination with the reference coordinate system of the assembly station to ensure spatial alignment and data consistency.

[0045] Specifically, during the imaging process of the industrial camera, the existing "Zhang Calibration Method" is used to calibrate the internal and external parameters of the camera, making the camera's data more accurately mapped to the three-dimensional space. The configuration of the depth perception module uses the "ICP" (Iterative Closest Point) algorithm to iteratively optimize the point cloud data to eliminate noise and errors. The three-axis torque data is obtained from the force sensor installed at the end of the contact tool, and the three-axis force values ([M x ,M y ,M z ] N·m) are mapped to the space through matrix transformation from the body coordinate system to the station reference coordinate system. The data synchronization of all sensors is verified by the system through the synchronization mechanism of the unique timestamp, ensuring the time continuity and spatial correlation of the data.

[0046] For example, in the assembly operation, the industrial camera captures a sequence of surface images of the assembly object (such as the edge profile and texture distribution of the parts), the depth module generates corresponding point cloud data, and constructs the depth information in space (such as the spatial surface shape of the parts). At the same time, the force sensor records the three-axis torque data applied at the contact point (for example, the torque values are [M x =3.2,M y =-1.5,M z =2.8] N·m). Through timestamp synchronization, the control system jointly encodes the image, depth, and torque data and verifies the spatial consistency between the data, which collectively provides high-precision raw data for subsequent generation of a set of key assembly regions.

[0047] The original image sequence and depth coordinate data are calibrated and registered, and the calibrated and registered original image sequence and depth coordinate data are combined with the station reference coordinate system to generate a three-dimensional visual model of the assembly component and real-time pose information. The three-dimensional visual model of the assembly component is fused with the real-time pose information to obtain a pose mapping field.

[0048] The image data of the industrial camera and the point cloud data of the depth perception module are jointly calibrated using a calibration registration technique to ensure spatial alignment and data accuracy. In the calibration stage, an orthogonal checkerboard is used as a calibration template to calibrate the geometric relationship between the camera and the depth sensor, and a reference marker point is used to construct a workstation reference coordinate system. The calibrated image sequence and depth coordinate data are input into a three-dimensional visual model generation module, and a joint filtering algorithm is used to fuse the image texture and depth data. Real-time pose information is estimated by combining the feature points of the front and rear frames of the image of the assembly component with the optical flow technique, and the position and attitude of the assembly component are updated in real time. Finally, the spatial configuration of the overall assembly workstation and the dynamic attitude information of the assembly object are obtained by merging the three-dimensional visual model and the real-time pose information through a pose mapping field construction algorithm.

[0049] Specifically, in the calibration process, the internal and external parameters of the camera are calibrated by the "DLT" (Direct Linear Transformation) algorithm, and the "RANSAC" (Random Sample Consensus) algorithm is used to remove abnormal points for the optimization of depth coordinate points. When generating a three-dimensional visual model, the edge texture information in the image data is superimposed with the point cloud surface data of the depth perception module in a multi-scale data fusion manner, and finally a spatial model is constructed. The generation of the pose mapping field is based on the gradient information of the depth data, combined with the dynamic changes of the pose information in the time sequence, and the existing distributed field calculation algorithm is used to generate the mapping field.

[0050] For example, during assembly, when the industrial camera captures the edge profile image of the assembly object, combined with the high-precision point cloud of the depth perception module, the texture information of the image edge is combined with the part depth surface to generate a three-dimensional visual model. Through the optical flow displacement estimation of the feature points of the continuous three frames of images, the position of the part is calculated as [x=1.2, y=0.5, z=-0.3] (unit: m), and the attitude angle is [α=30°, β=-12°, γ=45°]. The pose mapping field further reflects the spatial distribution around the assembly object.

[0051] The three-axis torque data is analyzed in time series to obtain the instantaneous stress fluctuation characteristics, the pose mapping field and the instantaneous stress fluctuation characteristics are jointly calculated to generate the assembly action influence domain information, and the three-dimensional visual model is dynamically regionally labeled using the assembly action influence domain information to obtain a key assembly region set.

[0052] The triaxial torque data is analyzed in the frequency domain and the time domain, the instantaneous fluctuation characteristics are extracted by using fast Fourier transform (FFT), the mechanical concentration trend of the key points is identified, the torque peak value data is correlated and analyzed with the key contact areas in the attitude mapping field. Finally, the dynamic area label is generated by using the joint influence domain calculation algorithm, so as to identify the key assembly area set.

[0053] Specifically, in the fast Fourier transform, the sudden change signal of the triaxial torque is extracted by the amplitude distribution of the frequency component, and the mechanical imbalance area near the contact point is taken as the focus of dynamic labeling. When the joint influence domain is calculated, the torque fluctuation characteristics and the time series data of the mapping field are combined through the concentration function of the spatial distribution, and the force concentration area is labeled. The region division logic is based on clustering analysis and fluctuation gradient calculation, and the dynamic area is defined as the key assembly area set.

[0054] For example, in actual operation, when the torque of the contact point changes (such as the instantaneous value is [M x =4.5, M y =2.1, M z =-1.8]N·m), the force gradient change of the key points in the attitude mapping field is greater than 0.75, and the key assembly area set is described through joint analysis, including the high-risk area on the part contact surface extending to the edge side 10%.

[0055] In step S200, the spatial configuration feature information is generated based on the key assembly area set, the spatial configuration feature information is evaluated and analyzed by using the triaxial torque data, and the stability index information and the assembly strain dynamic change map are obtained. The steps include: The assembly configuration contour surface is constructed based on the key assembly area set, the assembly configuration contour surface is combined with the three-dimensional pose information to perform topology modeling, and the spatial configuration feature information is obtained; wherein the three-dimensional pose information refers to the position coordinate information and the orientation attitude information of the current assembly object in the station coordinate system, including the translation vector and the rotation matrix in the three-dimensional space.

[0056] The spatial point data of each region in the key assembly area set is combined, and a surface reconstruction algorithm is used to generate the assembly configuration contour surface. The position coordinate information and the orientation attitude information (translation vector and rotation matrix) are mapped to the point set of the generated contour surface in combination with the three-dimensional pose information of the assembly object in the station reference coordinate system. In the topology modeling process, the method of discrete grid representation is adopted to form the topology structure of the spatial configuration feature, and the configuration properties of each region are labeled. Through these operations, the complete geometric feature and the dynamic attitude information of each key region in the assembly space are obtained.

[0057] Specifically, the surface construction is implemented using the "Poisson Surface Reconstruction" algorithm in the existing technology. The goal is to transform sparse point cloud data into a coherent and smooth assembly contour surface through global optimization. In topological modeling, the Delaunay algorithm based on triangulation is used to generate mesh segmentation. By connecting the adjacent point sets in the point cloud into a continuous triangular mesh structure, its spatial information is annotated. The three-dimensional pose information is obtained by the coordinate translation vector [t x ,t y ,t z ] and the rotation matrix R, where the rotation matrix R is a 3×3 orthogonal matrix used to describe the orientation of the assembly object, and the formula is expressed as: ; Each element of R describes the projection of the reference coordinate axis vector in the target coordinate system. Combining this information, the fusion of the contour surface and the 3D pose is completed.

[0058] For example, for a certain disk part, its outer contour is reconstructed through the point cloud of the key assembly area set to obtain an assembly configuration contour surface with a surface accuracy of 0.2mm; assuming that the current contour center coordinates are [100.5, 200.3, 50.1] mm, the orientation posture is determined by the rotation matrix ; Complete the topological combination of surfaces and postures to generate spatial configuration feature information.

[0059] The three-axis moment data is used to perform multi-region loading simulation analysis on the spatial configuration characteristic information to obtain the force uniformity distribution map and contact stiffness variation characteristics of each configuration area. The force uniformity distribution map and the contact stiffness variation characteristics are cross-matched to obtain high-risk contact sub-areas and low-stability feature point sets.

[0060] Based on the distribution characteristics of each key assembly area from the spatial configuration feature information, a multi-region loading simulation is performed. During the simulation, triaxial torque data is used as the mechanical input to construct a multi-physics field coupling simulation model to evaluate the stress distribution, contact stiffness, and force uniformity of each local area. The resulting force uniformity distribution map and contact stiffness variation characteristics respectively reflect areas of force concentration and stiffness weakening during the assembly process. By cross-matching these two types of data, high-risk contact sub-regions and low-stability feature point sets with overlapping force and stiffness variations are further screened.

[0061] Specifically, during simulation analysis, finite element analysis software (such as ANSYS) is used to apply loading conditions to the discretized assembly configuration mesh. The force uniformity distribution map is calculated from the stress gradient of the simulated mesh elements, and the contact stiffness characteristic is derived from the rate of change of force per unit displacement. A matching algorithm based on correlation analysis is used during cross-matching to assess the significance of the overlap between the two spatial distributions.

[0062] For example, in a certain assembly object, a load is applied to the configuration area, and its force uniformity distribution diagram shows that the stress concentration area is located at the point (30mm, 60mm); the contact stiffness change map shows that the point where the stiffness drops significantly coincides with the above-mentioned area. From this matching, it is concluded that this area is a high-risk contact sub-area, and the key area position of the low-stability feature point set is located at the same time.

[0063] The real-time contact pressure distribution of the high-risk contact sub-area during the current assembly process is matched with the force pattern of the same contact area in the historical task database, and the deviation score of each sub-area is calculated. Based on the deviation score, stability index information including spatial position, force stability score and contact duration is constructed.

[0064] The contact pressure distribution in high-risk contact sub-areas is recorded in real time and compared with standard contact patterns in a historical task database to generate a deviation score for the current assembly process. The deviation score comprehensively considers differences in maximum, mean, and fluctuation values ​​of contact pressure, combined with assembly time information for the sub-areas, to generate a stability index.

[0065] Specifically, during the matching calculation process, an algorithm based on cosine similarity is used to quantify the shape differences of the contact pressure distribution. Specifically, the pressure distribution is represented as a feature vector P, and the cosine similarity is: ; Among them, P c is the eigenvector of the real-time contact pressure distribution, P h is the characteristic vector of the historical contact pattern, Denotes the second norm. The deviation score is defined as 1-S.

[0066] For example, in a certain sub-area during the assembly process, the deviation score of the real-time distribution characteristic of contact pressure is 0.25, and the contact time is 5s. The stability index constructed is: position (50mm, 75mm), deviation score 0.25, and contact duration 5s.

[0067] The three-dimensional displacement time series of the low-stability feature point set during the entire assembly process is extracted, the strain rate change trend and contact area change trend of the three-dimensional displacement time series are calculated, and the strain rate change trend and contact area change trend are used to construct the contact area change information.

[0068] The three-dimensional displacement data of the area where the low-stability feature point set is located is continuously sampled, and a displacement time series is established based on time. The strain rate change trend and contact area change trend are calculated based on the sampled data. First, the displacement change of the feature point is mapped to the strain rate, and the overall strain trend is analyzed by the relative offset between adjacent points on the point cloud. At the same time, combined with the force concentration point in the area, the contact area change rate between the feature point and the contact surface is calculated to reflect the change in mechanical distribution. Finally, the strain rate change trend is combined with the contact area change trend, and the contact area change information is generated through a unified trend coding.

[0069] Specifically, the three-dimensional displacement time series is generated according to the following formula: i ,y i ,z i ) at any time t is calculated as the displacement ΔL t , the formula is: ; Among them, x i ,y i ,z i The displacement is the three-dimensional coordinate of the feature point at time t. The displacement change is used to analyze the strain change rate of the contact surface. The strain rate trend is determined by solving the strain tensor point by point and calculating the change in deformation intensity. The contact area change is calculated by dynamically adjusting the area of ​​the force-affected region. Both are then normalized and stored in a coded format.

[0070] For example, within a specific time period, the average displacement change of feature points within a low stability feature point set is ΔL t =0.15mm, the strain rate change trend shows a gradual linear increase (the initial strain rate is 0.03 / s and the final strain rate is 0.08 / s), and the contact area change rate is from 12mm 2 Reduced to 5mm 2 Combining the above changes, the unified trend information is encoded to generate the contact area change information, marking the sub-region as a potential unstable area.

[0071] The dynamic change map of assembly strain is constructed based on the contact area change information and stability index information.

[0072] By integrating contact area change information and stability index information, key parameters (such as strain rate changes, contact area dynamics, position coordinates, and stability scores) are mapped onto a spatial configuration model to construct a dynamic strain map of the assembly. This map uses 3D simulation technology to update the dynamic state of the assembly area in real time, revealing stress concentration trends, displacement change distribution, and stability evaluation results within the region.

[0073] Specifically, the generation of the assembly strain dynamic change map is based on the spatial configuration model, the contact area change information is added to the stability index part as a dynamic change layer to form a distributed change synthesis map. The data fusion follows the hierarchical mapping principle, in which the horizontal mapping is realized by marking the contact area change information to any spatial sub-region, and the vertical mapping is realized by dynamically supplementing the regional characteristics through the stability score. The map can be automatically adjusted in real time with the change of the assembly data, and finally presented through the graphical software.

[0074] For example, for a certain assembly object, the stress distribution characteristics of the assembly area are obtained through simulation, the distribution of the low stability feature point set is combined with the stability score, and finally the assembly strain dynamic change map is formed. The map clearly shows that the regions with high strain rate change are concentrated in x=50-75mm, y=20-30mm, and the contact area gradually decreases to the critical value. At the same time, the stability score warning point is located at x=75mm, y=25mm, prompting the operator to pay special attention to these areas.

[0075] Among them, the multi-region loading simulation analysis of the spatial configuration feature information is carried out by using the three-axis torque data, the stress uniformity distribution map and the contact stiffness change characteristics of each configuration region are obtained, and the high-risk contact sub-region and the low stability feature point set are obtained by cross matching the stress uniformity distribution map and the contact stiffness change characteristics. The steps include: According to the three-axis torque data, each key assembly area in the spatial configuration feature information is divided into multiple local regions, and multi-physical field coupling simulation is carried out on each local region to obtain the loading simulation result.

[0076] According to the stress characteristics of each key assembly area in the three-axis torque data, the spatial configuration feature information is divided into multiple local regions. The division of each key assembly area is based on the characteristics of force distribution and geometric topological characteristics, and the discretization processing is carried out by using the grid model (for example, using finite element grid division). Then, for each local region, a multi-physical field coupling simulation model of elasticity and thermodynamics is established. The three-axis torque data is input as the loading simulation data to calculate the stress distribution, deformation behavior and temperature field change of the loaded assembly area to generate the loading simulation result.

[0077] Specifically, the mesh division adopts the "adaptive mesh refinement algorithm" (AMR) in the prior art, and regions with severe stress changes (such as high stress gradient regions) are refined to improve the simulation accuracy. During the simulation process, the multi-physical field coupling model uses the finite element analysis (FEA) method, and the mechanical boundary conditions are set as: the position and size of the three-axis torque Mx, My, and Mz, and the elastic modulus and Poisson's ratio are applied to the material properties. The thermodynamic model considers the thermal effect generated by volume deformation. Finally, the loading simulation results of each region are output, including the stress field, displacement field, and thermal field changes.

[0078] For example, when dividing the key assembly area of a rectangular plate-shaped component in an assembly station, the three-axis torque data shows that the action points are located at (20, 30, 0) and (40, 60, 0), respectively. These regions are divided into 1000 refined grid points through mesh division and applied to loading simulation analysis. The simulation results show that the initial stress is distributed between 200 MPa and 300 MPa in the high stress area, and the maximum displacement fluctuation reaches 0.12 mm.

[0079] According to the loading simulation results, the stress distribution of each local region is calculated to generate a stress uniformity distribution map, which reflects the mechanical equilibrium state of each assembly region, and the stability of the stress of each region is analyzed. The stress uniformity distribution map is feature extracted to identify the contact stiffness variation characteristics of each assembly region, and a contact stiffness variation map is obtained, which reflects the dynamic changes of the stiffness of each region during the assembly process.

[0080] Based on the stress field data in the loading simulation results, the stress distribution of each local region is calculated to generate a stress uniformity distribution map reflecting the stress characteristics of the assembly region. The stress uniformity and stress change gradient of each region are analyzed to determine the mechanical equilibrium state. Subsequently, according to the relationship between local displacement change and load, the corresponding contact stiffness variation characteristics are calculated to generate a contact stiffness variation map, thereby comprehensively reflecting the dynamic change characteristics of the stiffness of each region during the assembly process.

[0081] Specifically, the stress uniformity distribution map is generated by the maximum principal stress value of the simulation grid element, and the calculation formula is σ = F / A, where σ is the stress per unit area, F is the force, and A is the area of the action region. The high stress gradient region in the figure is marked as an uneven mechanical region for subsequent analysis. The contact stiffness variation characteristics are calculated by the change rate of the stiffness k as: k = ΔF / Δδ, using a three-dimensional curve to mark the change trajectory of the contact stiffness with time, and generating a contact stiffness variation map.

[0082] For example, the simulation analysis of a certain assembly component shows that the stress concentration area in the stress uniformity distribution map is located at the coordinate point (25, 50, 0), and the corresponding maximum stress value is 250 MPa; the contact stiffness change atlas shows that the stiffness (k) of this area decreases from the initial value of 3500 N / mm to 2800 N / mm. These data show that the stiffness attenuation in the assembly process has a tendency to cause instability.

[0083] Cross-matching the stress uniformity distribution map with the contact stiffness change characteristics, the correlation of the stress and stiffness change in the local area is analyzed, and the high-risk contact sub-area and the low-stability feature point set are identified.

[0084] Using the cross-matching algorithm, the stress uniformity distribution map and the contact stiffness change characteristics are correlated and analyzed to evaluate the spatial distribution consistency and dynamic change trend between the two groups of data. By calculating the correlation index, the intersection of the uneven stress and the significant stiffness attenuation area is marked, and the contact sub-area with high-risk characteristics and the key point set with low stability characteristics (low-stability feature point set) are further screened out.

[0085] Specifically, the cross-matching uses the Pearson Correlation Coefficient (PCC) calculation formula: ; Where Cov(X, Y) is the covariance of the stress uniformity distribution map data X and the contact stiffness change characteristic data Y, σ X ,σ Y are the standard deviations of X and Y, respectively. According to the correlation coefficient ρ, the area with a threshold value ρ≥0.85 is marked as a high-risk contact sub-area, and the low-stability feature point set is marked by points outside the threshold range.

[0086] For example, in a certain assembly task, the cross-matching of the stress uniformity distribution map and the contact stiffness change characteristics shows that the correlation of the area point (30, 45, 0) reaches 0.92 and is marked as a high-risk contact sub-area; while the point (20, 35, 0) shows low stability due to a correlation of 0.65. Further analysis shows that the stiffness attenuation rate of this feature point exceeds 25% within 10 seconds, indicating the presence of mechanical abnormal phenomena.

[0087] In step S300, the stability index information is feature extracted to obtain a stability feature vector, and the stability feature vector is used to identify the contact mode of the assembly strain dynamic change atlas to obtain a contact area state sequence, which specifically includes: The principal component analysis and dynamic time warping are performed on the stability index information to extract the steady-state feature distribution and change rate, and the stability feature vector is constructed using the steady-state feature distribution and change rate.

[0088] The stability index information is taken as an input variable, and principal component analysis (PCA) is first performed on the data. By reducing the dimension, the high-dimensional characteristic data (such as the strain rate change, contact stiffness dynamic characteristics and position stability score) is mapped to a lower-dimensional space, so as to extract the steady-state characteristic distribution. At the same time, the dynamic time warping (DTW) algorithm is used to analyze the time series of the stability characteristics, calculate the change rate and correct the inconsistency in time. Based on the extraction results, a stability characteristic vector containing the steady-state distribution and the change rate is constructed.

[0089] Specifically, the PCA process includes calculating the covariance matrix of the stability index information data set, sorting the eigenvectors and eigenvalues of the covariance matrix, and selecting the first few principal components with the largest contribution rate as the reduced dimension feature representation; the dynamic time warping is used to calculate the optimal matching path of any two unequal length time series, and the formula is as follows: ; Where D(i,j) represents the total consumption distance before the current point, and d(i,j) is the Euclidean distance between the two data points. Finally, the principal components and the time change rate are combined into a stability characteristic vector.

[0090] For example, for a certain assembly object, the three main features of the stability index information are the strain rate change, the contact stiffness change and the stability score. After principal component analysis, the first two principal components with the largest contribution rate are PC1=80% and PC2=15%. Dynamic time warping analysis shows that after correcting the time inconsistency between the strain rate change feature points and the stiffness change, the final stability characteristic vector is [0.5, 0.2].

[0091] The assembly strain dynamic change map is divided into intervals and dynamically clustered to label the stress response patterns in different time windows, and the stress response patterns are obtained. The stability characteristic vector and the stress response pattern are jointly embedded in a unified feature space to obtain a fusion stability characteristic vector.

[0092] The assembly strain dynamic change map is divided into several small windows according to the time dimension, and the feature of each time interval image data is extracted and grouped by dynamic clustering algorithm, and its corresponding stress response pattern is labeled. Then, the feature vector extracted from the stability index information is fused with the stress response pattern in the unified feature space to generate a fusion stability characteristic vector containing geometric and mechanical characteristics.

[0093] Specifically, the time window is divided according to a fixed time interval (for example, every 1 second) or an adaptive segmentation of a significant inflection point of stress change; the stress response feature extraction of each window is performed by calculating the local stress peak value, gradient change and other feature vectors. The dynamic clustering algorithm can use K-means++, and the clustering classification is performed by minimizing the Euclidean distance between the feature points of each window and the cluster center point. The stability feature vector and the stress response mode feature vector are concatenated in a high-dimensional space to establish a unified feature vector representation.

[0094] For example, in the assembly process, the dynamic change map is divided into 5 time windows within 0-5 seconds, and the feature points of each window form a stress response mode; for example, the stress peak value in window 1 is 50 MPa, and the stress gradient is 0.8, which is calibrated as mode A. The stability feature vector [0.8, 0.5] and mode A are combined to obtain the fusion feature vector [0.8, 0.5, A].

[0095] According to the fusion stability feature vector, key contact events including contact establishment, slip transition and contact disappearance are identified; and a time sequence contact map is constructed using the key contact events, a contact state transition path of the time sequence contact map is extracted, and a complete contact area state sequence is obtained by structure coding processing of the contact state transition path.

[0096] Using the dynamic change rule of the fusion stability feature vector, key contact events (such as contact establishment, slip transition and contact disappearance) occurring in the assembly process are identified, and the time and spatial position of the occurrence of these events are analyzed. Based on these events, a time sequence contact map is constructed to represent the contact state evolution trajectory in the assembly process. At the same time, the contact state transition path is extracted from the map, and it is expressed as a time-sequenced state sequence by path structure coding processing.

[0097] Specifically, the nodes in the time sequence contact map are represented by key contact events, and the connections between the nodes represent the time and spatial relationship of the state transition path. For example, the contact establishment event is defined as the transition from no contact to stress > 5 MPa and contact area > 10 mm 2 The slip transition event is identified as a sudden drop in stiffness (> 30%); and the contact disappearance event is a rapid reduction in contact area. The transition path of the map is represented by a state transition matrix, and each element of the matrix includes an event number and a time interval. The path structure code is generated by depth-first search to traverse the entire transition path to generate a time-sequenced sequence.

[0098] For example, the contact event of a certain assembly process is identified as follows: contact establishment occurs at time 0.2 seconds, slip transition is detected at time 3.6 seconds, and contact disappears at time 6.0 seconds. A time sequence contact map is constructed based thereon, and the transition path in the map is represented as: Path = {(establish, 0.2s), (slip, 3.6s), (disappear, 6.0s)}.

[0099] The contact area state sequence generated after encoding is: establish-slip-disappear.

[0100] In step S400, the contact area state sequence is input into a preset assembly abnormality recognition model to obtain an assembly abnormality risk assessment result, and the assembly abnormality risk assessment result is classified and analyzed to obtain risk level information and a strategy candidate set. The steps include: The contact area state sequence is encoded and vectorized according to a time window and a state node to generate a dynamic contact behavior sequence, the dynamic contact behavior sequence is input into a preset assembly abnormality recognition model, and an assembly stability deviation degree and an abnormal pattern label are output.

[0101] According to the time and space variation characteristics of the contact area state sequence, the sequence data is divided into multiple time windows, and the state nodes of each window are time node encoded and feature vectorized. Using these encoded dynamic contact behavior sequences as input, they are imported into a preset assembly abnormality recognition model for real-time calculation and analysis to identify the stability deviation degree and abnormal pattern in the assembly process, and the result is output in the form of an abnormal pattern label.

[0102] Specifically, the time windows are dynamically divided according to the state change time points (such as contact transition time, force state mutation time) in the contact area state sequence, and the state node information of each window is converted through an encoding rule, for example: ; Where B i is the i-th node feature vector of the window, including the timestamp t i , the state s i (such as contact establishment, slip transition, etc.), and the force state feature f i (including contact area, stress, etc.). The assembly abnormality recognition model uses an existing convolutional neural network (CNN) to realize abnormal analysis of the dynamic contact behavior sequence through a feature extraction layer and a classification layer, and finally outputs the stability deviation degree and the abnormal label, such as "overstress", "force concentration", etc.

[0103] For example, in a certain assembly process, the contact area state sequence is recorded as follows: the state "contact establishment" is generated at 0.2 seconds, the state "slip transition" occurs at 3.0 seconds, and the state "contact disappearance" is detected at 7.0 seconds. The code is a time sequence dynamic contact behavior sequence: ; This sequence is input into the CNN model, and finally the deviation degree 0.35 and the abnormal mode label "force concentration abnormality" are output.

[0104] According to the assembly stability deviation degree and the abnormal mode label, an assembly abnormality risk assessment result is generated, including the abnormal type, the severity, and the triggering time period; the assembly abnormality risk assessment result is compared with a preset assembly database for similarity, to obtain risk level information, and the risk level information is matched with a preset strategy template library, to obtain a strategy candidate set.

[0105] The deviation degree and the abnormal label output by the model are transmitted to a subsequent abnormality assessment module, and according to the deviation degree and the label type, an assembly abnormality risk assessment result is generated, including the abnormal type, the severity, and the triggering time period. These results are compared with historical data in an assembly task database for similarity, to calculate the probability and the influence amplitude of the abnormal mode, and finally risk level information (such as "low risk", "high risk", etc.) is generated. Based on the risk level information, a strategy template library is matched to generate a strategy candidate set adapted to the current assembly state.

[0106] Specifically, the risk assessment grading is based on the following rules: let the deviation degree score be p, the severity range be [1, 5], when p < 0.2, it is low risk, 0.2 ≤ p < 0.5, it is medium risk, and p ≥ 0.5, it is high risk; the abnormal label type is graded by probability statistics of a historical database, for example, if the probability of over-stress type exceeds 70%, the risk level is increased. The strategy candidate set generation process uses a matching utility score algorithm: ; Where S is the utility score, e i is the current environment state, r i is the strategy template content, sim() is the similarity function, and w i is the weight coefficient. Strategy templates with high matching scores are added to the candidate set.

[0107] For example, in a certain assembly task, the deviation score is 0.45, the trigger abnormal type is "slip transition", and the time period is 2.5-4.0 seconds. The risk level is confirmed as "medium risk" (probability of occurrence 65%) by database comparison. After comparison with the strategy template library, the generated strategy candidate set includes: ① adjust torque direction, ② optimize assembly path, ③ dynamic position calibration.

[0108] In step S500, the dynamic evolution data of the spatial configuration feature information is obtained according to the risk level information, the dynamic evolution data is classified, a plurality of assembly state transition path sets are obtained, and the strategy candidate set is used to match all the assembly state transition path sets to obtain the assembly guidance strategy. The steps include: Based on the risk level information, the corresponding key spatial configuration region in the spatial configuration feature information is selected, and the spatial feature evolution trajectory of the key spatial configuration region with time is extracted to obtain the dynamic evolution data. The dynamic evolution data is subjected to multi-dimensional feature mapping and path structure analysis to obtain an analysis result, and the analysis result is divided into a plurality of assembly state transition units.

[0109] According to the risk level information of the assembly task, the spatial configuration region most related to the current risk level is selected. Based on these key regions, the spatial feature evolution trajectory of the key regions with time is extracted, and the data is derived from real-time monitoring and simulation systems. On this basis, multi-dimensional feature mapping is performed to convert to a unified feature space to capture the change trend of the spatial configuration in the assembly process. The path structure analysis is performed on these change data, and the results are divided into a plurality of assembly state transition units according to the transition relationship between the states in the assembly process.

[0110] Specifically, the extraction of the spatial feature evolution trajectory adopts time series analysis and feature extraction technology, and combines position, attitude, stress and other features to construct the evolution trajectory graph. In the path structure analysis process, the shortest path algorithm in graph theory is used to optimize the change path, and the continuous change state sequence is divided into a plurality of transition units by constructing a state transition matrix.

[0111] For example, in a certain assembly task, the risk level is "medium risk", so the regions with significant torque changes during assembly are selected for dynamic evolution data extraction. By analyzing the pose changes of these regions (for example, the position changes are x: 5.2 mm, y: 3.5 mm), the dynamic evolution trajectory is obtained. Further path analysis shows that these changes can be divided into two assembly state transition units: the first unit is "contact establishment", and the second unit is "slip transition".

[0112] All assembly state transition units are clustered into assembly state transition path sets according to the chronological order and evolution trend. Each strategy in the strategy candidate set is matched with each path in the assembly state transition path set to construct a strategy path adaptation matrix.

[0113] The clustering analysis of the chronological order and evolution trend is performed on all assembly state transition units. Each state transition in the assembly process is optimally clustered by a clustering algorithm such as dynamic time warping (DTW), so that each state transition path can reflect the real assembly operation steps. After clustering, each strategy in the strategy candidate set is matched with each path to evaluate the adaptation degree of the strategy to the path, and a strategy path adaptation matrix is constructed based on the score.

[0114] Specifically, in the clustering process, first, the change trend of each path is divided into time windows, and the optimal matching path combination is found by dynamic time warping analysis. Then, the similarity scoring algorithm (such as cosine similarity, Euclidean distance) is used to match each path with the strategies in the strategy candidate set. Finally, a strategy path adaptation matrix is constructed based on the matching results, and each matrix element represents the matching degree of the strategy and the path.

[0115] For example, in the assembly process, after dynamic time warping analysis, the assembly state transition units are clustered into three paths: path A (contact establishment - slip transition), path B (contact disappearance - stress recovery), and path C (contact establishment - torque fluctuation). For each path, by matching with the strategies in the strategy candidate set, the matching degree of path A with strategy 1 is 0.85, the matching degree of path B with strategy 2 is 0.72, and the matching degree of path C with strategy 3 is 0.90. Finally, the strategy path adaptation matrix is obtained.

[0116] The intervention effectiveness simulation prediction and response time evaluation are performed on each strategy-path combination in the strategy path adaptation matrix to obtain test evaluation results, and the optimal strategy-path combination is selected based on the test evaluation results to generate an assembly guidance strategy containing operation sequence, intervention content and guidance mode.

[0117] Each strategy-path combination in the strategy path adaptation matrix is simulated and predicted by the intervention effectiveness simulation system to evaluate the execution effect of each strategy under the corresponding path. During the simulation process, various factors such as assembly efficiency, accuracy, stability, etc. are considered to evaluate the intervention effectiveness of each strategy. At the same time, the response time is evaluated, considering the implementation time of the strategy and the real-time reaction in the assembly process. Based on these test evaluation results, the optimal strategy-path combination is selected, and finally the assembly guidance strategy is generated, containing detailed operation sequence, intervention content and guidance mode.

[0118] Specifically, in the simulation of intervention effectiveness prediction, the existing simulation platform (such as MATLAB, Simulink, etc.) is used for dynamic modeling, the simulation environment parameters (such as torque, contact pressure, component displacement) of the strategy path are input, the intervention effect is simulated through the control algorithm, and experimental verification is carried out. The response time evaluation is compared with the experimental data through time complexity analysis, and the strategy with the shortest time and the best effect is selected.

[0119] For example, in a certain assembly task, path A has the highest matching score with strategy 1. Through intervention effectiveness simulation, strategy 1 can effectively reduce the slip transition time, thereby improving the assembly accuracy. After response time evaluation, the response time of this strategy is 0.3 seconds. Compared with other strategies, strategy 1 performs best, so it is selected as the optimal strategy, and the generated assembly guidance strategy contains the operation sequence: ① calibrate the contact point position, ② optimize the force direction, ③ adjust the torque value.

[0120] In this embodiment, by collecting the original image sequence, depth coordinate data and three-axis torque data of the current assembly object on the assembly station, first, the multi-sensor fusion technology is used to generate a set of key assembly regions, combined with calibration registration and data synchronization analysis, a three-dimensional visual model of the assembly component, real-time pose information and attitude mapping field are constructed, and according to the three-axis torque data, the instantaneous stress fluctuation characteristics are extracted to realize the dynamic region labeling of the assembly action. Subsequently, based on the set of key assembly regions, spatial configuration feature information is generated, through topological modeling of three-dimensional pose information and multi-physical field coupling simulation, the stress uniformity distribution and contact stiffness change trend of the local region are analyzed, and through cross matching, high-risk contact sub-regions and low-stability feature point sets are identified, combined with real-time contact pressure distribution and historical database matching, stability index information and contact region change information are comprehensively evaluated to generate an assembly strain dynamic change map. In the contact mode recognition stage, through principal component analysis and dynamic time warping of the stability index information, the steady-state feature distribution and change rate are extracted, and a stability feature vector is constructed, and at the same time, combined with the interval segmentation and dynamic clustering of the assembly strain dynamic change map, a fusion stability feature vector is generated, key contact events in the assembly process are identified, and a state sequence of the contact region is constructed. In the assembly abnormality recognition and guidance strategy generation process, through dynamic analysis of the contact region state sequence and the assembly abnormality recognition model, an assembly abnormality risk evaluation result (including deviation, risk level and abnormal type) is generated, and a strategy matching tool is used to generate a strategy path adaptation matrix based on the set of assembly state transition paths, the optimal strategy path combination is selected through simulation evaluation, and an assembly guidance strategy containing operation sequence, intervention content and guidance method is obtained. This embodiment realizes accurate perception and adaptive adjustment of dynamic changes in complex assembly environment, effectively improving assembly efficiency and quality.

[0121] Embodiment 3: The application also provides a visual guidance system 10 for intelligent auxiliary assembly, comprising a collection module 11, an analysis module 12, an extraction module 13, an input module 14 and a matching module 15.

[0122] As shown in the figure, the collection module 11 is mainly used for collecting the original image sequence, depth coordinate data and three-axis torque data of the current assembly object on the assembly station to construct a key assembly region set. Figure 2

[0123] The collection module 11 collects and comprehensively processes the image data, depth coordinate data and three-axis torque data of the current assembly object on the assembly station in real time through a high-precision industrial camera, a depth perception module and a force sensor, thereby constructing a key assembly region set. This module ensures the completeness and real-time of the assembly data, and provides high-quality data input for subsequent analysis.

[0124] The analysis module 12 is mainly used for generating spatial configuration feature information based on the key assembly region set, and performing stress state evaluation analysis on the spatial configuration feature information by using the three-axis torque data to obtain stability index information and assembly strain dynamic change atlas.

[0125] The analysis module 12 generates spatial configuration feature information based on the collection data, and performs stress state evaluation analysis on the configuration feature information by using the three-axis torque data to obtain the stability index and the assembly strain dynamic change atlas in the assembly process. Through this module, the stress distribution, stress uniformity and dynamic change trend of the assembly object can be clearly reflected, which provides an important mechanical basis for identifying the assembly abnormal tendency.

[0126] The extraction module 13 is mainly used for feature extraction on the stability index information to obtain a stability feature vector, and uses the stability feature vector to perform contact mode recognition on the assembly strain dynamic change atlas to obtain a contact region state sequence.

[0127] The extraction module 13 further extracts features from the stability index information generated by the analysis module 12 to calculate a stability feature vector, and performs contact mode recognition on the assembly strain dynamic change atlas through a dynamic clustering algorithm to calibrate the key state of the contact region in the assembly process. Through this module, the dynamic contact mode of the assembly object and the contact abnormality occurring can be accurately identified, which provides input for abnormal risk assessment.

[0128] The input module 14 is mainly used for inputting the contact region state sequence into a preset assembly abnormality recognition model to obtain an assembly abnormality risk assessment result, performing hierarchical analysis on the assembly abnormality risk assessment result to obtain risk level information and a strategy candidate set.

[0129] ​The input module 14 inputs the extracted contact area state sequence into a preset assembly abnormality recognition model, recognizes and classifies the abnormal risk based on a deep learning algorithm, and finally generates risk level information and a strategy candidate set. This module can effectively distinguish the severity and occurrence area of different risks in the assembly process, and provide reliable basis for the generation of assembly guidance strategies.

[0130] The matching module 15 is mainly used to obtain dynamic evolution data of spatial configuration feature information according to the risk level information, classify the dynamic evolution data to obtain a plurality of assembly state transition path sets, and match the strategy candidate set with all the assembly state transition path sets to obtain an assembly guidance strategy.

[0131] The matching module 15 selects dynamic evolution data in the spatial configuration feature information based on the risk level information, and classifies the data to generate an assembly state transition path set. This module uses the strategy candidate set to intelligently match all path sets, selects the optimal strategy path combination through an algorithm, and finally generates an assembly guidance strategy containing intervention measures and guidance content. Through this module, the assembly process can be dynamically optimized in a complex assembly environment, and fast and effective guidance and intervention can be provided.

[0132] In this embodiment, real-time data acquisition, multi-dimensional mechanical analysis, abnormal risk identification, and intelligent generation and guidance of assembly strategies are realized through modular design, forming a complete closed-loop assembly optimization process. This system has the significant advantages of high efficiency, precision and flexibility, and helps to improve the automation level and assembly precision of complex assembly tasks, and solves the defects of existing technologies in dealing with dynamic changes and real-time control.

[0133] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and each module described above can refer to the corresponding process in the foregoing embodiment 1, which will not be repeated here.

[0134] The structures, proportions, sizes, etc. shown in the drawings of the present specification are only used to cooperate with the content disclosed in the specification, to enable those skilled in the art to understand and read, and do not define the limiting conditions for the implementation of the present application, so they do not have technical substantive significance. Any modification of the structure, change of the proportion relationship or adjustment of the size, without affecting the effect and purpose that the present application can produce, should still fall within the scope of the technical content disclosed by the present application.

[0135] The above-described embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalent replacements; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A visual guidance method for intelligent assisted assembly, characterized in that: include: Collect the original image sequence, depth coordinate data, and three-axis torque data of the current assembly object at the assembly station to construct a set of key assembly areas; Generate spatial configuration feature information based on the key assembly area set, and use the three-axis moment data to perform stress state evaluation and analysis on the spatial configuration feature information to obtain stability index information and assembly strain dynamic change map; Extracting features from the stability index information to obtain a stability feature vector, and using the stability feature vector to perform contact mode recognition on the assembly strain dynamic change map to obtain a contact area state sequence; Inputting the contact area state sequence into a preset assembly anomaly recognition model to obtain an assembly anomaly risk assessment result, performing hierarchical analysis on the assembly anomaly risk assessment result to obtain risk level information and a strategy candidate set; Dynamic evolution data of the spatial configuration feature information is obtained according to the risk level information, the dynamic evolution data is classified to obtain a plurality of assembly state transition path sets, and strategy matching is performed on all the assembly state transition path sets using the strategy candidate set to obtain an assembly guidance strategy.

2. The visual guidance method for intelligent assisted assembly according to claim 1, characterized in that: The step of collecting the original image sequence, depth coordinate data, and three-axis torque data of the current assembly object at the assembly station to construct a set of key assembly areas includes: The industrial camera, depth perception module and force sensor installed at the assembly station are used to synchronously collect the original image sequence, depth coordinate data and three-axis torque data of the force applied on the contact surface of the current assembly object; Calibrate and register the original image sequence and the depth coordinate data, combine the calibrated and registered original image sequence and depth coordinate data with a workstation reference coordinate system to generate a three-dimensional visual model and real-time pose information of the assembly component, and fuse the three-dimensional visual model of the assembly component with the real-time pose information to obtain a pose mapping field; A time series analysis is performed on the three-axis torque data to obtain instantaneous stress fluctuation characteristics, the posture mapping field and the instantaneous stress fluctuation characteristics are jointly calculated to generate assembly action influence domain information, and the assembly action influence domain information is used to dynamically annotate the three-dimensional visual model to obtain a set of key assembly areas.

3. The visual guidance method for intelligent assisted assembly according to claim 1, characterized in that: The step of generating spatial configuration feature information based on the key assembly area set, performing stress state evaluation and analysis on the spatial configuration feature information using the three-axis moment data, and obtaining stability index information and an assembly strain dynamic change map includes: Constructing an assembly configuration contour surface based on the set of key assembly areas, and topologically modeling the assembly configuration contour surface in combination with three-dimensional pose information to obtain spatial configuration feature information; wherein the three-dimensional pose information refers to the position coordinate information and orientation posture information of the current assembly object in the workstation coordinate system; Using the triaxial moment data, a multi-region loading simulation analysis is performed on the spatial configuration feature information to obtain a force uniformity distribution map and a contact stiffness variation characteristic of each configuration region. The force uniformity distribution map is cross-matched with the contact stiffness variation characteristic to obtain a high-risk contact sub-region and a low-stability feature point set. Matching the real-time contact pressure distribution of the high-risk contact sub-area during the current assembly process with the force pattern of the same contact area in the historical task database, calculating the deviation score of each sub-area, and constructing stability index information based on the deviation score; Extracting a three-dimensional displacement time series of the low-stability feature point set during the entire assembly process, calculating a strain rate change trend and a contact area change trend of the three-dimensional displacement time series, and constructing contact area change information using the strain rate change trend and the contact area change trend; An assembly strain dynamic change map is constructed based on the contact area change information and the stability index information.

4. The visual guidance method for intelligent assisted assembly according to claim 3, characterized in that: The step of performing multi-region loading simulation analysis on the spatial configuration feature information using the triaxial moment data to obtain a force uniformity distribution diagram and a contact stiffness variation characteristic of each configuration region, and cross-matching the force uniformity distribution diagram with the contact stiffness variation characteristic to obtain a high-risk contact sub-region and a low-stability feature point set includes: Dividing each key assembly area in the spatial configuration feature information according to the three-axis moment data to obtain a plurality of local areas, and performing a multi-physics field coupling simulation on each of the local areas to obtain a loading simulation result; Calculating the stress distribution of each local area based on the loading simulation results to generate a force uniformity distribution map, performing feature extraction on the force uniformity distribution map, identifying the contact stiffness variation characteristics of each assembly area, and obtaining a contact stiffness variation map; The force uniformity distribution map is cross-matched with the contact stiffness variation characteristics, the correlation between the force and stiffness variation in the local area is analyzed, and the high-risk contact sub-areas and low-stability feature point sets are identified.

5. The visual guidance method for intelligent assisted assembly according to claim 1, characterized in that: The step of extracting features from the stability index information to obtain a stability feature vector, and using the stability feature vector to perform contact pattern recognition on the assembly strain dynamic change map to obtain a contact area state sequence includes: Performing principal component analysis and dynamic time warping on the stability index information to extract steady-state characteristic distribution and change rate, and constructing a stability characteristic vector using the steady-state characteristic distribution and the change rate; Performing interval segmentation and dynamic clustering on the assembly strain dynamic change map to obtain a stress response pattern, and jointly embedding the stability feature vector and the stress response pattern into a unified feature space to obtain a fused stability feature vector; Key contact events are identified based on the fused stability feature vector, and a temporal contact map is constructed using the key contact events. The contact state transition path of the temporal contact map is extracted, and the contact state transition path is structurally encoded to obtain a complete contact area state sequence.

6. The visual guidance method for intelligent assisted assembly according to claim 1, characterized in that: The step of inputting the contact area state sequence into a preset assembly anomaly recognition model to obtain an assembly anomaly risk assessment result, and performing hierarchical analysis on the assembly anomaly risk assessment result to obtain risk level information and a strategy candidate set includes: Encode and vectorize the contact area state sequence according to the time window and state node to generate a dynamic contact behavior sequence, input the dynamic contact behavior sequence into a preset assembly anomaly recognition model, and output the assembly stability deviation and an abnormal mode label; An assembly anomaly risk assessment result is generated based on the assembly stability deviation and the anomaly pattern label, and a similarity comparison is performed on the assembly anomaly risk assessment result using a preset assembly database to obtain risk level information. A preset policy template library is matched based on the risk level information to obtain a policy candidate set.

7. The visual guidance method for intelligent assisted assembly according to claim 1, characterized in that: The steps of acquiring dynamic evolution data of the spatial configuration feature information according to the risk level information, classifying the dynamic evolution data to obtain a plurality of assembly state transition path sets, and performing strategy matching on all the assembly state transition path sets using the strategy candidate set to obtain an assembly guidance strategy include: Selecting a key spatial configuration area corresponding to the spatial configuration feature information based on the risk level information, extracting a spatial feature evolution trajectory of the key spatial configuration area over time to obtain dynamic evolution data, performing multidimensional feature mapping and path structure analysis on the dynamic evolution data to obtain an analysis result, and dividing the analysis result into a plurality of assembly state transition units; Clustering all the assembly state transition units into an assembly state transition path set according to the time sequence and evolution trend, performing matching scores between each strategy in the strategy candidate set and each path in the assembly state transition path set, and constructing a strategy path adaptation matrix; An intervention effectiveness simulation prediction and response time evaluation are performed on each pair of strategy-path combinations in the strategy-path adaptation matrix to obtain a test evaluation result. An optimal strategy-path combination is selected based on the test evaluation result to generate an assembly guidance strategy.

8. A vision guidance system for intelligent assisted assembly, characterized in that: include: The acquisition module is used to collect the original image sequence, depth coordinate data and three-axis torque data of the current assembly object at the assembly station to construct a set of key assembly areas; an analysis module for generating spatial configuration feature information based on the set of key assembly areas, and performing stress state evaluation and analysis on the spatial configuration feature information using the triaxial moment data to obtain stability index information and a dynamic change map of assembly strain; an extraction module, configured to perform feature extraction on the stability index information to obtain a stability feature vector, and use the stability feature vector to perform contact mode recognition on the assembly strain dynamic change map to obtain a contact area state sequence; An input module is configured to input the contact area state sequence into a preset assembly anomaly recognition model to obtain an assembly anomaly risk assessment result, and perform hierarchical analysis on the assembly anomaly risk assessment result to obtain risk level information and a strategy candidate set; A matching module is used to obtain dynamic evolution data of the spatial configuration feature information based on the risk level information, classify the dynamic evolution data to obtain a plurality of assembly state transition path sets, and use the strategy candidate set to perform strategy matching on all the assembly state transition path sets to obtain an assembly guidance strategy.

Images