A method for real-time monitoring and shape correction of thin-walled components based on binocular vision measurement

Abstract

The present application belongs to the field of intelligent manufacturing, and discloses a kind of thin-walled component surface shape real-time monitoring and shape correction method based on binocular vision measurement.The method combines the acquisition of overall stiffness matrix and mass matrix in finite element analysis, and constructs component surface shape reconstruction equation through modal truncation model order reduction method, obtains three-dimensional direction displacement in measurement point space through binocular vision, obtains servo motor displacement information using servo motor control system, constructs surface shape monitoring and shape correction discrete data, realizes real-time monitoring and shape correction of thin-walled component surface shape state through pre-set Unity3D software and UDP transmission protocol.The present application can effectively improve the measurement speed of thin-walled component deformation, make the deformation of thin-walled component visualized and intuitive, and get rid of the dependence on human experience, which is beneficial to realize in-situ monitoring and real-time adjustment of thin-walled component, and can effectively improve the shape correction speed and accuracy of thin-walled component.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of intelligent manufacturing, and particularly relates to a thin-walled component surface shape real-time monitoring and correction method based on binocular vision measurement. BACKGROUND

[0002] The assembly quality of composite materials significantly affects the safety and other performance indicators of an aircraft during the aircraft manufacturing process. The composite materials used in aircraft manufacturing are mostly large thin-walled components, which are prone to deformation during assembly due to their own weight and other reasons, resulting in reduced assembly quality. Therefore, the surface shape of the composite materials needs to be continuously detected and adjusted during assembly.

[0003] At present, the surface shape measurement of composite plates mainly uses three-coordinate measuring machines, theodolite measurement, laser radar measurement, and structured light measurement. These methods have slow measurement speed, and in addition, laser radar and structured light measurement are easily affected by the environment, resulting in a decrease in measurement accuracy.

[0004] For surface shape regulation of large structural parts, Liu Tingyu and Yang Jiahua innovatively invented a "large structural part surface shape regulation method based on digital twinning" (patent number CN116108722A), which performs finite element analysis on large structural parts, simulates the deformation of large structural parts at different assembly stages, estimates the pose, solves the adjustment value, and finally realizes the assembly optimization of the surface shape. Experimental results show that this method improves the surface shape assembly quality of the component to some extent. However, the model establishment and analysis process is time-consuming and difficult to meet the real-time requirements of assembly.

[0005] Hao Bo, Gu Jiming, and others invented a "method for detecting the assembly quality of an aircraft surface based on digital twinning" (patent number CN114777651A), which uses a laser tracker to measure the surface of an aircraft and compares the obtained data with theoretical data to evaluate the assembly quality. This method can master the assembly out-of-tolerance information through the evaluation results, but the point-by-point measurement method using a laser tracker is slow and time-consuming.

[0006] The above existing measurement methods all require a long time and cannot meet the real-time monitoring requirements in the production and assembly process, and are difficult to break away from the dependence on manpower. Therefore, the current main surface shape detection and regulation method is low-efficiency manual adjustment. SUMMARY

[0007] The present application aims to overcome the defects of the prior art, and invents a kind of thin-walled component surface shape real-time monitoring and shape correction method based on binocular vision measurement, which is mainly aimed at thin-walled component forming, assembly process, to realize the real-time monitoring of thin-walled component surface shape with fast, high-precision and visualization, to help the accurate and rapid real-time regulation of thin-walled component.Using binocular vision measurement, fast non-contact measurement of marker points is realized;Using model degradation method, the surface shape reconstruction speed is improved;Introducing UDP protocol into the system, realizing real-time transmission of data between different software and devices;Introducing Unity3D software, realizing real-time display of thin-walled component surface shape deformation, and using servo motor to realize real-time regulation of thin-walled component surface shape, improving the real-time monitoring and shape correction ability of detection personnel to thin-walled component surface shape change.

[0008] The technical scheme adopted by the present application is:

[0009] A kind of thin-walled component surface shape real-time monitoring and shape correction method based on binocular vision measurement, the acquisition of stiffness matrix and mass matrix in finite element analysis is combined, and the component surface shape reconstruction equation is constructed by modal truncation model reduction method, the three-dimensional direction displacement in space is obtained by binocular vision, the servo motor displacement information is obtained by using servo motor control system, the surface shape monitoring and shape correction discrete data are constructed, the real-time monitoring and shape correction of thin-walled component surface shape state are realized by pre-set Unity3D software and UDP transmission protocol;Specific steps are as follows:

[0010] First, stiffness matrix and mass matrix are combined to construct surface shape reconstruction equation;

[0011] According to the need, mark points are pasted on the surface of thin-walled component, the mark points are used as measured points, and measurement is carried out by binocular vision measurement system;And determine the contact point of servo motor and thin-walled component, and the contact point is used as the controlled point of thin-walled component;

[0012] According to the need, the thin-walled component is discretized by finite element, and the overall stiffness matrix and mass matrix of the thin-walled component are obtained by combining finite element analysis;According to the position of the pasted mark points, the observation matrix is constructed by combining the finite element discretization of the thin-walled component;Eigenvalue decomposition is carried out on the stiffness matrix and mass matrix to obtain the full-order modal shape, and then the displacement of the corresponding mark points in space and the reconstruction relationship of the surface shape of the thin-walled component are constructed by modal truncation model reduction method, to obtain the surface shape reconstruction equation of the thin-walled component;

[0013] An initial model of thin-walled component is established, and node segmentation is carried out on the finite element discretization of the thin-walled component to obtain the finite element model of the thin-walled component;

[0014] The related equations, models and parameters obtained in the above process are stored in Unity3D software for later calling;

[0015] The second step is to obtain the relevant discrete data of the thin-walled component and the servo motor.

[0016] Configure the necessary UDP communication protocol settings in Unity3D, the binocular vision measurement system, and the servo motor control system.

[0017] Discrete data related to thin-walled components and servo motors are sent to Unity3D software in real time via UDP communication protocol;

[0018] Two-dimensional images captured by a binocular camera are reconstructed in three dimensions to obtain the displacement of the measured point in the x, y, and z directions in space.

[0019] The displacement information of the servo motor is obtained through the servo motor control system;

[0020] Discrete data related to thin-walled components and servo motors are sent to Unity3D software in real time via UDP communication protocol;

[0021] The third step is to implement real-time reconstruction and detection of the surface shape of thin-walled components in Unity3D;

[0022] The Unity3D software receives the displacement data of the measured point in space obtained by the binocular vision measurement system and the servo motor displacement information obtained by the servo motor control system.

[0023] The obtained information is decoded and parsed to construct a computation matrix that conforms to the computation rules;

[0024] Combining the previously stored relevant equations and parameters, the calculation is completed, and the surface shape of the thin-walled component is reconstructed in Unity3D in real time.

[0025] At the same time, it provides the tester with relevant deformation parameters of thin-walled components, so as to enable the tester to accurately monitor and control the surface shape of thin-walled components.

[0026] The specific implementation steps are as follows:

[0027] The first step is to construct the surface reconstruction equation by combining the stiffness matrix and the mass matrix;

[0028] m marker points are attached to the surface of the thin-walled component as needed. These marker points are used as the measurement points and are measured using a binocular vision measurement system. The contact point between the servo motor and the thin-walled component is determined and used as the control point of the thin-walled component.

[0029] The thin-walled component is discretized using the finite element method to obtain n nodes. When discretizing the thin-walled component using the finite element method, only the three displacement degrees of freedom in the x, y, and z axes need to be used, while the rotational degrees of freedom in the x, y, and z axes are ignored.

[0030] Combining the results of the finite element discretization of the thin-walled component, the stiffness matrix K and the mass matrix M of the thin-walled component are obtained through finite element analysis, at this time the dimensions of the stiffness matrix K and the mass matrix M are both 3n x 3n:

[0031]

[0032]

[0033] wherein l = 3n, K ij is the element of the i-th row and the j-th column in the stiffness matrix K, M ij is the element of the i-th row and the j-th column in the mass matrix M;

[0034] After pasting m markers, according to the positions of the pasted markers, the observation matrix C is constructed in combination with the finite element discretization of the thin-walled component, the observation matrix C is the position of the m markers in the reference system of the n nodes divided by the thin-walled component, so the dimension of the observation matrix C is m x 3n:

[0035]

[0036] wherein l = 3n, C ij is the element of the i-th row and the j-th column in the observation matrix C, if the node represented by the i-th marker is the k-th node in the n nodes, and the y-axis direction displacement of the node represented by the marker is used to construct the surface shape, then the value of C i(k+n) is 1, for the elements of the i-th row of the observation matrix C, the values of the remaining elements are all 0 except C i(k+n) ;

[0037] Eigenvalue decomposition is performed on the stiffness matrix K and the mass matrix M to obtain the following formula:

[0038] [vv i ,DD i ] = edg[K, M] (4)

[0039] wherein vv i is the eigenvector, DD i is the eigenvalue, K is the stiffness matrix, and M is the mass matrix;

[0040] From formula (1), formula (2) and formula (4), the corresponding i-th order eigenvector vv i is obtained, and the obtained eigenvector is the corresponding modal shape vector, and the modal shape vectors vv i of all orders are stacked in order to obtain the full-order modal shape VV of the thin-walled component, which is a 3n x 3n matrix, and there are 3n modes, each mode has 3n degrees of freedom corresponding to the deformation condition:

[0041]

[0042] where l = 3n, W ij is the element in the i-th row and j-th column of the full-order modal shape VV;

[0043] The full-order modal shape VV is modal truncated to retain a modal, and then a reduced-order matrix Model is constructed, which has a dimension of 3n x a:

[0044]

[0045] The reduced-order matrix Model is cross-multiplied with the observation matrix C to obtain a conversion matrix T, and the conversion matrix T is shown in the following formula:

[0046]

[0047] Therefore, the conversion matrix T is an m x a matrix;

[0048] The obtained reduced-order matrix Model and conversion matrix T are saved in Unity3D, so as to be called and calculated in the subsequent process of real-time reconstruction of the surface shape;

[0049] The z-axis direction displacement of each measured point can be obtained by using a binocular vision measurement system, and saved in a displacement matrix D with a dimension of m x 1;

[0050]

[0051] where D i is the i-th element in the displacement matrix, representing the displacement size of the i-th measured point in the z-axis direction;

[0052] The data in the displacement matrix D can be used to obtain the displacement of the n nodes of the thin-walled structure in the x, y, and z directions, which is saved in a Dis matrix with a dimension of 3n x 1, and the surface shape reconstruction equation of the thin-walled structure is obtained;

[0053]

[0054] where Dis i represents the i-th element in the matrix Dis;

[0055] According to the geometric parameters of the thin-walled structure and the finite element discretization result of the corresponding thin-walled structure, an initial model of the thin-walled structure is established, and the nodes of the finite element discretization of the thin-walled structure are segmented to obtain a finite element model, which is stored in Unity3D software and placed in the corresponding position. The initial coordinates of each node in the finite element model are obtained by using the built-in function of Unity3D, and an initial position matrix D0 is constructed.

[0056]

[0057] wherein, X i , Y i , Z i the meaning of the i-th node in the x direction, y direction and z direction of the initial position; due to the existence of n nodes and the existence of x direction, y direction and z direction, the dimension of the initial position matrix D0 is n x 3;

[0058] Get servo motor and experimental bench model, and place according to the design position in Unity3D; Wherein, according to the contact point of servo motor and thin-walled component to place servo motor to ensure the accuracy in the monitoring process;

[0059] After completing the preparation work and fixing the measured plate, start the binocular vision measurement system and servo motor control system;

[0060] Secondly, the acquisition of discrete data related to thin-walled components and servo motors;

[0061] Set the required UDP communication protocol related settings in Unity3D, binocular vision measurement system and servo motor control system;

[0062] In Unity3D software, create A0 port for receiving signals from binocular vision measurement system and B0 port for receiving signals from servo motor control system;

[0063] Create A1 port in binocular vision measurement system for signal transmission, so that the signal is sent to A0 port in Unity3D software; Create B1 port in servo motor control system for signal transmission, so that the signal is sent to B0 port in Unity3D software;

[0064] Since UDP protocol provides a connectionless communication method, that is, there is no need to establish a specific link between devices, only the address domain network of data sending and receiving between devices needs to be considered; Therefore, only need to place Unity3D software, binocular vision measurement system and servo motor control system in the same domain network, and ensure that all systems are connected to the same domain network to establish effective communication and complete the sending settings;

[0065] Using binocular vision measurement system to measure the displacement of m markers, the displacement of the markers in x direction, y direction and z direction can be obtained. However, the displacement of the markers in x direction and y direction is relatively small compared to the displacement in z direction, which has little effect on the final result, so it is not considered. Therefore, the displacement matrix D of the markers in z direction is constructed as a matrix of m x 1:

[0066]

[0067] The numerical value of the real-time displacement matrix D is sent to the corresponding receiving port A0 of the Unity3D software in the form of ASCII code value using the pre-adjusted UDP protocol, with semicolons as the separator;

[0068] The real-time displacement of the end of the servo motor is obtained through the servo motor control system. The servo motor control system is essentially the use of the Simulink module in MATLAB software in combination with the Speedgoat real-time target machine to complete the control of the servo motor driver and encoder, while obtaining the servo motor shaft rotation data, and sending the data to the corresponding receiving port B0 of the Unity3D software in the form of ASCII code value through the UDP protocol;

[0069] Thirdly, the surface shape real-time reconstruction and monitoring of the thin-walled component is realized in Unity3D;

[0070] The ASCII code value of the displacement matrix D value of the measured point sent by the binocular vision measurement system is accepted in the corresponding port of Unity3D, and the ASCII code value is converted into normal value and symbol, and the displacement matrix D is reconstructed according to the semicolon as the separator;

[0071] The displacement of the thin-walled component in the x direction, y direction and z direction at the corresponding points of the finite element model divided into n nodes is calculated, and the formula is as follows:

[0072]

[0073] Wherein, matrix Dis is the overall displacement matrix of the x direction, y direction and z direction displacement of the n nodes on the measured thin-walled component; the matrix is reconstructed to ensure the additivity of the matrix, and then a new overall displacement matrix Dis' is obtained:

[0074]

[0075] Wherein, if Dis' is used to represent the element in the i-th row and j-th column of the reconstructed Dis' matrix, it satisfies the following relationship with the element in the Dis matrix before resetting ij

[0076] Dis' ij =Dis 3i-3+j

[0077] At the same time, when the thin-walled component finite element model has been stored into Unity3D in advance, the initial position matrix D0 of the thin-walled component has been read in, and the overall displacement matrix Dis is added to the original coordinates, so that the new real-time node coordinate matrix D1 is obtained: ​

[0078] D1=D0+Dis' (14)

[0079] Using this real-time node coordinate matrix D1, the surface reconstruction of the thin-walled component in Unity3D can be completed;

[0080] And real-time monitoring can be carried out, that is, the repeated measurement process, and real-time display in Unity3D can be realized;

[0081] In addition, further, by the corresponding node displacement obtained, the surface shape change, stress and strain of the surface shape can be calculated as needed, and the real-time reconstructed surface shape of the thin-walled component in Unity3D can be colored as needed, so that the detection personnel can more intuitively and obviously observe the detected thin-walled component;

[0082] Meanwhile, further, the ASCII code value of the real-time servo motor shaft rotation data sent by the servo motor control system is accepted in the corresponding port of Unity3D, data extraction is carried out, real-time servo motor shaft rotation data is reconstructed, and the real-time servo motor shaft rotation data is converted into the real-time movement of the electric push cylinder end driven by the servo motor through a multiple relationship, and is added to the motor movement in Unity3D, so that the real-time monitoring of the servo motor movement state in Unity3D is completed;

[0083] Further, the detection personnel can also use the servo motor control system to drive the motor and other power elements to regulate and control the surface shape of the thin-walled component.

[0084] The beneficial effects of the present application are: compared with the current mainstream equipment manufacturing process using artificial correction method, the present application can effectively improve the measurement speed of the thin-walled component deformation, make the deformation of the thin-walled component visualized and intuitive, and get rid of the dependence on human experience, which is beneficial to realize the in-situ monitoring and real-time adjustment of the thin-walled component, and can effectively improve the speed and accuracy of the shape correction of the thin-walled component. BRIEF DESCRIPTION OF DRAWINGS

[0085] Figure 1 It is a flowchart of the thin-walled component surface shape real-time monitoring and correction method based on binocular measurement.

[0086] Figure 2 It is a case device schematic diagram of the thin-walled component surface shape real-time monitoring and correction method based on binocular measurement. Wherein, 1 is a left camera, 2 is a right camera, 3 is a measured thin-walled component, 4 is a servo motor, 5 is a surface shape monitoring and correction system, 6 is a servo motor control system, 7 is an experimental bench, and 8 is a marker point. DETAILED DESCRIPTION

[0087] The specific embodiments of the present application are described in detail below with reference to the technical solutions and the accompanying drawings.

[0088] Figure 2 is a case device schematic diagram of the thin-walled component surface shape real-time monitoring and shape correction method based on binocular measurement.

[0089] The thin-walled component 3 used in the experiment has a thickness of 2 mm, a length x width of 460 mm x 460 mm, is composed of CYCOM977-2 epoxy resin matrix and Toshiba T300 carbon fiber, and 0° / 90° carbon fiber / epoxy resin layers are laid on the upper and lower surfaces and are fixed on the experimental bench 7 by a clamp.

[0090] In order to obtain the position information of the mark point 8 on the thin-walled component 3, in the experiment, a binocular vision measurement system composed of a left camera 1 and a right camera 2 is fixed to the top end of the experimental bench 7, both the left and right cameras are ME2P-1230-23U3C model cameras of Daheng image, and the position information of the mark point 8 is tracked through corresponding software and programs. In order to complete the shape correction of the thin-walled component 3, the servo motor 4 is fixed on the platform of the experimental bench 7, the servo motor control system 6 composed of a Speedgoat real-time target machine, a driver and an encoder of the servo motor 4 and a matching Simulink code is used to control the servo motor 4 and obtain the rotation data of the motor shaft, wherein the servo motor 4 adopts a Panasonic MHMF022L1V2M servo motor, and the driver of the servo motor 4 adopts a Panasonic MADLT15BF servo driver.

[0091] Attached Figure 1 is a flowchart of the thin-walled component surface shape real-time monitoring and shape correction method based on binocular measurement. The specific steps of the method are as follows:

[0092] Firstly, the component reconstruction equation is combined with the stiffness matrix and the mass matrix;

[0093] The stiffness matrix K and the mass matrix M of the thin-walled component 3 and the observation matrix C corresponding to the mark point 8 are obtained by combining the finite element analysis software, the formula (5) is obtained by solving and reconstructing the eigenvalues of the stiffness matrix K and the mass matrix M, the formula (6) is obtained by modal truncation processing, then the conversion matrix T is calculated according to the formula (7), and finally the final reconstruction equation, i.e. the formula (9), can be constructed by combining the subsequently measured displacement matrix D.

[0094] The reconstruction equation, the initial model of the thin-walled component 3 and the models of the servo motor and the experimental bench are stored in Unity3D according to the design and placed in the designated position. After completing the preparation work, the binocular vision system and the servo motor control system are started after the to-be-measured plate is fixed.

[0095] Secondly, the acquisition of the discrete data related to the thin-walled component and the servo motor;

[0096] The related content of the UDP protocol link is configured, the related sending and receiving ports are created in the Unity 3D software, the binocular vision measurement system and the servo motor control system, and the system is connected to the same domain network, so that effective communication can be established, and the sending setting is completed.

[0097] The displacement of the mark point 8 is measured by using the binocular vision measurement system, and the displacement matrix D is constructed; the rotation data of the motor shaft are obtained by using the servo motor control system; and the two are sent to the corresponding port of the Unity 3D software in the form of ASCII code values through the UDP protocol.

[0098] Thirdly, the surface reconstruction and detection of the thin-walled component are realized;

[0099] The ASCII code values sent by the binocular vision measurement system and obtained by the servo motor control system are received in the corresponding port of the Unity 3D, and data extraction is performed on the ASCII code values, so that the displacement matrix D and the rotation data of the motor shaft are reconstructed.

[0100] The new overall displacement matrix Dis' is obtained in the Unity 3D software by combining formula (12), the real-time node coordinate matrix D1 is obtained by formula (14), the real-time surface reconstruction of the thin-walled component is completed by using the real-time node coordinate matrix D1, the surface change, the stress and the strain of the surface are calculated according to the matrix, the surface of the thin-walled component is colored in the real-time reconstruction of the Unity 3D, so that the detection personnel can more intuitively and obviously observe the condition of the thin-walled component; according to the motor rotation shaft data, the real-time movement of the end of the electric push cylinder driven by the servo motor can be calculated, and the real-time monitoring of the motor movement state in the Unity 3D is completed.

[0101] The detection personnel can control the servo motor 4 to adjust and control the surface of the measured thin-walled component 3 through the servo motor control system 6 according to the surface display of the composite plate, so as to achieve the ideal surface state, improve the assembly precision and strength of the thin-walled component in the later period, and effectively reduce the out-of-tolerance situation.

[0102] The following is the comparison of three experiments, the deformation of a random point (not the mark point used in the method) on the thin-walled component is measured by using the method of the application and directly using the binocular vision measurement method, a total of 16 points, the displacement unit is millimeter.

[0103]

[0104]

[0105] From the comparison of the above table, it can be concluded that the method used in the application has sufficient accuracy and can accurately and quickly monitor the surface shape change of the thin-walled component.

[0106] The application solves the problem that it is difficult to detect and control the surface shape of the thin-walled component in real time during production and assembly, and compared with the existing research method, the surface shape of the thin-walled component is detected and controlled in real time, the over-difference problem in production and manufacturing can be effectively reduced, and the application has great potential in industrial production.

Claims

1. A method for real-time monitoring and shape correction of thin-walled components based on binocular vision measurement, characterized in that, The specific steps are as follows: First, the stiffness matrix and the mass matrix are combined to construct the surface shape reconstruction equation; According to the need, mark points are pasted on the surface of the thin-walled component, the mark points are used as measured points, and the measurement is performed through a binocular vision measurement system; and the contact points of the servo motor and the thin-walled component are determined, and the contact points are used as controlled points of the thin-walled component; At the same time, the thin-walled component is discretized according to the need, and the overall stiffness matrix and the mass matrix of the thin-walled component are obtained by combining finite element analysis; The stiffness matrix and the mass matrix are subjected to eigenvalue decomposition to obtain full-order modal shapes, and then a corresponding relationship between the displacement of the mark points in space and the surface shape of the thin-walled component is constructed according to a modal truncation model reduction method, so as to obtain a surface shape reconstruction equation of the thin-walled component; An initial model of the thin-walled component is established, and node segmentation is performed on the finite element discretization of the thin-walled component to obtain a finite element model of the thin-walled component; The related equations, models and parameters obtained in the above process are stored in Unity3D software for later calling; Second, the related discrete data of the thin-walled component and the servo motor are obtained; The required UDP communication protocol related settings in Unity3D, the binocular vision measurement system and the servo motor control system are set; The related discrete data of the thin-walled component and the servo motor are sent to the Unity3D software in real time through the UDP communication protocol; The x, y and z direction displacements of the measured points in space are obtained through three-dimensional reconstruction of the two-dimensional images captured by the binocular camera; The servo motor displacement information is obtained through the servo motor control system; The related discrete data of the thin-walled component and the servo motor are sent to the Unity3D software in real time through the UDP communication protocol; Third, the surface shape of the thin-walled component is reconstructed and detected in real time in Unity3D; The displacement data of the measured points in space obtained by the binocular vision measurement system and the servo motor displacement information obtained by the servo motor control system are received in the Unity3D software; The obtained information is decoded and analyzed to construct a calculation matrix that conforms to the calculation rule; The surface shape of the thin-walled component is reconstructed in real time in Unity3D by combining the related equations and parameters stored previously and completing the calculation; At the same time, the related deformation parameters of the thin-walled component are provided to the detector to realize accurate monitoring and regulation of the surface shape of the thin-walled component.

2. The method of real-time monitoring and correction of the surface shape of thin-walled components based on binocular vision measurement according to claim 1, characterized in that, First, the stiffness matrix and the mass matrix are combined to construct the surface shape reconstruction equation; Mark points are pasted on the surface of the thin-walled component according to the need, the mark points are used as measured points, and the measurement is performed through a binocular vision measurement system; and the contact points of the servo motor and the thin-walled component are determined, and the contact points are used as controlled points of the thin-walled component; The thin-walled component is discretized by finite elements, and n nodes are obtained, and only three displacement degrees of freedom in x, y and z axis directions are used when the thin-walled component is discretized by finite elements, and the rotation degrees of freedom in x, y and z axis directions are ignored; Combining the results of the finite element discretization of the thin-walled component, the stiffness matrix K and the mass matrix M of the thin-walled component are obtained through finite element analysis, at this time the dimensions of the stiffness matrix K and the mass matrix M are both 3n x 3n: where l = 3n, K ij is the element of the stiffness matrix K in the i-th row and j-th column, M ij is the element of the mass matrix M in the i-th row and j-th column; After pasting m markers, according to the positions of the pasted markers, the observation matrix C is constructed in combination with the finite element discretization of the thin-walled component, the observation matrix C is the position of the m markers in the reference system of the n nodes divided by the thin-walled component, so the dimension of the observation matrix C is m x 3n: where l = 3n, C ij If the node represented by the ith landmark point is the kth node in the n nodes, and the displacement of the node represented by the landmark point in the y-axis direction is used to construct the surface shape, then the element in the ith row and jth column of the observation matrix C is i(k+n) The value of 1, for the elements of the ith row of the observation matrix C, except for C i(k+n) The values of the remaining elements are 0. The stiffness matrix K and the mass matrix M are subjected to eigenvalue decomposition to obtain the following formula: [vv i ,DD i ] = edg[K,M] (4) where vv i is the eigenvector, DD i is the eigenvalue, K is the stiffness matrix, and M is the mass matrix. The corresponding i-th order eigenvector vv i The obtained eigenvectors are the corresponding modal shape vectors, and the modal shape vectors vv i According to the order of each order, superposition can be obtained The full order modal shape VV of the thin-walled component is a 3n×3n matrix, and there are 3n modes, each mode has 3n degrees of freedom corresponding to the deformation condition: where l = 3n, W ij is the element of the full modal shape matrix VV in the i-th row and j-th column; The full-order modal shape VV is subjected to modal truncation processing, and a is retained, then the reduced-order matrix Model is constructed, and the dimension is 3n x a: The reduced-order matrix Model and the observation matrix C are subjected to cross multiplication processing to obtain the conversion matrix T, and the conversion matrix T is shown in the following formula: Therefore, the conversion matrix T is an m x a matrix; The obtained reduced-order matrix Model and conversion matrix T are saved in Unity3D for subsequent real-time surface shape reconstruction process; The z-axis direction displacement of each measured point can be obtained by using the binocular vision measurement system, and saved in the displacement matrix D with the dimension of m x 1; where D i is the i-th element of the displacement matrix, representing the displacement of the i-th measured point in the z-axis direction; The x, y and z direction displacements of the n nodes of the thin-walled component can be obtained by using the data in the displacement matrix D, and saved in the Dis matrix with the dimension of 3n x 1, that is, the surface shape reconstruction equation of the thin-walled component is obtained; where Dis i denotes the i-th element of the matrix Dis; According to the geometric parameters of the thin-walled component and the finite element discretization results of the corresponding thin-walled component, the initial model of the thin-walled component is established, the node segmentation of the finite element discretization of the corresponding thin-walled component is carried out, the finite element model is obtained, and is stored in Unity3D software and placed in the corresponding position, the initial coordinates of each node in the finite element model are obtained by using the built-in function of Unity3D, and the initial position matrix D0 is constructed: wherein X i , Y i , and Z i have the meaning of the initial position of the i-th node in the x-direction, y-direction, and z-direction, respectively; since there are n nodes and there are x-direction, y-direction, and z-direction, the dimension of the initial position matrix D0 is n x 3; The servo motor and the experimental bench model are obtained, and are placed according to the design position in Unity3D; wherein, the servo motor needs to be placed according to the contact point of the servo motor and the thin-walled component to ensure the accuracy in the monitoring process; After completing the preparation work and fixing the measured plate, start the binocular vision measurement system and the servo motor control system.

3. The method of real-time monitoring and correction of the surface shape of thin-walled components based on binocular vision measurement according to claim 1, characterized in that, Second step, obtaining the related discrete data of the thin-walled component and the servo motor; Set the related settings of the UDP communication protocol in Unity3D, binocular vision measurement system and servo motor control system; In Unity3D software, create A0 port for receiving signals from binocular vision measurement system and B0 port for receiving signals from servo motor control system; In binocular vision measurement system, create A1 port for sending signals, and send the signals to A0 port in Unity3D software; in servo motor control system, create B1 port for sending signals, and send the signals to B0 port in Unity3D software; Because UDP protocol provides a connectionless communication method, that is, without establishing a specific link between devices, only need to consider whether the address of the data sending and receiving between devices is in the same domain network; therefore, only need to place Unity3D software, binocular vision measurement system and servo motor control system in the same domain network, and ensure that all systems are connected to the same domain network to establish effective communication and complete the sending settings; The displacement of the m markers in the x, y and z directions can be obtained by using the binocular vision measurement system, but the displacement of the markers in the x and y directions is much smaller than that in the z direction, which has little effect on the final result, so it is not considered. Therefore, the displacement matrix D of the markers in the z direction is constructed as an m×1 matrix: The real-time displacement matrix D is sent to the corresponding receiving port A0 of the Unity3D software in the form of ASCII code value using the pre-adjusted UDP protocol with semicolon as the separator; The real-time displacement of the servo motor end is obtained through the servo motor control system. The servo motor control system is essentially using the Simulink module in MATLAB software to complete the control of the servo motor driver and encoder with Speedgoat real-time target machine, and at the same time, the servo motor shaft rotation data is obtained and sent to the corresponding receiving port B0 of the Unity3D software in the form of ASCII code value through the UDP protocol.

4. The method of real-time monitoring and correction of the surface shape of thin-walled components based on binocular vision measurement according to claim 1, characterized in that, Thirdly, the surface shape real-time reconstruction and monitoring of the thin-walled component is realized in Unity3D; The ASCII code value of the displacement matrix D value of the measured point sent by the binocular vision measurement system is accepted in the corresponding port of Unity3D, and the ASCII code value is converted into normal value and symbol, and the displacement matrix D is reconstructed according to the semicolon separator; The displacement of the thin-walled component in the x, y and z directions of the corresponding points of the finite element model divided into n nodes is calculated, and the formula is as follows: Where matrix Dis is the overall displacement matrix of the x, y and z directions of the n nodes on the measured thin-walled component; the matrix is reconstructed to ensure the additivity of the matrix, and a new overall displacement matrix Dis' is obtained: where, if Dis' is used ij where, if Dis' is used where, if Dis' is used Dis' ij = Dis 3i-3+j At the same time, the initial position matrix D0 of the thin-walled component finite element model has been stored in Unity3D, and the overall displacement matrix Dis is added to the original coordinates to obtain a new real-time node coordinate matrix D1: D1=D0+Dis' (14) The real-time node coordinate matrix D1 can be used to complete the surface reconstruction of the thin-walled component in Unity3D; And real-time monitoring can be performed, that is, the measurement process can be repeated, and real-time display can be realized in Unity3D.

5. The method of real-time monitoring and correction of the surface shape of thin-walled components based on binocular vision measurement according to claim 1, characterized in that, The face shape change, stress and strain of the thin-walled component can be calculated according to the obtained node displacement, and the real-time reconstructed face shape of the thin-walled component in Unity3D can be colored according to the need, so that the detection personnel can more intuitively and obviously observe the detected thin-walled component.

6. The method of real-time monitoring and correction of the surface shape of thin-walled components based on binocular vision measurement according to claim 1, characterized in that, The ASCII code value of the real-time servo motor shaft rotation data sent by the servo motor control system can be accepted in the corresponding port of Unity3D, the data can be extracted, the real-time servo motor shaft rotation data can be reconstructed, the real-time servo motor shaft rotation data can be converted into the real-time movement of the end of the electric push cylinder driven by the servo motor through a multiple relationship, and the real-time movement of the electric motor in Unity3D can be added, so that the real-time monitoring of the servo motor movement state in Unity3D is completed.

7. The method of real-time monitoring and correction of the surface shape of thin-walled components based on binocular vision measurement according to claim 1, characterized in that, The detection personnel can also use the servo motor control system to drive the power elements such as motors to regulate and control the face shape of the thin-walled component.

Images